I guess with that loose of a definition, Android could be a walled garden. But once you get past the broad strokes, the walled garden of "we have a kill switch and could potentially use it for anything, but so far only for security purposes" and the walled garden of "we approve everything unless you exploit a security hole and have blocks apps just because it was in our interest" are very different.
There is always a balance of security and freedom. To this point, I'd say Google has erred on the side of freedom. And at least with Android, if Google were to go rogue in a doomsday scenario, we could always fork it.
The most important point about Android is that apps can be installed from third-party sources just by toggling a switch in the settings on the stock firmware.
While Google probably technically could use the remote uninstall function on an app not installed via their Market, it seems extraordinarily unlikely that they would. And as you suggest, there already exists CyanogenMod and other third-party firmwares; it's fairly easy to strip Android of any connection to Google if you so desire.
There is always a balance of security and freedom. To this point, I'd say Google has erred on the side of freedom. And at least with Android, if Google were to go rogue in a doomsday scenario, we could always fork it.