That's true, but still, a single individual or small group living isolated in the real world will, over a lifetime, learn only a tiny fraction of what we can learn from the written knowledge accumulated over millennia.

Having said that, I tend to agree that having AI interact with the world may be key: for one thing, I'm not sure whether there is any sense in which LLMs understand that most of the information content of language is about an external world.