Even on my relatively beefy comp, though, the miku animation was enough to get my fan going full blast. Papervision otoh has the potential to build a real, high-quality, 3d game. I don't see JS getting there any time soon, but nevertheless an interesting proof of concept.
I don't see any "hard barriers" but a lot of soft issues that will take time to overcome. As of now, there isn't even a uniform event handling framework in JS. So, first, we'd need a good generic 3d framework. Next, we'd need a significant speedup in performance across the major browsers. If OpenGL is a model, there would probably have to be some graphics-specific optimizations made, which might be difficult to get everyone on board for. Third, I'd either need to buy a new computer or JS would need gpu access, because ATM this site brought my relatively face computer to a grinding halt... and I don't think I'm going to have money for a new comp for at least a year or two :).
You could tell people "download Firefox to use my spiffy 3D site" (or download Chrome or whatever) just like people now tell them "download Flash Viewer 9". Does Papervision in Flash take advantage of 3-D acceleration hardware? In those ways I'm not sure there's so much of a difference between JavaScript in browsers and ActionScript in Flash.
Graphics-specific optimizations are probably going to happen, if by that you mean "fixing Canvas so it isn't painfully slow".
Even on my relatively beefy comp, though, the miku animation was enough to get my fan going full blast. Papervision otoh has the potential to build a real, high-quality, 3d game. I don't see JS getting there any time soon, but nevertheless an interesting proof of concept.