"I have no good suggestion how to escape from that situation"
Everyone knows what the solution is; a sandboxable arch-independent batteries-included bytecode format for the web. The solution is obvious, the problem is in organising the detailed work necessary for such a format to emerge.
No, the technical solution is the hard part. Designing a bytecode format for the web would not be a trivial undertaking.
Assuming this work was completed, widespread adoption is mostly a matter of browser support and time. I suspect the Chrome team would be open to implementing it after it's been designed (there are parallels with the Native Client work), and other browser teams would be unlikely to resist if there are clear performance and workflow benefits (which there undoubtedly would be). Basically it's a "if you build it, they will come" situation.
To understand the scale of the task, consider the 'batteries-included' aspect. The core libraries for the bytecode format should be standardised (i.e. core library API set in stone for each major version of the bytecode). That's a huge amount of work. Then you'd also have the security testing needed (to ensure the bytecode was securely sandboxed). You'd also need to work on ensuring the bytecode was efficiently implemented for all processor architectures. I don't know about you, but I certainly don't feel qualified to do that.
No, it wasn't. It wasn't sandboxed and the libraries were not restricted enough to ensure a reliable target. Java applets were essentially running on the desktop, not in a website.
Everyone knows what the solution is; a sandboxable arch-independent batteries-included bytecode format for the web. The solution is obvious, the problem is in organising the detailed work necessary for such a format to emerge.