It's not so much about removing the burden or making life easier, but instead making manual optimization more straightforward and predictable. Currently, manual optimization often means appeasing or even fighting the compiler's optimizer passes, with small innocent changes completely changing the output of the compiler.

Now that I think about exposing cache hierarchy in a programming language, that makes much more sense. One can imagine a programming language with explicit control over CPU caches and their hierarchy. Also, this could make GPU/CUDA programming more explicit, safe and efficient.

Still, this requires the hardware to cooperate with software instead of pretending that random access memory is truly random access. This functionality just isn't there at this point.

Edit: this would require programmable caching hardware to make caching analysis and algorithms introspectable and changeable. For now, fixed caching algorithms implemented in hardware do provide lots of benefits.

DSPs and other application-specific processors expose the cache hierarchy as a set of scratchpads. This works very well for them, but not for any CPU that is shared between applications, like a server.

What you're describing is reminding me of how Itanium tried to bring VLIW to non-embedded spaces and found "make the compiler smart enough to use this well" to be much easier said than done.

I do wonder if the long-term future of programming looks like https://www.microsoft.com/en-us/research/publication/coq-wor... , and providing libraries to help the programmer optimize themselves rather than black boxes that do it for them.