There's always ways to control this stuff adequately by setting the properties of memory ranges and the like. All CPU's have to support some equivalent of "volatile" because that's what memory-mapped hardware expects.

Compilers will often delete code that has volatile stores, if it thinks that would be OK. Not everybody agrees with this policy. But they are not who makes decisions about what compilers do.

There "are always ways", but not everybody always knows what they are, or even knows they need to know them. And, what you think ought to suffice too often does not.

How can a compiler remove a volatile store? There exists hardware where a store has a side effect, regardless of the value itself. For example, writing to a FIFO. That’s the entire point of volatile, no?

You may think so, and I may even agree. But those opinions do not control what compilers actually implement.

For example, in

  void f() {
    volatile char sekrit[256];
    sekrit[0] = 0;
  }

a compiler may reason that, since sekrit is on the stack, and no hardware device could know where it is, it may elide the whole block. And, some do.

This is not true. memset() can be optimized away, memset_s() cannot be optimized away by the compiler or the linker.

but the usual memset_s implementation does not flush the caches, only my safeclib implementation does so, crypto libs do not care.

Right. In particular, a subsequent write to the same address may overwrite (part of) the cache line representing that bit of memory, pushing it to the back of the LRU line. This might happen over and over again, and that cache line then might not get flushed until the end of process's time slice. Anything out in memory backed by that cache line keeps its old value until the writeback finally happens.

Sure, but obviously OS developers know how to deal with this stuff.

Not only OS kernels need to keep secrets, and not only OS kernel coders need to know how.

Sometimes you have to put the zeroing code in its own .o file so the caller's compiler won't know what it does.