When the first C++ Standard (C89) was being created, about half the compilers implemented "sign preserving" semantics, which is what you're advocating, and the other half implemented "value preserving" semantics.
A great battle ensued, and many champions were slain.
The value preserving folks carried the field, and the sign preserving folks changed their compilers.