I think this makes for an interesting cautionary tale: when removing features, no matter how little-used they may appear to be, there's always the possibility that it will break something very important and useful that depends on them. The path to the solution also shows how this attempt to remove complexity actually resulted in even more complexity later on, in the form of diverging virtualisation extensions and non-uniform support for segment limits.
Segmentation isn't the only one, however; many of the existing seldom-used one-byte instructions also inexplicably went missing (became completely invalid - they didn't reuse them for prefixes or anything like that), and among them were SAHF/LAHF which also turned out to be important for virtualisation. In this case the solution was that both AMD and Intel put them back sometime later, and had to add an extra "feature bit" to indicate this. It's quite absurd considering that these instructions were present in 32-bit mode, and leaving them unchanged in 64-bit mode from the beginning would've avoided this issue completely.
As successful as x86-64 is, I definitely think the 64-bit transition could've been much better, similar to how the 32-bit extensions that came with the 386 fitted nicely into the existing instruction set and could even be used from 16-bit mode. In contrast, there is no way to use 64-bit registers in 32-bit code despite there being reservations in the existing instruction set that would've made it possible. V86-mode, which has been present since the 386, could've been extended in a relatively straightforward manner to make virtualisation easier.
Backwards compatibility has to be used with care: too much of it and your new product will never have software of its own and will only be a newer version of your old product. The Commodore 128 is a good example of this.
> In contrast, there is no way to use 64-bit registers in 32-bit code despite there being reservations in the existing instruction set that would've made it possible.
No, x32 is still long mode (64-bit), it's just an ABI that restricts all memory allocations to 4 gigabytes of address space so that 4-byte pointers can safely be used instead of 8-byte ones. You can get a similar effect by manually mmaping memory into the lower parts of the address space even in a regular 64-bit program (luajit relies on this), but you can't stop malloc, the kernel, other libraries, etc. from using the rest of the address space.
On x86 CPUs since the the 386, you don't need a hack to make 32-bit mode look like a 16-bit mode, you literally can access the full 32-bit registers in 16-bit modes with an operand size prefix.
Windows x64 used to emulate SAHF/LAHF and 3DNow! PREFETCH. This was eliminated in x64 Win8.1, ending support for all 90nm Prescott etc CPUs (the newer 65nm Cedar Mill etc are still supported).
Segmentation isn't the only one, however; many of the existing seldom-used one-byte instructions also inexplicably went missing (became completely invalid - they didn't reuse them for prefixes or anything like that), and among them were SAHF/LAHF which also turned out to be important for virtualisation. In this case the solution was that both AMD and Intel put them back sometime later, and had to add an extra "feature bit" to indicate this. It's quite absurd considering that these instructions were present in 32-bit mode, and leaving them unchanged in 64-bit mode from the beginning would've avoided this issue completely.
As successful as x86-64 is, I definitely think the 64-bit transition could've been much better, similar to how the 32-bit extensions that came with the 386 fitted nicely into the existing instruction set and could even be used from 16-bit mode. In contrast, there is no way to use 64-bit registers in 32-bit code despite there being reservations in the existing instruction set that would've made it possible. V86-mode, which has been present since the 386, could've been extended in a relatively straightforward manner to make virtualisation easier.