> the baseline of open models running on cheap third-party inference providers, or even on-prem. This is a bit of a challenge for the big proprietary firms.

It’s not a challenge at all.

To win, all you need is to starve your competitors of RAM.

RAM is the lifeblood of AI, without RAM, AI doesn’t work.

Assuming high bandwidth flash works out, RAM requirements should be drastically reduced as you'd keep the weights in much higher capacity flash.

> Sample HBF modules are expected in the second half of 2026, with the first AI inference hardware integrating the tech anticipated in early 2027.

https://www.tomshardware.com/tech-industry/sandisk-and-sk-hy...

How does HBF compare to the discontinued 3D XPoint?

HBF is NAND and integrated in-package like HBM. 3D XPoint or Optane would be extremely valuable today as part of the overall system architecture, but they were power-intensive enough that this particular use probably wouldn't be feasible.

(Though maybe it ends up being better if you're doing lots of random tiny 4k reads. It's hard to tell because the technology is discontinued as GP said, whereas NAND has kept progressing.)