You jest but I agree. Also I think the "stochastic" arguments is getting old. What if XML was stochastic? Does it matter if it is "stochastic" or does it matter if it is correct?
You know my compiler generates a different binary every time I compile the exact same code. My CPU definitely is not fully deterministic yet it makes a nice show of it being so. I don't care and nobody cares as long as it works. And what "works" means exactly is quite a bit more involved than parroting "determinism".
I don't know what compiler you're using but our C/C++ builds are definitely deterministic given the same source files. We have CI tests to ensure this remains the case.
The argument is getting old in the sense that it was first used longer and longer ago.
However, it's still just as applicable as ever. Perhaps more.
> Does it matter if it is "stochastic" or does it matter if it is correct?
In this case, we can only determine whether it's correct after it's too late to do anything about it. So if it was correct, we can say it didn't matter, but only in retrospect.
Despite the well-established mental concept of doing things correctly the first time, mistakes continue to be made. Perhaps by people unaware of the mental concept.
I don't know what "cop on" means.
I've been feeling a bit despondent about the situation as well.
It's a tricky problem for sure. Even on CPUs this separation is maintained by architectural guardrails. The CPU will happily execute whatever it is permitted to fetch. There is and cannot be a fundamental divide betwixt the two. It's always going to be an artificial externally managed issue. I suppose this is no different for LLMs.
My thinking is we are in the 50s/60s. Stuff is starting to come forward, it's all very exciting but very, very raw. I don't think this will last.
The notions of "tokens" and how inference works will become arcane insider knowledge like how CPU registers and interrupts work. You don't work with CPUs, you work with "computers" and even then mostly "operating systems" or even "browsers". Reality has been abstracted away from you to a very impressive degree. I don't think it'll be different here, but we haven't had our Xerox PARC and Bell Labs moments yet.
They are moving at breakneck speed deploying on scales most of us can't even imagine. They are working in a space that's completely unexplored where getting information as quickly as possible is preferred above iterating on some feature until it's "done" while your competitor has released fifteen other features, all sucky, but one of which turns out to be a killer and makes a billion bucks overnight.
> They are moving at breakneck speed deploying on scales most of us can't even imagine. They are working in a space that's completely unexplored where getting information as quickly as possible is preferred above iterating on some feature until it's "done" while your competitor has released fifteen other features, all sucky, but one of which turns out to be a killer and makes a billion bucks overnight.
Not on the client, they're not. It's a chatbot as glue between different backend systems.
You know my compiler generates a different binary every time I compile the exact same code. My CPU definitely is not fully deterministic yet it makes a nice show of it being so. I don't care and nobody cares as long as it works. And what "works" means exactly is quite a bit more involved than parroting "determinism".