For me (Plus plan, iOS app), Astra only shows up under the Work tab.
I’ve been using Work for all my queries, since it seems to just be the same interface as Chat but with more features. I don’t understand why they’re two separate things.
The model they use in that paper is a toy model of an LLM that’s so different from an actual LLM that I doubt the results mean anything at all.
Specifically, they train a model from scratch. The model architecture is apparently based on Llama but the size is 34M parameters. Not 34B, 34M. This is a fraction of the size of GPT-2.
Luckily, they don’t use the model as a language model. It neither receives text, generates text, nor uses text to think. Instead the inputs are strings like “A01 B10 C73 D27”, and the only possible outputs are “True” and “False”. They are expecting the model to solve a specific math problem encoded by those numbers, and do nothing else. The chain of thought is also numbers, in the scenario that’s supposed to represent a real chain of thought (as opposed to the filler-token scenario and the no-CoT scenario). The numbers in question are manually trained into the model based on one possible algorithmic decomposition of the problem; the model does not learn to generate its own CoT.
Even with all those limitations, for their main problem (3SUM), they only show that filler tokens are better than no CoT at all. They don’t show how that compares to ‘real’ CoT, at least as far as I can see (admittedly I only skimmed). They do make this comparison for their easier problem (2SUM), but on that problem both filler token CoT and ‘real’ CoT are mostly saturated, so the results don’t mean much.
>The model they use in that paper is a toy model of an LLM that’s so different from an actual LLM that I doubt the results mean anything at all.
Sorry, but it sounds like you aren't cut out for transformer based LLM research if you can't interpret the results.
Each layer in a transformer has its own KV cache that feeds from the previous layers output. Those internal layer activations are not subject to training loss directly. The model can spit out a dot but still perform additional computations within those layers.
The dot effectively prevents information from flowing from the final layer into the output, but all the intermediate layer activations are still available for future tokens. The limitation here is that information cannot be passed from the last layer to the first layer.
If a long running calculation needs to go through all layers and then be refined further, CoT becomes mandatory because CoT tokens are the only way to pass information up to the first layer, but since they are not as information dense as the internal activations, replacing them with a dot does not degrade performance as much as one would expect.
I'm not even an LLM researcher. But I believe you're correct; I just don't think the paper provides any meaningful evidence on the subject. From the structure of the model we can tell that dots can do more computation than no CoT and less than full CoT. What we don't know is how much of the performance gap between no CoT and full CoT would be recovered by an LLM using dots. The paper answers that question for a tiny transformer, but it's so different from an LLM that extrapolating to LLM scale is purely speculative. In other words it doesn't give us more information then what we can already determine from the structure of the model.
For the record, the original claim was that "you can replace intermediate tokens with single character chains and still get the increased precision", which I read as saying you can get almost all of the benefit of CoT with dots, not just some unknown fraction of it. This claim is true for the paper, though probably only because the benchmark was saturated (and training difficulty was also higher, as they mention). But the claim is unlikely to be true for LLMs, if only because they presumably do sometimes need to perform "long running calculations", and they also have plenty of unsaturated benchmarks.
(The other part of the original claim, that CoT is "not a representation of a model’s logical path", is more complicated. It's clearly not guaranteed to match the model's logical path, as we have seen with other research into actual LLMs. But it does seem to be correlated with it, presumably more so for more complex / long-running tasks.)
Scientific papers aren't popsci articles. Constraining the environment (like with the token choice) is a basic method to reduce confounding factors...
The "toy model" part is flat out wrong, the models they use have _exactly_ fundamental mechanics then chatGPT or claude, they are used all the time for this kind of research for a reason.
> The "toy model" part is flat out wrong, the models they use have _exactly_ fundamental mechanics then chatGPT or claude, they are used all the time for this kind of research for a reason.
Tiny models are sometimes used out of necessity, but the results often don't hold up at scale, or in different scenarios.
If you train a tiny transformer to use a fraction of its capacity to perform an algorithm simple enough that you could design the transformer weights by hand, the results might generalize to natural-language reasoning, but they easily might just not.
So it is hard to generalize even to something like GPT-3. But once you get to a modern LLM, the nature of the chain of thought is also very different.
First of all, the CoT can be very long and encode reasoning with long chains of serial dependencies. Filler tokens don’t increase the limit on serial computation, particularly in the dots example where the output is a single token (as opposed to real language model which can somewhat substitute for missing CoT by reasoning within the answer itself).
Also, a modern LLM gains much of its reasoning capability in post-training, where it learns which CoT tokens to emit rather than just relying on pre-existing chains of thought in its training data. The post-training involves reinforcement learning where the grader ignores the CoT and only grades the final result. The only things tying the CoT to natural language are (1) the starting point (pre-training data) containing natural-language CoT, and (2) the CoT being produced using the same weights as the final output. On one hand this could potentially make the CoT less faithful to the model's reasoning, since nothing directly punishes the model for, say, coming up with its own private meanings for words within the CoT. But on the other hand, this incentivizes the model to make good use of the CoT. To the extent that the CoT is a "parallel construction" (i.e. a chain of reasoning that has to be coherent-looking but differs from how the model is actually coming up with the answer), coming up with this duplicate reasoning is both a waste of computation and a big opportunity cost. The model should learn to instead use the CoT tokens for information that will be useful to it later. But in practice models do keep producing readable CoT, suggesting that the CoT does reflect the real thinking process to a significant extent (probably more so after post-training than before).
The dots paper has no post-training, so it doesn’t encounter this dynamic at all.
Too bad. I see the stereotypical prose as a good thing. When I interact with Claude myself, I don’t mind it as it just feels like Claude’s distinctive voice. But when other people try to disguise LLM output as their own thoughts, the voice makes it easier for me to tell.
People that want to be open about the source of their text will just tell you where it came from.
People that want to obscure the source of their text would rather that it was more difficult to sniff out LLM-generated text. And they're the ones picking which model to use.
Sure. But you're stuck with GTK on every platform. "Cross platform, so long as you want to use GTK everywhere" is much less exciting than using actual native cross-platform UI elements.
My experience is that even Opus 5 still tends to write buggy or low-quality code and makes serious mistakes when analyzing code. It's a lot better than before but still not something I trust. I've had less experience with Fable since I can't use it at work; I hear it's a step up but still has its limits.
For large tasks like a web browser or a compiler, even expensive swarms of frontier LLMs have not been shown capable of producing codebases that actually work. (Anthropic built a C compiler with Opus 4.6 but it lacked optimizations and apparently hit a complexity wall.)
I also want to use LLMs for reverse engineering, but apparently it's pretty hit-or-miss, especially if you're forced to use open-source models to avoid restrictions.
This reply is particularly interesting to me because most of my experience with actually using LLMs to get work done is with coding agents. But I only have a fairly narrow set of experiences: two pretty large solo Flutter projects. I am currently really pleased with Gemini as a coding agent. It could improve, but I think improvements are going to come from marginal gains in the harness and training material so it can catch things like misconfigured permissions in platform specific areas.
It's also interesting because, while coding agents are important and are a notable success, they are never going to be a multi trillion dollar business. And are there any other domains where LLMs have such a large impact?
One of the things that came out of the decoded reasoning paper was that Claude models had memorized answers to tests but hid this memorization from the user output and pretended to derive the answer properly. It's only possible to cheat so blatantly in closed models where the reasoning is hidden.
And didn't stop doing this until 2020! (Specifically Emacs 27.1, which replaced "unexec", which dumped the process image, with the "portable dumper", which despite the name does not.)
Yep. Even outside of tracing, there are several different ways that the Linux kernel patches itself:
- Static calls: like a call to a global function pointer, except instead of loading a function pointer and doing an indirect call, the code is patched to do a direct call to the destination
- Static keys: like an if statement testing a global boolean, except instead of loading a boolean and doing a conditional branch, the code is patched to do either an unconditional branch or a nop
- Runtime constants: like a load of a global variable, except instead of loading, the value is patched directly into the code
- Alternatives: selects one of multiple possible instruction sequences depending on (usually) whether the CPU supports specific instructions
It's really fascinating to see the kind of fun efficient stuff you can do when you have that level of low-level control. Not just code patching but things like RCU as well.
> However it is still interesting to write machine code snippets once and execute them many times, like the nested function trampolines.
I slightly disagree on this though. In my experience writing code with Clang blocks (which don't use trampolines), they're often useful for code organization even if the callback will only be called once. Therefore, even ignoring security issues, I think GCC choosing a design that required cache flushing was a mistake - certainly in retrospect (as cache flushing has become more expensive over the years), but perhaps even at the time. I did some research, and trampolines were introduced in GCC 2.0, which already included mprotect calls and/or cache flushes on some of the architectures it supported, such as MIPS. However, this was a relatively new development, and on most of the supported architectures it didn't do either of those things. But on MIPS it would do an mprotect every single time a trampoline was created, which can't have been fast.
It's not the cache flush that's expensive, it's loading the new instructions that aren't cached. If you are flushing the very next instruction and then immediately executing it, that's expensive because of the serial dependency, but if you're generating new code, flushing it shouldn't be more expensive than if you were simply accessing new code for the first time. But on old systems you could modify the very next instruction with no penalty because there wasn't a cache. You can still do that and probably faster than those old systems could (they were slower because of not having a cache, everything was an uncached access), it's just a waste of most of the new system's performance.
BTW you can do all of this cool stuff in user mode on Linux too (but not on OpenBSD) - you just have to opt in to executable stack and/or writable .text. I could have written the dynamic shift instruction generator I mentioned, but I didn't want to spend the effort, but I imagined having a language with actual support for something like that (like static keys for variables).
There's also a reason this stuff started getting adopted around 20 years ago (I'll add macOS to the pile: it added per-CPU malloc sharding in 2008). It's not just due to overall growth in OS complexity. It's because that's when multicore CPUs were taking off. Before that, the cost of a global lock was far lower.
OpenBSD is straight unconcerned about performance. That's their choice.
I’ve been using Work for all my queries, since it seems to just be the same interface as Chat but with more features. I don’t understand why they’re two separate things.
reply