> It's not reasoning about your code, nor about the explanation it gives you.
We don't really know what "reasoning" is. Presumably you think humans reason about code, but humans also only have statistical models of most problems. So if humans only reason probabilistically about problems, which is why they still make mistakes, then the only difference is that AI is just worse at it. That's not an indication it isn't "reasoning".
“We don’t know how we do it, so we can’t say this isn’t how we do it” isn’t a valid argument.
We may not know exactly how we reason, but we can rule out probabilistic guessing. And even if that is a part of it, we’re capable of far more sophisticated models. We can recurse and hold links. We can also make intuitive leaps that aren’t quite built on probability.
> “We don’t know how we do it, so we can’t say this isn’t how we do it” isn’t a valid argument
Yes it is, assuming we don't know of any specific things that "this" literally can't do but that we can. Which we currently don't, we merely have suspicions.
> We may not know exactly how we reason, but we can rule out probabilistic guessing.
No we can't.
> even if that is a part of it, we’re capable of far more sophisticated models.
Yes, but that would be a difference of degree not of kind. This is what scaling proponents have been saying, eg. that scaling does not appear to have a limit.
> We can also make intuitive leaps that aren’t quite built on probability.
I don't think we have evidence of that. "Intuitive leap" could just be a link generated from sampling some random variable.
But it’s not replicating results that a human would give you.
Since it’s not giving the same type of results, then it’s not doing the same thing. If anything, LLMs have definitively ruled out probabilistic guessing as the model for human intelligence.
Even now, you’re trying to force LLMs onto human intelligence. Insisting it is despite it not delivering the results. And I’m sure you believe if we just fire up another few million gpus, we’d get there. But we’ll just get wrong faster. LLMs don’t produce new, they just remix old
> Even now, you’re trying to force LLMs onto human intelligence
I'm not forcing anything, I'm specifically refuting the claims that we know that LLMs are not how humans work, and that LLMs are not reasoning. We simply don't know either of these, and we definitely have not ruled out statistical completion wholesale.
Also, I don't even know what you mean that LLMs are not giving the same types of results as humans. An articulate human who was hired to write a short essay on given query will produce what looks like ChatGPT output, modulo some quirks that we've forced ChatGPT to produce via reinforcement learning.
We don't really know what "reasoning" is. Presumably you think humans reason about code, but humans also only have statistical models of most problems. So if humans only reason probabilistically about problems, which is why they still make mistakes, then the only difference is that AI is just worse at it. That's not an indication it isn't "reasoning".