That he mentions his barber using it without prompting by him means that we're at least on the part of the curve from the early mid up to the very end.

I've noticed this too with friends and coworkers. People that I would think would have no business knowing about chatGPT are already using it in their work and their lives.

Almost axiomatically, that means that we have to be in the 'mass adoption' phase to some degree.

But I don't agree. I think we're in the early shallow phase still.

'The Kids' aren't the ones using it. I'm not being left behind by the 'hip new thing' like I should be. To me, that means that the nerds are still gawking about with it, trying on new things. Once 'The Kids' get a hand on it and use it in the innocent and unabashed way that they always do, then I can say we're in the early-mid phase. And by 'The Kids', I mean not the nerds at school, but the 'It Girls' and the 'Jocks', those kinds of people. If you hear your stoner nephew talking about it, but not your stoner brother, then you're in the early-mid phase, at least to me.

Id say we are still very much on the flat part and going to be there at sometime.

ChatGPT is just a really, really, REALLY good search. Things that used to take multiple google searches now take one query. This will definitely disrupt quite a few things and render some obsolete, but generally the effect will just be making things more efficient.

There are three things IMO that is going to make us start shooting up the final part of S curve.

First is efficient on the fly learning. If you think about how humans learn, we don't retrain ourselves every time we recieve a new piece of information that doesn't go contrary to what we know - a neural network however needs to be iteratively trained on it, with weight changes in most every single one of its parameters. We need models like Chat GPT to be presented with new text, and not only "remember" it, but also contextualize it.

Second is the contextualization mechanic itself. Its well known that generative models are statistical at their core, ChatGPT is wrong in a lot of cases. There needs to be some way to encode truth into the model.

Third is applying neural nets towards generating neural net architectures (or more specifically optimized compute graphs). The tricky part here is to determine what the loss function is (you wouldn't want to generate a net, train said net, evaluate its performance, then feed that back into the original generative net).

It's difficult to say. We're certainly making impressive advances now, but this is not the first time that AI has had a period of rapid improvement. The first and second AI winters were both preceded by periods of intense optimism [1].

I personally believe in the long-term vision of AI/ML/RL. Progress will continue to be made in the long run. However, we don't know what we don't know. We may hit a wall in a few years that takes decades to overcome. On the other hand, perhaps the current rate of innovation will continue for the forseeable future. Time will tell.

[1]: https://en.wikipedia.org/wiki/AI_winter

I agree. We're just barely uncovering the tip of the iceberg right now. Everything we've seen up til now has been a proof-of-concept. The true scope of this field has yet to be realized.

Since AI isn't meaningfully involved in designing new AI yet (unsure if it will be), I'm not convinced we're on an S-curve let alone an exponential curve.

It seems closer to this: https://xkcd.com/605/

I'll concede there have been large breakthroughs, but to extrapolate from that there will be even larger breakthroughs (without a really solid account of why that would be) seems very sketchy. I get that the people who are trying to sell the idea to investors are pitching this line, but applying a modicum of critical thinking, that's relatively rarely how things work. Returns tend to diminish with the amount of effort put in, not exponentially increase.

I think we are past the midpoint. AI research is a half a century old, the current models are trained on all of human language to date, moore’s law has ended, and the development costs for improvements might be like a needle in the haystack with each experiment costing tens of millions of dollars.

I don't think so. The end point of that curve is models with human like reasoning or even superhuman ability.

I think we may be on the S part of the curve, but thanks to processing concerns we will be in for veeeeeerrryyyy gentle slope on that S. Capital is getting more expensive, processors getting less fast, and in general tech is burdened with a slowdown.

We are in for a revolution, I believe, but one that will be a bit slower than the internet. More akin to maybe the revolution of chemical engineering, something that lasts a lifetime instead of a decade.

> I don't think so. The end point of that curve is models with human like reasoning or even superhuman ability.

What makes you think that?

Because we are a living example of what is eventually possible with "artificial" intelligence. If we exist, it can at least match us.

Expert systems and various other previous artificial intelligence systems failed to rise to our level of capability. Why should we expect large language models to be different?

I'm not really talking about LLMs specifically. I'm talking about AI as a field.

Although neural networks are capable of any form of reasoning or understanding and human language is a very strongly diverse field so I do believe LLMs are a strong contender for the production of general AI.

Lots of data.

Easy to train thanks to text being very simple.

Easy to express just about any concept or information.

Requires some amount of domain knowledge to make good prediction of what comes next.

> I'll start playing with ChatGPT and the like when I can run them on my own hardware.

Koboldai