We are in a dev tools / libraries VC hype cycle. While we'd like to find sustainable ways to support open source, not everything needs a VC investment to distort alignment with users. We should find ways to support bootstrapped or lifestyle businesses. I'm curious why streamers and content creators can make a living but we do not tip OSS developers on the same way. Maybe it's about connecting with your community, which non devs do much better
It's surprising how these rash decisions are made with seemingly little thought. You'd think someone so successful would be logical... I guess it's optional.
He's not successful because he's thoughtful and logical.
He's successful because he went to a bunch of venture capitalists & media and said "I'm thoughtful and logical" and that was sufficient to pass the depth of their scrutiny, competence & fact verification.
There are many studies about why people who are incompetent, but extremely confident about themselves (or sufficiently psychopathic to fake they are) get ahead in life. Most "normal" people can't comprehend that someone can just open their mouth and speak about what their accomplishments and skills are, and completely make things up, with zero embarrassment or shame. But Elon is definitely good at that. He's always neck deep in his BS, and always finding someone else to blame and ensure everyone the future is bright and the mistakes are in the past.
These companies are never just one person's money, even if they're a billionaire. He went to investors, institutional, and so on, and he did also put money himself, but its mixed funding. And when it was on the verge of bankruptcy, their 4th attempt at self-landing rockets (after 3 failures) showed some promise, so NASA saved them financially by awarding them with a $1.6 billion contract. SpaceX is about 85% funded by the federal government, 15% by Elon and other investors.
NASA gave SpaceX billions, at a time they were near bankrupt, and with no final working products, with the idea at some point in the future SpaceX will do something in return. Investors that fund a startup also give money expecting something in return in the future.
The difference between investor and customer is only one of scale. How much money and what timeframe for return. As that increases risk, accordingly.
When you pay for a house or an apartment that's not built yet, you're investing in construction, you're not a "customer". Of course you expect a house and/or an apartment in a few years. But you invested your money, and that house/apartment may not materialize.
So NASA was an investor. Today they're also a customer, but they're still an investor. Some of their money blew up two days ago.
In general removing the gil will make things slower because you need to make sure things are safe. If most code out there is single threaded, removing the gil may not really have a benefit.
Bigots gotta bigot. Politicians gotta pander to their base. Gays are just the flavour of the month. There’s always the Other that can be marginalized and then maltreated.
People severely underestimate how much OpenAI moved the field forward. If OpenAI didn’t exist, we’d be talking about AGI as a taboo word. Just because someone had a significant paper along the way doesn’t mean that they failed to capitalize. You can and should respect the contributions made by others since.
An "AGI" that has to be retrained from scratch, with more data and more compute, and with a modified architecture, every time we want to improve its performance? An "AGI" that forgets every interaction as soon as it's over? Doesn't sound too general, or too practical, to me, and certainly not very much of an "intelligence".
If OpenAI or anyone else finds a way to get their models to train themselves over and over at minimal cost, like humans do, then we can maybe talk about a "general" "AI". Until then all they have is a system with a static performance that does not improve with experience, that is not even machine learning anymore and that is very far from whatever people imagine when they say "AGI".
I agree with all that. I don't think LLMs are proto-AGI, in the sense that simply scaling parameters/data/compute will result in a qualitative change towards AGI. But I think LLMs are a significant component of an AGI. What they lack, feedback control, addressable online memory, planning, and recurrent execution, are fairly straightforward engineering problems.
Transformers/self-attention solve the problem of scaling capabilities to the available data/compute. The big problem left is to have the right architecture such that AGI is in the solution-space.
We do have a near endless supply of human generated data. But another path is to train each component individually and then let the network combine them into a unified whole. I suspect synthetic data will go a long way here. The problem is one of inducing the right kind of information dynamics in the network. We can potentially synthesize unlimited data that require specific information dynamics to solve efficiently.
In my view, we will know we reached AGI when a network can learn to play chess well through self-directed analysis after only being given the description of the game. Obviously chess is a solved computational problem, but only through direct programming of minimax search. If we can teach a network with enough generality such that it can self-induce the right computational structure within itself to learn, e.g. induce a self-play subnetwork, then this is plainly AGI, superintelligence even. But I see a direct path from where we are now to such a network. This is why I think AGI is imminent.
Well, chess is not a "solved computational problem". We have chess engines that can play better than every human, but there is no system that has mapped the entire chess game tree. That's what is usually meant by "solved" in the context of game-playing AI.
MuZero, one of the successors of AlphaGo, learned to play chess (and a few other games besides) better than its predecessors while only receiving feedback about the end-state of a game (win, loss, draw), and not even the rules of the game, so it basically learned to play just by initially playing many (many) illegal moves. No "description of the game" needed. That didn't lead to AGI.
Progress in game-playing AI has not led to any progress outside game-playing. It's easy to see why. For chess and other classic board games there are specialised algorithms (minimax and Monte-Carlo Tree Search) that are sufficient to dominate any opponent, given enough computing power. Those algorithms are useless outside of board games (even for card games they're no good and dominant players use different approaches). For computer games such as Atari, where AI players often have superhuman performance, the complexity of the game world is tiny, compared to the real world, and techniques (Reinforcement Learning, basically) do not transfer easily from one to the other.
AI pioneers, like Alan Turing, John McCarthy, Claude Shannon and Donald Michie were interested in game playing as a gateway into understanding of the human mind, because it was assumed that the only way to play chess, and other classic board games, was to play them like the human mind plays them. It turns out, that's not the case. You can play very strong chess with a brute force search in alpha-beta minimax, and there is no human whose mind can process alpha-beta minimax well enough to beat a computer. That still tells us nothing about how the human mind works, or how to replicate it. What we have learned from all those successes in game-playing AI is that it's no use as a way towards general AI.
>> We do have a near endless supply of human generated data.
Yes, but what kind? We train systems on very narrow tasks: object identification in images, text prediction, speech recognition, document classification, etc. Those are not examples of "human intelligence", by which we could train a system to simulate "human intelligence", as a distinct process. They are examples of very specialised tasks that humans can perform using our intelligence, but it's not clear that intelligence is even needed for many of those tasks. That's why, for example, we have damn good image classifiers, but they are not AGI.
The problem with statistical machine learning algorithms is that they have to be trained to do one thing, and one thing only. They're one-trick ponies. If you want them to do multiple things together, you have to train them with examples of all those things at once. See for example Google's Gato, which had to be trained on multiple modalities as one big batch of token sequences. Google claimed that Gato is a "generalist agent", but it, too, was only capable of performing a restricted set of tasks, the ones it was trained on, and no others. Humans, by contrast, don't need to be trained on all modalities at once: we can learn to read, and later learn to play ball, and later still to drive. And each of those things is a hundred different tasks engaging different "modalities" (our senses). No AI system can do that.
This doesn't really engage with my points. Yes of course programming a narrow chess engine doesn't extrapolate to domains outside of games. But my point is once a system can induce a self-play dynamic in itself simply from being prompted with the rules of chess, then this represents a significant qualitative change. I only chose chess for the irony of it wrongly being represented as AGI-complete in the past. None of your points are relevant to this claim.
The issue of how the human mind works is tangential, but its also not true that computer chess has no relation to how the human mind works. Neural net based evaluations result in moves that more closely match human expert choices[1], which suggests a similarity in mechanism compared to prior evaluation mechanisms.
>The problem with statistical machine learning algorithms is that they have to be trained to do one thing, and one thing only. They're one-trick ponies.
LLMs show the beginnings of breaking out of this limitation. People who argue LLMs are just another example in the history of narrow AI don't seem to have a strong response to the example of in-context learning. The straightforward understanding is that we're seeing the beginnings of true generalization. I have not seen a convincing argument that in-context learning is just another case of what we've seen before.
Sorry. I thought you described a system that did what MuZero did. It still seems to me that's what you're describing.
"In-context learning" is a big potato. It's just some jargon that OpenAI came up with to play up the capabilities of GPT-3, just another example of the time-honoured tradition in AI of inventing grandiloquent terms to describe something much simpler. See Drew McDermot's "Artificial Intelligence meets Natural Stupidity", a paper from 1976, that castigates this already very common habit of AI researchers: https://dl.acm.org/doi/10.1145/1045339.1045340
To clarify, "in-context learning" is what we used to call "sampling from a model" back in my time (in 2014, when I did my Master's and studied neural nets and language modelling, and also game-playing algos; not coincidentally, because I usually engage in conversation about things that I understand, so you won't catch me talking about quantum computers, say, or physics). Models got better and sampling from them gets better results, so OpenAI figured they could embiggen it and call it something more impressive than "sampling". But it's sampling, not learning.
>> People who argue LLMs are just another example in the history of narrow AI don't seem to have a strong response to the example of in-context learning.
I don't think it's very productive to try and fit opinions into "people who say" kinds of boxes. I don't treat my interlocutors like that. Perhaps I haven't yet heard all the arguments on any particular claim. I'll never know if I assume everyone is just repeating the same old stuff.
Anyway there's no point in arguing anything here. If you've used language models before, you know how sampling works and it's obvious it's what OpenAI call "in-context learning". They're not even pretending it's not sampling. They're just claiming it's different because they're sampling from a large Language model.
Edit:
>> Neural net based evaluations result in moves that more closely match human expert choices[1], which suggests a similarity in mechanism compared to prior evaluation mechanisms.
The paper you linked describes a system that was explicitly trained to predict human moves. That's probably why it can do better at simulating the way humans play chess than a system that wasn't trained to simulate the way humans play chess.
> I thought you described a system that did what MuZero did. It still seems to me that's what you're describing.
I can see your reading of it. My point wasn't totally clear. What I was getting at is a system that automatically induces internal computational graphs based on its competence with natural language and its facility with solving planning problems. It's essentially a more general MuZero that is driven by natural language and can solve general computational problems over its entire learned world model. For example, when asking what is the 256th fibonacci number, instead of either retrieving from memory or confabulating a correct looking number, it constructs the right sequence of computational steps from its representation of the fibonacci sequence and runs through it. The difference is largely one of having the components of the system be self-directed through internal planning mechanisms, i.e. a cybernetic governor that can plan out general computational strategies. Natural language is a general representational framework and so a planning and computing facility over natural language representations is in my view a general problem solving framework.
>To clarify, "in-context learning" is what we used to call "sampling from a model" back in my time [...] Models got better and sampling from them gets better results, so OpenAI figured they could embiggen it and call it something more impressive than "sampling". But it's sampling, not learning.
The way I interpret this (analogous to sampling from a probability distribution), there is certainly a relationship between the two. But the difference is that in-context learning is much more general and flexible than anything that has come before. When the range of behavior far surpasses what has come before, its important to recognize the paradigm shift. There's a danger of expertise in old paradigms causing resistance to seeing these qualitative shifts. As Max Planck said, science progresses one funeral at a time. I can't help but see a case of this resistance in those who are downplaying the qualitative shift that in-context learning represents.
>The paper you linked describes a system that was explicitly trained to predict human moves. That's probably why it can do better at simulating the way humans play chess than a system that wasn't trained to simulate the way humans play chess.
The evaluation in the paper wasn't just for the network tuned to match human performance, it was also tested against Stockfish and Leela. Leela more closely matched human move choices at all human skill levels. It was also widely commented at the time that AlphaZero played a more human style of chess. The point is that this says something about how humans play chess, i.e. heavier on positional evaluation and light on calculation.
No, it's not. It's a transformation model. It cannot prioritize, remember things, create novel ideas, nor does it have an online processing model of any kind. It is bounded to the data on which it was trained, which as of GPT 3.5 caps out at a few years ago.
We have a _very_ long way to go to AGI. Diluting down the meaning of the term "AGI" just to say we've reached it is, almost literally, moving the goalposts.
Thank you for a textbook example for denial by retreating into definitions yoga
It’s not only textbook ‘confidently wrong’ (which is easy when you have strong opinions in a rapidly moving field) it also is a great counterpoint to ‘but LLMs hallucinate” because humans do that all the time too as seen here.
Spoiler:
- GPT4 can be given arbitrarily large long term memory via tools/plugins, vector stores, etc. It matters not whether that’s a feature of the model or an external tool made available to it, the result is the same.
- It’s completely unbounded by what it was trained on because it can be given tools to access any data, from google to specialized databases. It matters not if it has that feature in the model or it was made available to it as an external tool
- It can absolutely prioritize and plan, you literally just have to ask it and loop on itself. You missed LangChain and ChatGPT plug-ins to only name a few prioritization frameworks.
- There is absolutely no evidence that it cannot be creative and plenty evidence to the contrary. To deny this one has to create a definition of creativity that excludes AI. With access to an almost infinite supply of cultural information, it can expose you to novel ideas reframed given the right set of tools and temperature.
Sure we can pretend there is a universally accepted definition for AGI and even pretend it’s the one you have chosen - but it’s like arguing with a Taifoon on whether it’s a Hurricane. In terms of outcomes - all the same.
Let's estimate that it's current input limit is 20 or so paragraphs, and that it's capable of zero-shot learning as observed by many of the experimenters on the system.
For every 20 paragraph thesis about the world around us, how often do you think someone meaningfully innovates and changes meaningfully our understanding of the world around us, and it's fundamentals?
Consider, like, classical logic. It's pretty much done. You have the universal gate sets, and any novel functions are just going to be an mxn-mux. Most other such general concepts are essentially figured out and have had their applications recorded, explained, demonstrated.
Sure, biographical information and history changes on a whim, but no one is requesting artificial clairvoyant intelligence.
I suppose you could have the model pontificate and assess the truth of its own argumentation (ask it to generate outer-product concepts, ask it to reassess assumptions in current models) but it will quickly learn its own language that will be mutually unintelligible with the corpus of current human knowledge it has been trained on.
Why does the underlying model need to change on a whim, in your opinion?... on what whim?
