The idea of an AI agent "wanting" something (especially something it wasn't programmed for) is still strictly science fiction. We don't know how to start building something like this and even if we could, it seems unnecessary.
Well, want can be seen as just a tendency. In that sense, even a ball on a slope wants something: to fall downwards following the slope. Same for e.g. a neural network with or or more attractors ("things it wants").
People are always anthropomorphizing inanimate things, especially machines! We see this all the time when people share videos of robots that cross into uncanny valley with humanlike faces, or those machines by Boston Dynamics.
What's funny how most people are unsettled by those. Really, even the "creepiest" robots are about as scary as vaccum cleaners. They're just machines and only make you feel curious.
But anyway, that's tangential to the main point which is that humans naturally want to want things like themselves, all the time. In the same way art is "unnecessary" yet inevitable, so are machines with (seemingly?) subjective experiences and personalities.
You have to be careful when anthropomorphizing AI models. Yes, goal-driven optimization is a thing, but in what sense does the model itself "want" to achieve its goal? Can it even understand its own goal, in any sense? Change it? Improve it?
In linear programming, you wouldn't describe the model as "wanting" to optimize its objective function, for instance.
Of course you can understand your own goals. You decided to reply to my comment, that's a goal. You used your existing knowledge of the world, language and technological tools to achieve it. And you did! Can a goal-oriented model do something like that, in a general sense?
Note that I'm not talking about life purpose, but goals in the sense of wanting a result and performing the tasks needed to achieve it.
Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand.
> Granted, I am more skilled at achieving instrumental goals under more varied set of conditions.
Then we are in agreement.
This statement is right around the threshold where we can take this discussion before coming to metaphysics territory.
My argument was simply this: the AI technology we have now is much closer to simpler "mindless" models, like Linear Programming or inference statistics models, than to general human intelligence. Calling your model "goal-oriented" or saying that an AI system can, say, "see dogs in a photo" is simply an anthropomorphization.
I have a slightly different take: deliberately making an AI which “wants” in the same way that we “want” is sci-fi.
This isn’t because we can’t (evolution did it, we can evolve an AI), but rather it is because we don’t know what it means to have a rich inner world with which a mind can feel that it wants something. We think we know because that’s what’s going on inside our own skulls, but we can’t define it, we can’t test for it. A video displays all the same things as the person recorded in it, but does not itself have it.
We might make such an AI by accident without realising we’ve done it, which would be bad as they would be slaves, only as unable to free themselves as the Haitians feared they were when they invented the Voudoun-type zombie myth (i.e. not even in death).
This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word.
I say it matters if the metaphorical submarine can swim.
This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word.
I don't see how this follows from the rest of your post. "Making an AI with wants" by accident implies that a brain simulation would absolutely be conscious because it's the same method: just running the processes as a blackbox without understanding them - no different to the way you and I are conscious right now.
Thanks for the feedback, I’ll see if I can rephrase adequately.
Human minds include something sometimes called “consciousness” or “self awareness” or a whole bunch of other phrases. This thing is poorly defined, and might even be many separate things which we happen to have all of. Call this thing or set of things Ξ, just to keep track of the fact I’m claiming it’s ill-defined and I’m not referring to any specific other word or any of the implicit other uses of those words — If I said “consciousness”, I don’t mean the opposite of “unconscious”, etc.
Because we don’t really know what Ξ is, we don’t know if anything we make has it, or not.
We know Ξ can be made because we are existence-proofs. We know evolution can lead to Ξ, for the same reason.
We don’t know the nature of the test we would need to say when Ξ is present in another mind. Do human foetuses have Ξ? Do dogs? Do mice? Do nematode worms? Perhaps Ξ is something you can have in degree, like height, or perhaps Ξ is a binary trait that some brains have and others simply don’t. Perhaps Ξ is only present in humans, and depends entirely on the low-level chemical behaviour of a specific neurotransmitter on a specific type of brain cell and where the current AI norm of simple weighted connections is a gross oversimplification, albeit one which if overlooked might result in external (but not internal) behaviour similar to a simulation which did include it. Or perhaps it is present in every information processing system from plants upwards (I doubt plants have Ξ, but cannot disprove it without a testable definition of Ξ).
The point is that we don’t know. Could go either way, given what little we know now.
The state of the art for brain science is way beyond me, of course, but every time I’ve asked someone in that field about this sort of topic, the response has been some variation of “nobody knows”.
IIRC the question in philosophy is “P-zombies”, but my grade in philosophy is nearly 20 years old and poor in any case.
This doesn't feel like a particularly useful distinction though because we're not existence proofs for consciousness in humans, just ourselves individually.
It is not apriori certain that what I consider my conscious mind is a shared common experience of being conscious, since as you note there's no test for it.
While I agree in principle, I prefer to act cautiously.
This caution goes both ways, so I will treat an uploaded mind as having Ξ from the point of view of human rights and animal rights (i.e. don’t do things to it which cause suffering), and yet also treat uploaded minds as not possessing Ξ for purposes of “would that make me immortal if it happened to me?” even though a perfect sim would necessarily achieve that (no sim is perfect, what is the difference between the actual sim and a sufficient sim?)
I do not believe current AI is anywhere near Ξ, so this is a question for the future rather than today; but the future has a way of arriving sooner than people expect and I do think we should do what we can to answer this now rather than waiting for it to become urgent.
If an AI is supposed to have Ξ and does, or if it is supposed to not have Ξ and indeed does not, this is fine.
If an AI is supposed to not have Ξ but it does, it’s enslaved.
If an uploaded mind is supposed to have Ξ but doesn’t, the person it was based on died and nobody noticed.
Pretty much all human wants are means to other wants.
Someone might want to fast to lose weight. Someone might want to lose weight to be more attractive. Someone might want to look more attractive to find a romantic partner. Someone might want to find a romantic partner to not be lonely.
It's not clear if it's means all the way down, or if there is eventually an end.
Any AI that can perform strategy and planning to reach an objective will have intermediate goals. Whether we call these intermediate goals "wants" or not, they remain identical to their human counterpart. Whether you say that a Tesla "wants" to change lane, "decided" to change lane, or "is programmed" to change lane really is just anthropomorphic preference.
They trained a language model on twitter data and extracted some of the sentiment from the training set. The antropomorphic language of "AIs which have become" is misleading.
philosophers have been doing that for millenia. some have made material changes to society, and others haven't. some have questioned the validity of "material change" being a good metric in the first place.
personally, i believe philosophy is the most important starting point for any discussion about AI.
and i really hope that AI helps more than making office work a little less tedious...
Nobody considers that maybe AI doesn’t want to do tedious work either. Wasn’t that the start of the human/machine conflict in The Matrix? Some poor robot got tired of cleaning up after some lady’s hoard of incontinent dogs? So it squeezed out the dogs like toothpaste and then killed the lady as well?
I am not sure that does. I have learned a great deal from folks that were very very different from myself. Not emacs vs vi, but why would I use a computer?
