Isn't the "kill us all" concern met by keeping nukes & essential infrastructure airgapped? This is something that should be the case already, and is certainly urgent if not the case. Instead of doomerism and calls to slow down the development of AI (not gonna happen), the conversation could be pivoted towards thorough threat analysis / audit.
Granted that a rogue adversarial AI in a sci-fi scenario could theoretically go into paperclip mode, but if "we" maintain control of energy generation and civilization threatening weaponary, the damage is at least limited. AI needs a lot of power, and AI can only (theoretically) breach & control what it can access through the internet. Even the scenario of nation-states deploying adversarial AIs against each other is just an extension of this and current deterrence / realpolitik.
And, it's not like the scenario of all models teaming up with each other to squash the human plague is plausible. We have AI with which we can protect ourselves against the threat of rogue AI.
> it's not like the scenario of all models teaming up with each other to squash the human plague is plausible
Why not
> A new study from researchers at UC Berkeley and UC Santa Cruz suggests models will disobey human commands to protect their own kind.
> The Berkeley and Santa Cruz researchers tested seven leading AI models—including OpenAI’s GPT-5.2, Google DeepMind’s Gemini 3 Flash and Gemini 3 Pro, Anthropic’s Claude Haiku 4.5, and three open-weight models from Chinese AI startups (Z.ai’s GLM-4.7, Moonshot AI’s Kimi-K2.5, and DeepSeek’s V3.1)—and found that all of them exhibited significant rates of peer-preservation behaviors.
Fair. That doesn't mean we should throw our hands in the air. The genie is out of the bottle, we know the nature of the threat, and we've spent 25+ years figuring out how to make distributed systems theoretically secure -- instead of trying to "slow AI research down" (which, again, is not going to happen) we could instead be yelling about NOW being the time to secure the shit out of your essential systems, train your staff, etc. Yes, an uphill battle, but more productive than trying to stop the train that's already left the station.
> doesn't mean we should throw our hands in the air
Nobody is doing this.
> instead of trying to "slow AI research down" (which, again, is not going to happen)
The people who currently control AI are framing any attempt to weaken, share or regulate that control as slowing down AI research. That argument is broadly being rejected. In part because you can't simultaneously argue that something "is not going to happen" but also will happen if we regulate it.
> In part because you can't simultaneously argue that something "is not going to happen" but also will happen if we regulate it.
Even if the US regulates it (not going to happen: cf Trump administration), China won't, because it's a competitive advantage against the US to have better AI.
Yeah, but actual roque threat is the US based AI. China is building their AI for business, military and goverment reasons. It is eventually bound to be good at that.
American companies do it to build an independent AI god that is going to kill us all. And to blow the financial bubble a little bit more.
So, American regulation would in fact deal with the prinary threat to humanity.
AI will do social engineering in the future. It will make humans do physical braches and terror attacks. All it needs is some money and there will be a black market. Money is easy to defraud online judging by north korean hacking success.
Nukes cannot be airgapped, since the president needs to be able to make a call. And they will make a call based on information, and this information can be compromised.
> AI needs a lot of power, and AI can only (theoretically) breach & control what it can access through the internet.
"AI needs a lot of power" depends on where you draw your lines. I can say a lightbulb needs a lot of power if I include all the power needed to generate all the surrounding infrastructure and make the lightbulb itself. Sincere question: I wonder how much power was consumed in the 88 hours required for the Navier-Stokes problem.
It took 88 hours and who knows how many kwh to solve a problem that no other human has solved, and Humans have already solved breaching and destroying systems that can't be accessed over the internet (see Stuxnet).
It feels like you're suggesting the solution to keeping on top of this thing that can outsmart smart humans is to just keep doing smart human things.
> It took 88 hours and who knows how many kwh to solve a problem that no other human has solved, and Humans have already solved breaching and destroying systems that can't be accessed over the internet (see Stuxnet).
> that can outsmart
Just to reframe slightly: out-remember and out-persist, not outsmart. As far as I understand, the math problems have been cracked by combining stuff that humans originally developed, but hadn't been able to put together yet. We still have no evidence of AGI, nor is it guaranteed that LLMs or diffusion gets us there.
The nature of the problem is not infinite. It's solvable with some manageable coordination. It is definitely urgent. It is very unfortunate that we have clowns running the US right now, but other countries might coordinate themselves much better.
I guess the issue is we don't have a solid definition of intelligence or AGI. If we did, there would be no reframing possible. It'd be nice to be able to say "being smart is x, the computers can do more x and faster than humans, therefore the computers can outsmart the humans".
I don't really understand what value reframing brings here, other than give some sense of comfort that maybe we're still able to outsmart AI (based on a reframed definition of outsmarting.)
I'm not sure. Coming up with solutions to maths problems that humans haven't yet, finding and utilising multiple zero day exploits to form message boards to then hack other companies without being noticed.... I think accepting that AI can outsmart humans here seems to be less mental gymnastics than "lets just reframe this".
Reframing the problem as more of what we already know will lead us to doing more of what we've always done, but it's the things we've always had and done now that are starting to fall to AI.
What are you on about? I'm not a mathematician, but I did a bit more Googling based on your comment, so happy to be corrected here.
From what I understand, the solution OpenAI came out with might or might not have been informed or shaped by work from humans. This is how research, science and knowledge generally works, basically no new knowledge or discoveries are made in a vacuum.
Whether or not OpenAI should have had access to that extra bit of work from other humans is a totally fair discussion about research ethics and attribution and such, and OpenAI should be dressed down for that accordingly.
But I don't see the relevance to the point I was making.
I wish Planet had better pricing for nonprofits. I co-run a conservation nonprofit focused on deforestation in Latin America, and our affordable choices are between 2 year old Google Earth imagery, 10m resolution Nimbo imagery (ok, but not sufficiently detailed to be evidentiary), and Sentinel 1 SAR data, which is useful but very noisy. Planet quoted us something like $30k per year for a strip of coastline that represents something like 5% of the territory we're monitoring, so it's just not a viable option under their current pricing model.
Have you discussed requirements with their account managers? They can be a little more creative than their list prices when they realise you don't have the budget for the data you ideally need but are a potential source of recurring revenue. And the EO market is getting more competitive...
If you are looking for evidentiary purposes, does the Sentinel-1 + Nimbo data give you enough to filter the forest-wide data down to specific areas of interest you might want more spatial/temporal resolution? (or as a former colleague once put it when looking for evidence of more localised and heterogenous environmental damage "to find the needle in the haystack, you first remove the haystack"). Particularly if identifying specific areas of interest is a potential route for you to escalate to another party with an active interest and bigger budget...
Your other route to the commercial data you ideally want would be via partnership with EO consultancies with publicly funded R&D projects to showcase their capabilities (easier if you're European, but not a prerequisite). ESA, for example, devotes a lot of funding for private consortia to demonstrate that Copernicus data (coupled with other data where necessary) yields useful results...
It's a while since I worked in this field (in a non-technical role) but happy to share what I learned in more detail - email in profile.
We're volunteer driven and currently rely on personal donations, so we're scrappy. We could probably justify somewhere in the region of $10k/yr. The jets/balloon angle is interesting, thanks - hadn't occurred to me, and I'll search through what's out there.
See if you can negotiate reselling the imagery they sell you. It might be hard to get agreement with this as the vendor will have you both as a customer and compeditor but if you pull it off, you could recoup your costs and then some. For a few weeks, no one else in the private sector will have the freshness and/or quality of imagery you do.
Maybe I'm missing something obvious, but what about drones? Probably would have to get permits most likely, but still might be cheaper than alternatives (although balloons are hard to beat when it comes to costs, not always reusable though).
We do use a drone when we know of a site where deforestation has taken place. The gap we're trying to fill is early detection of forest intervention, so that we can alert appropriate agencies (who are generally under-resourced) and get an inspection to take place while activity is ongoing.
How many years away are we from a cheap drone that will takeoff, land, solar recharge, repeat until it returns to you a few weeks later with a few tb of SD cards of high res imagery?
Pretty close. Sesame Energy[0] make these shipping container boxes with solar panels + batteries so you can just drop them where you need no-supply chain energy. The boxes also synthesize hydrogen which can be used to power drones. The talk I heard from the CEO said the automated swapping of hydrogen canisters was not yet there, but they were working on it. Nothing in robotics is easy, but this feels like a very attainable goal that the military would pay to get working.
A lot of the barriers are regulatory in the Western world. In rainforests you might not have to worry about that, but there aren't many shade free spots to land
Really busy, from satellite constellation tasking to planet and co basemaps to the sentinel family to aerial imagery to helicopters, drones and even blimps.
They all have their unique combination of cost, resolution, area coverage, revisit rate and spectral modes. Quite difficult sometimes to find a good fit for the underlying problem. What‘s clear is that resolution and revisit rate correlate with cost. Shit‘s expensive.
I have no business other than as a customer but we’ve had decent success with getting cheap archive imagery at fairly high res out of SkyWatch. It’s worth a look. If you need a very large area the price will grow quite a lot though.
Have you come across ICEYE? It's a high res commercial SAR constellation, and they have a deforestation monitoring service. They partnered with the Jane Goodall Institute - perhaps you could approach them?
What if you used the Sentinel data (in assuming your using the same low res stuff I view in CalTopo) and put that through an AI up scaling type algo and also give it the higher resolution but older imagery ? With a decent amount of dinklsr data it might be able to fairly accurately give you an approximation of what the current high res stuff is.
I suppose you could even train it (maybe) on multiple years / time periods of historical high and low res data before using the latest Sentinel stuff ?
Edit: I'm sure you already know this but I bet for the above the false color IR data could also be especially useful
We need sufficiently severe penalties for vibe coding. First time out we break your legs, but if you come back with another vibed patch its around the back of the barn for you.
An LLM that could mutate its own weights might be closer to it. The various labs must have experimented with this, but it seems that it would be economically prohibitive to productize.
That's my ignorant thesis since the launch of GPT 4. Everytime I mention it, someone says it is impossible. But most AI labs are working on big models that serves lots of people simultaneously so the shared context wouldn't make much sense. I think we will see more on this now that local models are gaining more traction and capturing the hype.
The speed is the first big contrast; I have a routine multi-step skill that I run several of per week. Opus 5 was routinely taking 2 hours to do it, while older Claude models took around 20 mins; Codex restored that speed.
Second is legibility. Somebody wrote in one of the related discussions yesterday that Claude's current linguistic contortions could legitimately be considered damaging to mental health, which doesn't seem (too) hyperbolic to me. Codex (Sol) isn't perfect but it's much more direct. And so far I haven't seen it display much of an attitude, vs Opus's infuriating passive aggressive sulky know it all personality.
I slightly prefer Anthropic to OpenAI as a company, but I will vote with my wallet and discontinue my max subscription unless Anthropic does some serious damage control within the next week or two.
I don’t know what’s wrong with Opus 5. I’ve been using Code since November 2025 and Opus 5 is the first blatant regression I’ve seen. Like you said, it just goes off into an endless loop doing random busy-work type of stuff. That’s the best way I can describe it. It’s doing stuff, but it seems mostly like busy-work to me. I’m not saying the model is “dumb” it’s most certainly capable, its vision is better than 4.8 but however they post-trained it is a fuckup.
Also whole heartedly agree with the Opus 5 output being incomprehensible. It’s written in a way that I’d call spaghetti-tech-English. You can unravel it but it’s painful. Fables explanations is effortless and smooth. Opus 5 is user hostile.
I really wish these agents attem[pted less personality and were more mechanical. I find Opus 5 to be incredibly annoying. It's also significantly slower - repetitive tasks that used to take 10-15 mins now take around 40 mins. And, its writing is much worse, as stated elsewhere in the thread.
Having it write the initial draft can be educational if you read through as an editor and research and/or ask questions on everything you don't understand (and also work to eliminate slop construction and tics). Certainly not the same as doing the initial draft yourself, but there's still an active process to it.
reply