> There's way more to lose by slowing down AI because of emission control than to gain by reducing emissions.
If AI works out as proponents would make you believe, then either you'll see masses of people out of a job - for which society is unprepared. Or you'll see a big, society-wide productivity boost. Read: consuming non-renewable resources on this planet even faster (not just energy).
Society as a whole would benefit if rollout would slow down. Give us time to reflect on those 2nd-order effects & how to address them. But instead we have the opposite: a crazy race with big AI labs trying to out-spend & out-do the other guy, ignoring externalities everywhere.
Interesting. By this definition, a poorly chosen algorithm is also "bloat", which doesn't seem right to me.
I see bloat mostly as "unnecessary dependencies." Things which, if you removed them, the product would improve.
e.g. importing a whole library for what could have been a simple function.
If something justifies its "weight", then it's not bloat. It's functional.
But there's also "feature bloat." If a feature or change doesn't need to exist, or makes things worse, then it's bloat. If you removed it, the project would improve.
It's ironic when data center operators, who I'd expect to be experts on calculating complex cost/benefit scenarios, check for & arrange what's needed, sidestep better solutions because there's a bunch of requirements attached.
Attaching to the grid, at the powers desired, in the locations desired involves years of waiting beyond construction. Onsite genetation can be built within the overall construction timeline.
Maybe poor site selection, but I'm sure there's a cost optimization there. And my understanding is that there's not a whole lot of excess grid capacity around the country at the moment. Utilities are spending a lot of resources on changing the mix of generation and wildfire mitigation and don't have a lot of resources for adding generation capacity beyond what is immediately needed or increasing interconnection capacity.
This is different than the data center boom in the .com era; site selection was different, the grid situation was different, and the data centers being built were smaller loads.
I'm not an expert in the US grid, but where I am, the grid moves at a different pace.
It also has a bunch of constraints that are not that appealing. For example there is a strict supply limit. You can't just attach 4x6MW connections in a few weeks. More over, that kind of cable is £900 a meter, at the least. Its also oil filled and needs a bunch of other infra to keep it cold.
Then, the lead time for transformers is also quite high. I'm not sure about the switch gear and meters though.
Once you have all that, you need to make sure your local spur of the grid has enough capacity to provide what you need.
Then you can bid for a connection. THe further away you are from the grid the more it costs.
AI inference is basically 1 operation iterated over every single model weight. That's what's makes computer in memory attractive and feasible.
General purpose workloads would require much more silicon real estate and are not (typically) nearly as bandwidth hungry. Just like not all CPU workloads benefit from being adapted to a GPU, even fewer will benefit from being adapted to processing in RAM.
AI is not intrinsically matmul. Current models are a framing of the problem as matmul to enable the bulk operations necessary. There are many calculations that can (and have been) reframed into matrix form to achieve similar results. More than once I have seen a matmul version of something that I thought couldn't have been representable that way. I'm not sure if the greater benefit is the performance or the ability to conceptually manage the problem in you head.
Not all algorithms can be converted of course, but a lot of that is because there are some inherently unparallelizable problems.
Many algorithms can be reasonably converted into matmul. Not many algorithms can be reasonably converted into very low precision matmul. And that's all that these things are going to do.
I looked at this for a client close to 20 years ago. The problem is that you need to get software support for it. Arguably easier these days with open source but still doesn't magically happen.
The nice thing is that for GPUs you're rarely writing directly against the metal but instead pseudocode lowered by the driver; the driver could know to just lower it this way. On an NVidia GPU they control the whole stack and don't need to get a further ecosystem going there. Possibly the same for Apple.
Apple is such a vertical stack, I'm not sure how to generalize from that. Certainly the same was true on IBM mainframes. And, when I wrote open source, I could probably have written abstraction more broadly.
Not sure why the negativity in previous comments. There's 2 types of results that may come from research like this:
1) Developing cures for human diseases / health conditions. And 2) Understanding the mechanisms behind those diseases.
Obviously findings in mice don't apply to humans 1:1. Humans != mice. But that doesn't negate 2). Every single link in a complex process that's better understood, brings solutions closer to becoming reality. So 2) helps with 1) as well. Just in a less direct manner.
If AI works out as proponents would make you believe, then either you'll see masses of people out of a job - for which society is unprepared. Or you'll see a big, society-wide productivity boost. Read: consuming non-renewable resources on this planet even faster (not just energy).
Society as a whole would benefit if rollout would slow down. Give us time to reflect on those 2nd-order effects & how to address them. But instead we have the opposite: a crazy race with big AI labs trying to out-spend & out-do the other guy, ignoring externalities everywhere.