The cost of GPU time isn't just the cost that you see (buying them initially, paying for service if they are not yours, paying for electricity if they are) but the cost to the environment. Data centre power draws are increasing significantly and the recent explosion in LLM model creation is part of that.

Yes, things are getting better per unit (GPUs get more efficient, better yet AI-optimised chipsets are an order more efficient than using GPUs, etc.) but are they getting better per unit of compute faster than the number of compute units being used is increasing ATM?

But is it dropping faster than the needs of the next model that needs to be trained?

Short answer is yes.

Also, GPU pricing is hardly relevant. From now on we will see dedicated co-processors on the GPU to handle these things.

They will keep on keeping up with the demand until we meet actual physical limits.