It's worse than that, isn't it? Not only are the algorithms presumably super linear, the transistor count has been increasing exponentially, but the compute power per transistor has been decreasing over time. See e.g. [1].
Although I suppose if the problem is embarrassingly parallel, the SpecINT x #cores curves might just about reach the #transistors curve.
yeah, that plots single-threaded performance, not total compute power. the point it's making is that now those transistors are going to parallelism rather than to single-threaded performance, and also the compute power per transistor stopped increasing around 02007 with the end of dennard scaling
your problem doesn't have to be ep to scale to 10² cores
i suspect it's true that compute power per transistor is dropping because thermal limits require dark silicon, but that plot doesn't show it
Although I suppose if the problem is embarrassingly parallel, the SpecINT x #cores curves might just about reach the #transistors curve.
[1] https://substackcdn.com/image/fetch/w_1272,c_limit,f_webp,q_... via https://www.semianalysis.com/p/a-century-of-moores-law figure 1