What makes me wonder is how slight modifications make the algorithm miss gross, highly visible features, e.g mistake a blue sign for a red sign, or an upward-pointing triangle for a downward-pointing. I suspect it won't be very hard to make the algorithm pay more attention to it, by specially teaching it to tell between such differences, and maybe by running several separate networks taught to tell apart particular narrow features, not complete signs.
It seems to indicate that these machines are nowhere near as smart as they appear to be based on earlier successes. I'm starting to get worried about the possibility of another AI Winter if it turns out that reality and hype are too far apart.
The AI Winter will come only when the investment money runs out. As the Fed is keen to raise rates as slow as humanly possible (similar to 2004, and unlike 1994), I think the AI practitioners still have a quite a bit of runway left.
