It seems like we're hitting a solid plateau of LLM performance with only slight changes each generation. The jumps between versions are getting smaller. When will the AI bubble pop?
SWE-bench pro is ~20% higher than the previous .1 generation which was released 2 months ago. For their SWE benchmark, the token consumption iso-performance is down 2x from the model they released 2 months ago.
If this is a plateau I struggle to imagine what you consider fast progress.
Does the blog post explain how this happened exactly? Did he leak his API key in frontend code somehow, or was his project itself vulnerable to misuse? I'm curious how someone racked up 30k in a few hours.
Some of these sound just as made-up as a lot of Western dining "rules." Maybe someone more familiar with the culture can say whether or not these are true faux pas in an everyday ramen shop or similar.
No one is going to get mad at you for violating these, but they will judge you. If you're trying to get along with a person from a proper Japanese family, you'll fail unless you know all of these and more. For example, placing bowls/plates on the table too hard, or not trying hard enough to pay the bill, not serving others, pouring your own drink...the list goes on and on. Most people think these things are silly, but some absolutely do not and will treat you accordingly if you're making these mistakes. Whether or not you care is up to you and the situation. This is all also true in almost every other culture, by the way.
Even expensive restaurants in Japan use disposable chopsticks. And you only get splinters on your chopsticks because you're rubbing them in your hands and making pieces break off.
In all my decades of using chopsticks, I've never had a splinter poke me. But I've seen people rub their chopsticks then complain about splinters.
I was really confused by this because I've spent about 6 months of my life in Tokyo and got very very very few disposable chopsticks at restaurants a tier above, like, shokken ramen shops.
But the internet informs me that the composite chopsticks that I am used to seeing went away during covid and now disposable wooden chopsticks are the norm.
I don't exactly know the system for which restaurants pull out of the disposable chopsticks but I think that for example "normal" tempura, katsudon, or like soba restaurants will tend to be those.
I almost associate the cheapo reusable plastic chopsticks with some food courts or Matsuya at this point.
There are the ones that are partly rounded and only attached for a cm or so at the top. They are fine. Then there are the square ones that are attached for half or more of the length and don't always break apart cleanly. They have never poked me, but they have shed bits into my food before that I had to pick out. I will stop cleaning up the ones that don't actually need it. I didn't realize it was offensive.
he he... is that the equivalent of when I was a kid we differentiated by "drive-in", "paper-napkin restaurant" and "cloth-napkin restaurant" in order of how much trouble you would be in if you embarrassed your parents.
This is a great point and the reason why I steer away from Internet drama like this. We simply cannot know the truth from the information readily available. Digging further might produce something, (see the Discord Leaks doc), but it requires energy that most people won't (arguably shouldn't) spend uncovering the truth.
The fact that we don't (can't) know the truth doesn't mean we don't have to care.
The fact that this tech makes it possible that any of those case happen should be alarming, because whatever the real scenario was, they are all equally as bad
I think it's more likely the introduction of the ability to say "fix this for me" to your LLM + "lgtm" PR reviews. That or MS doing their usual thing to acquired products.
reply