This kind of spam was already destroying the web before LLMs came along. Now it’s being accelerated by thousands of times.
Mainstream web search is probably cooked. Kagi and other niche players might have a chance if the fact that they are not beholden to advertisers lets them introduce features to do things like downrank content with ads. Kagi has “small web” which I think includes this in its weighting.
Open social media is probably cooked too. In the future it’s going to require proof of human identity and will be more heavily moderated.
The future of social is closed forums. Even those are really having to fight bots though.
The other pervasive awful trend of the moment is everything becoming like John Deere tractors: cloud connected, DRMed, with subscriptions and/or planned obsolescence.
Capitalism is supposed to reward people for creating value, but today it seems like it’s far easier and more profitable to just extract rent or scam. I am not sure how to fix this.
> Even those are really having to fight bots though.
We recently released a closed-system, iOS-only app, that has a fairly rudimentary, privacy-first system of user registration. It is currently restricted to the US and Canada.
Each signup request is manually vetted. There is no automatic registration. The app is designed for a specific demographic, and we do our best to ensure that new accounts are real people, that fit the demographic.
This is an iOS-only app (free), restricted to the North American continent, and with no accessible server API. The server is a bespoke server, and has no connections or dependencies that we don't control.
We are flooded with bots, and, most likely, scammers. So far, they have been pretty easy to spot, but that could change.
I think we'll end up moving away from shallow signifiers of trustworthiness to reputational networks and evidence someone has invested into an identity or entity.
That's not to say it's a solved problem, even in the real world with thousands of years of battle tested strategies.
A very simple example would be a web browser where I could blacklist chronically-unhelpful sites, and share that metadata among friends.
1. People get on the web and make real content with care because they're just excited to share stuff with each other.
2. Advertisers talk them into putting some ads on their pages so they can get some compensation for their work.
3. Shitty people figure out you can just make content where the main incentive is to get people to go to the page and see the ads.
4. Those people then outsource writing the content to the lowest bidder.
5. The lowest bidder becomes an LLM.
6. Search engines cut out the middle-man entirely and just send your search query to an LLM, stuff some ads in, and show the result to the user without ever hitting the web (except to periodically scrape it for model training).
7. Because of 6, people stop putting new content on the web at all. The models get shittier and stupider with regards to current events.
8. To counter that, LLM companies make deals with news organizations and other primary source information provides and pay them to have direct access to content to train their models.
9. Those organizations get such a large fraction of their income from those deals that eventually they get out of the business of giving human readers direct access to it because it's not worth the effort. Newspapers become B2B companies.
10. The only way to get information is via a handful of giant tech companies sitting on top of huge LLMs saying who-knows-what trained on a slurry of actual information and giant piles of ads.
I hope that somewhere in the process people start to get tired of talking to machines all day and hop off the ride entirely and starting calling up their friends and getting information the old fashioned way.
The only consolation I have is the belief that people have a deep seated desire to connect to actual humans and know the real truth about the world.
IMHO a major part of the problem is that the Internet never had a mechanism for paying for good content. Everything is “free” therefore ads emerge as the only monetization strategy and you did a good job outlining the rest.
I’ve started trying to pay for good journalism, especially good indie journalism. I also Patreon a bunch of podcasts, buy high quality software if the price is reasonable, buy albums of my favorite music, buy films, and so on, while actively avoiding both gratuitous subscription models and the ad web.
Pay for it or it either doesn’t get made or it pays for you. Free is a lie and piracy undermines quality.
Edit:
All the paying for good stuff I outlined above averages out to around $100-$150/month. It’s less than I usually spend on restaurants and coffee shops and far less than groceries for our family. Restaurants in particular feel like a far more frivolous expense.
You're assuming that LLMs will boost spam and scams more than LLMs will tone them down by marginalizing them automatically. I have the opposite view. SMTP spam used to be mostly unavoidable, then one day it was not.
IMO, the number of engineers and moderators needed to offset one scammer is about to take a huge dive.
Your analysis overlooks a massive cost asymmetry[1] in favour of the spammer. THey only have to pay the LLM cost once to generate a message which they can use thousands/millions of times versus the receiving side would need to pay an LLM to check and classify each incoming message.
[1]Either you pay a SaaS LLM provider or you pay the cost of compute to run the LLM yourself
My strategy been to put my money where my mouth is and start paying for services that provide value to me (Kagi is one example - I’m a paying customer, and actually found this article using their small web site)
Mainstream web search is probably cooked. Kagi and other niche players might have a chance if the fact that they are not beholden to advertisers lets them introduce features to do things like downrank content with ads. Kagi has “small web” which I think includes this in its weighting.
Open social media is probably cooked too. In the future it’s going to require proof of human identity and will be more heavily moderated.
The future of social is closed forums. Even those are really having to fight bots though.
The other pervasive awful trend of the moment is everything becoming like John Deere tractors: cloud connected, DRMed, with subscriptions and/or planned obsolescence.
Capitalism is supposed to reward people for creating value, but today it seems like it’s far easier and more profitable to just extract rent or scam. I am not sure how to fix this.