I came across this recently. I was scanning for tiny models from HF using their search API. The script was generated by an agent. When I ran it, Qwen 3.5 did not make it at the top. Turns out, models generally prefer older content (training) but that the scanner also did not give any importance to recency.
I was learning Rust slowly when the LLM enabled coding became good enough. I switched from learning to full on building with Rust. I still learn high level concepts as needed but I will not be able to write Rust on my own at all.
And that sounds scary but the way I got over the fear is by realizing there are many things that I do very well but I do not know their internals very well. Driving is an example. I barely understand what the steering wheel, clutch or brake pedals do. I have driven over 130,000 Kms and I will perhaps drive more than double that in the next many years.
I have been building software since PHP/Drupal days. Got into AWS S3 as a beta user. Adopted Memcached (and MQ) in 2008 out of necessity. Then Python/Django for 10 years. Then Rust. And tons of JS/TS. I owe a lot to my curiosity. I believe we can keep learning what we need and still delegate most of programming to agents.
There are two types of programmers: the pragmatists who see programming as a chore and would gladly never write a line of code again given the right tools, and the gardeners who don't want their enjoyable and rewarding garden-tending work taken away from them.
The "pragmatists" who get excited developing a prototype for a week before they realize they will never be able to ship something anyone else will use because each trivial change becomes exponentially more difficult for the LLM to implement and completely impossible for the "pragmatist" to reason about, with every new commit liable to break something else.
Still waiting for this revolution of amazing 10x software! It's been 10 months since Everything Changed in November, surely the 10x pragmatists could have leveraged their effective 8 years of development time? Or maybe we'll move the goalposts again and say that actually, Everything Changed with Astra, we'll just need to wait another three months?
I am not an expert in this domain but as an engineer-turned-researcher, this looks a lot like GliNER with a fitting harness.
This is something I focus on in a bunch of my experiments - how to get immense value out of tiny models (<1b params). There are lots of different architectures out there and there is so much to optimize if you know what you are asking and have a grammar to constrain with.
Great to see this and I hope this is a lot on top of what is already openly available.
More and more such experiments. I felt sad for a couple months when I realized that writing code will not be the same since. Now I am on the other side.
LLMs are interesting in their own ways but as an engineer, this is a way to unlock a new way of building software.
I recently build a Claude-assisted Excel/CSV parser for a US based property management system (tax compliance). Uses Haiku and has a lot of deterministic code to extract column/row combinations to check known formats and finally handing out the headers to Haiku to give us a translation plan to our support columns.
These would eventually become part of the software, in a tiny LLM. The gap between training (such tiny LLMs) and inference will shrink. We can consult Claude for edge cases, create sample dataset and train a the tiny LLM on demand so we go to Claude less.
The tooling that a project needs is really important. Something I have been feeling as well. Not just in LLM building projects, but regular software projects that are LLM generated.
I think I am on your arc as well. My learning on different topics is growing every day, but there’s a limit to how much I can absorb. With the LLMs the experiments stay just beyond that horizon and I keep chasing.
Stated too strongly, but I think this could be the model for education (some subjects anyway). Everything personalized to your learning goals, grounded in experiments that give a tight feedback loop and with a model that never gets tired of re-explaining something for the 10th time.
Yes they are great for learning at own pace, trying out new things.
I have accepted two things that make me a happy engineer now: AGI is not here no matter what they say and LLMs are still very useful if one knows how to use them.
They are another layer of abstraction and like you said they do not tire. There is a lot of optimization needed so we can reduce wastage (running 1T+ LLMs for most work is wastage).
Haha, I thought about this today. I'm traveling and walked through a university campus. The new students are coming in and I thought wow, wouldn't it be wild to be a fresh in student in the age of AI (it's been a long time since I was college). But, then I thought, actually this could be an incredible time to be a student. If your curious and motivated the agents can be amazing partners. I think the students who crack how to use AI now are going to be the ones who really run the table as this economic wave crests. But, yeah, you're right on with the Wall-E comment. I would guess many (most?) will turn their brain off and then what's the point of even being there?
When we hyper focus on finding something, we find it all the time.
The world of tiny LLMs is so interesting. It is unlocking novel ways to encode information. Why focus on the style of writing instead of the subject matter?
This is a tiny LLM doing all the heavy-lifting. Any mention of the training process? I am obsessed with tiny LLMs and the do-one-thing-really-well approach that they seem to fit very well.
I experiment a lot with local LLMs, particularly small ones like Qwen3.5 4B and 9B. I have build multiple experiments to make harnesses that use these models for code generation, planning, local search, etc.
These are really good models but the harness has to be built around them. I have a ton of generated system prompts for specific purposes. Even parts of a SolidJS stack, for example Route management, has its own prompt. These are experiments but the results are real. If we build harnesses around small models, we can build a locally running WYSIWYG editor which works on plain text prompts.
The performance, in simple tokens/second, is not the most important factor. For many private data points, like emails, I would rather have a local graph based search and LLM on top where the harness is specific to problems like calendar, contacts, finance, etc.
I run all experiments on an 16GB M4 Mac Mini but coding agents building the harness are a mix of Codex, Claude Code and opencode.
Somewhere around early 2000, I used to work part-time with a local video editing agency. Matrox video capture cards were the really high bar, other than Avid. At least that is what I remember. We had a couple of Matrox capture cards, expensive for our tiny budget. Other than that and a few 3D cards also around that time, I forgot about Matrox.
I do not think Cloudflare was a less-than-peers optimized product when they launched. This is one of their blog posts which describes taking one aspect even further.
I think Cloudflare became big only because they were so much more optimized than others that they offered some services for free that others were not offering. If running costs are high, you only burn (VC) cash and then you exit.
There is the entirely plausible option of the NSA indirectly bankrolling them to counteract the growing number of TLS connections. We terminate your TLS for free in our server and you don't have to change anything was a sweet deal for website operators _and_ those that want to look at unencrypted traffic. Given enough sustained funding they could undercut competitors and grow to what they are today.
I use Claude Code, Codex and opencode pretty much interchangeably. I am currently using Claude more this month because (stupidly) I paid for Max ($100) since I have a large client project.
I barely spend time or thought in making prompts. I have a simple text file with a few combinations. They refer all the common files (README, AGENTS, DEVELOP, etc.)
All reference software is cloned locally and the docs mention that. The prompt templates then boil down to research mode (write Epic) or worker mode (write software) or review mode (leave review notes in Epic). That's it.
Many of my harness experiments are about text manipulation, text search, graph on text. Because that is what LLMs are - text processing systems. Cut parts of prompts, cut parts of response, cut parts of user's intent. Join, break into epics/tasks, run with LLMs, repeat.
reply