I think we should aim to move these security violation rules away from the prompt, to a deterministic place. Not sure how your orchestrator works, but is it possible to add a check between the agent's decision and its execution? e.g. the agent decides to read a file, that decision goes somewhere that checks if the agent has permission to do that or not before it actually reads the file.

Totally agree and this is actually on my short list. I use opencode, so one will need a custom plugin to gate tool calls. They have permission config, which I'm using to block file reads, but the orchestrator is allowed to use specific subagents for more targeted reviews, and they have file read permissions, which the main agent is abusing.

Some of them can be deterministic rules, but others cannot. For example, if you want to permit the GH cli for adding comments, but not merging...

1. GitHub has not provided granular enough tokens

2. You can wildcard in the opencode config

3. The agent can work around this with bash, if it has access

4. The agent apparently will also use subagents, who do have the permission, to work around it's own permission limitations. (This is the problem I'm actually facing)

This pattern is the "relentlessly proactive" as Simon Willison calls Fable, or "artificially incessant" as I called it this week (kimi in this case). It's the same training that enables the long-horizon task completion and mythos style hacking, double edged sword.

I think it unlikely we can block all avenues with deterministic only tools. I'm also looking at policy tuned micro llms, and then creating a merged 'or' signal from the various checks. Later I can look into loosening the signal if there are too many false-positives

This is interesting. Thanks for sharing more. Looks like it's a trade-off of it being relentless, which is something we want in some cases. We need to figure out a way of closing the door and at the same time signaling the door is closed so it does not keep trying different manners.

I've been thinking about this but in a different context: non-coding agents. e.g., an AI agent that approves travel expenses is not allowed to approve expenses bigger than X USD (no matter what). In this case, it looks closer to an ACL thing.

take a look at ADK, they have features for exactly this, mixing determ with agentic in a dag

You can't enforce everything with a deterministic layer. Some things have to be communicated as a policy and tactically remanded on trigger.

Every relation will probably go down that path. Building trust will be interesting.

> Then I compared the implementation, for correctness, to other systems, finding that other implementations sometimes contained more errors. I researched more, and found that the local inference world is full of subtle errors that accumulate and damage the model output, issues in the attention implementation causing performance slopes after the context is over a certain limit because indexed attention implementations are broken (do more work than they should, for instance), and so forth.

I agree 100% that AI helps a lot with that. But I feel like there's something missing between "AI helps a lot with that" and "I believe reading code is mostly pointless". I genuinely wonder how the above can be accomplished without reading any code.

> I genuinely wonder how the above can be accomplished without reading any code.

We can't see the code for the laws of physics, and yet experiment by experiment we've come a long way.

That's not usually going to be the most efficient way to understand code, but I've found that it's pretty useful to be one of the few empiricists in an organization full of rationalists (software engineers are typically rationalists in my experience). I wouldn't propose that we dispense with rationalism altogether, but it does seem that the we'll benefit by achieving a better balance than we've traditionally had.

> We can't see the code for the laws of physics, and yet experiment by experiment we've come a long way.

Interesting perspective. Haven't really thought through that lens

I write 2% of code. And read 5-10%.

I feel like everyone is being too black and white. And everyone has different ways of staying in flow with a software task.

Yeah, totally

Having looked at his code, I doubt this.

Yeah antirez made a lot of big claims in that paragraph. Sounds like a case of AI psychosis.

he said he read it though, and check the last line

The idea that you need to motivate yourself to do a thing you don't want to do is an idea that needs deeper investigation. I've caught myself trying to do that a bunch of times. Why the hell I think I need to do this thing in the first place?

I totally get things like I have a job and there's a task that needs to get done. But what about outside the job life?

Fitness is hard because there’s very little to zero immediate benefit particularly if you are out of shape. I also think for many things, particularly those related to lifestyles and behaviors, the goal is to do it often enough so that it becomes a habit. Most people probably don’t enjoy brushing their teeth, but we also don’t think much about it, we just do it. I feel the same way with running, I don’t think I enjoy it, certainly not all the time like in winter, but it became a morning routine that is easy to execute.