IDEs have changed a lot in the last 50 years. Just like we shouldn't advocate for hand writing assembly for all code, we shouldn't be stuck using CLI tooling the same way.

I share your apprehension regarding the current AI landscape changing so quickly it causes whiplash but I don't think a mindset of "it's been fine for 50 years" is going to survive the pace of development possible by better LLM integration.

The reason that tools have not changed that much is that our needs haven't changed that much either. Even something like `find` or `ffmpeg`, while complex, are not that complicated to use. They just require you to have a clear idea of what you want. And the latter is why most people advocating for LLMs want to avoid.

IDEs have not changed that much. They've always been an editor superchaged with tools that will all share the same context of a "project". And for development, it's always been about navigation (search and goto), compile/build, and run.

Many of the changes that would work for LLMs would also be beneficial to users.

Not at all. The shell already provide us ways to get contextual information (PS1, ...). And the commands generally provides error message or error code.

In one of the example provided:

  $ sdfsdf
  zsh: command not found: 'sdfsdf'
  zsh: current directory is /Users/ryan
  zsh: Perhaps you meant to run: cd agent_directory; sdfsdf

You could just use `pwd`, like most people that put the current directory in the $PS1 to make sure that the agent stays in the correct directory.

Yeah, this example isn't great - you can just tell the llm to run pwd more frequently or something.

But for the `$command | head -100` example, the usage is a bit different. I run into this myself on the cli, and often ended up using `less` in similar context.

Two cases

1) sometimes I use head to short circuit a long running, but streaming output, command so I just assess if it is starting to do the right thing but not bear the time/computational cost of full processing

2) sometimes the timing doesn't matter but the content is too verbose, need to see some subset of the data. But here head is too limited. I need something like wc & head and maybe grep in one command line with context. Maybe something like

$command | contextual-filter -grepn 5 -grep error -head 10

some data ... first the first 10 lines ... an error message with 5 lines of context surrounding before and after

Summary: 100000 total lines 15 printed exited with code 0

You can do all that already with grep and others, but you need to run multiple commands to get all the context

1) That's why some tools have a simulate option, or you can just do a kill 9 on the processes you've just launched. Just make sure you've capture their output in a file

2) Again logs, if actions needs to be taken after the command has stopped. For immediate action, you can use `tee`.

Managing context isn't hard. I see more issues with ensuring the right command.

What man page? I have never worked on a product with one. We're not teaching the LLM how to use `ls`; we are talking about the code being written today.

edit: Mea culpa.

> I think watching the agents use our existing command line utilities get confused and lost is a strong indicator that the information architecture of our command line utilities is inadequate.

Seems pretty clear the article is talking about teaching LLMs how to use 'ls'.

> The agents may benefit from some training on tools available within their agents. This will certainly help with the majority of general CLI tools, there are bespoke tools that could benefit from adapting to LLMs.

Definitely not just `ls`.

I also feel like command line agents are pretty simple. It's tailor made for tool-use.

while(true):

>> User requests something

<< The LLM picks a cli tool from an index

<< LLM grabs the manual for the tool to get the list of commands

<< Attempts to fulfill the request

I would not be shocked if engineers have already managed to overcomplicate these agents.

You can pretty much obviate that with an alias that catches the user requesting something then operates deterministically. What is nice about aliases is you don’t need to learn other peoples semantic patterns, you craft ones that make sense to you and your use cases then they always work and consume virtually no resources to work.