Agreed... there's often other perspectives people never thought of like this, which is why they say "strong opinions about issues do not emerge from deep understanding."

Even if you're just inexperienced in the language you're communicating in and are trying to have better conversations, it's very helpful.

For cases like that, I say just don't tell people... I think it's unlikely anyone will be able to tell either way.

If you regularly have to deal with hung connections or slow/unreliable links, I suggest trying out mosh.

https://mosh.org/

The trick is getting people to believe you.

As a rule, strong feelings about issues do not emerge from deep understanding.

Which patch are you referring to?

They're not forcing you, you can register as a conscientious objector.

A 10GB 3080 still beats even an M2 Ultra with 192GB... memory bandwidth is not the only factor.

https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...

If the model is small enough to fit in to 10GB of VRAM the GPU can win.

But the bigger models are more useful, so that’s what people fixate on.

with regular llama.cpp on a 3070ti I get 60tok/s TG with the 9B model, it's quite impressive.

Don't sleep on the 9B version either, I get much faster speeds and can't tell any difference in quality. On my 3070ti I get ~60tok/s with it, and half that with the 35B-A3B.

wow I had no idea you could do that. this changes everything for me.