> Ruff is fast, sure, but the benchmarking seems a little disingenuous, as I believe their number includes caching, but doesn't necessarily include caching for other tools.

It looks like the ruff benchmark is run with the `--no-cache` arg: https://github.com/charliermarsh/ruff/blob/main/CONTRIBUTING...