Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

with regular llama.cpp on a 3070ti I get 60tok/s TG with the 9B model, it's quite impressive.
 help



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: