Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Tokens are also numbers in practice, but they're indexes into a lookup table of character sequences so yes there's very little between the two definitions. Embeddings are in turn the result of looking up that index in a table, and the result is a vector. So:

character sequence (string) -> token (small integer) -> embedding (vector of floats)



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: