Interesting that for these small models, it is optimal for the embedding paramet...

		highfrequency 8 months ago \| parent \| context \| favorite \| on: Gemma 3 270M: Compact model for hyper-efficient AI Interesting that for these small models, it is optimal for the embedding parameters to be a huge fraction of the total (170e6/250e6) = 68%!