... because that's literally the IOMMU's job? Why should we trust the TPM or the CPU or a YubiKey or anything, really? I don't completely trust any of it but to get anything done you have to trust something at some point.
Nice. I've been looking at doing something similar, more on the order of running a 1T model with less than half the available VRAM.
One workup indicated it was theoretically possible to modify a piece of SGLang's routing layer to support JIT predict-ahead expert swaps from Gen5 NVMe storage straight into GPU memory.
I'm hoping that proves true. The setup relies on NVIDIA Dynamo, so NIXL primitives are available to support that.
That would be nice to see. Actually I was thinking about getting another 3090 and a mobo upgrade since I'm bottlenecked by pcie3 to tryna run glm 4.7 or 5 at q4_k_m, it should be possible.
Odd reply, but OK. For what it's worth I largely agree with everything else you said.
>>The problem, to me, is deeper and is rooted in our education system and work systems that demand compliance over creativity. Algorithms serve what Users engage with, if the Users were to no longer be interested in ragebait, clickbait, focused on thoughtful content -- the algorithms would adapt.
Technically that's true. Thing is, the UI/UX isn't built for long-form content. The platform, interface and algorithm when taken as a whole represent more of a dopamine delivery system heavily biased towards short-form content.
That dynamic in turn ends up being deleterious to cognition to the point it ends up fighting any external factors that which could change user behavior for the better.
In other words the algorithm is part of a larger format, and that format is arguably the real drag. Of course, the algorithm being properly transparent and accountable to its users would certainly help.
I think we're relatively aligned. But you're sharing another chicken & egg problem of whether the algorithm (and engagement with it) is driving the design of the feeds or the other way around.
Arguably, the initial design was a shot in the dark and they're approaching some local maxima with data-driven design trying to improve metrics that we probably all agree aren't the best for our mental health or wellbeing.
Nice chat, apologies if my response was off-putting. It was intended to be self-deprecating humor.
>Arguably, the initial design was a shot in the dark and they're approaching some local maxima with data-driven design trying to improve metrics that we probably all agree aren't the best for our mental health or wellbeing.
Yeah, the way I look at it is product managers and everyone above them in the reporting chain make more money for their respective companies the more they optimize short-form content delivery. Pretty much what you just said.
So, what we're left with is a hyper-optimized content pipeline over the years that's pretty rough to get away from when quite a large number of people are already accustomed and/or addicted. In other words it's really hard to close up Pandora's Box again, but fortunately not impossible.
>Nice chat, apologies if my response was off-putting. It was intended to be self-deprecating humor.
No worries, wasn't sure and didn't want to read into it wrong. Wasn't trying to be snarky on my end. Cheers.
"It's a test - designed to provoke an emotional response. "
I was going to follow this with something like "except the role of analyzing the emotional response is reversed", and then I wanted to expound with an "ooh but.. wait, there's another metaphor here since ..." but thought I've already potentially approached "spoiler alert" territory so I'll just stop there. Those who know the reference I am replying to will know; those who don't, well, don't google any of this or its parent cuz spoiler alert
Apparently there's already a few projects with the latter name.
reply