Could there theoretically be a swappable hypervisor for oxide some day? I'm imagining a oxide Linux distro of sorts running on the hosts, and using kvm for vms, but still oxide control plane bits downstream. Then you get nested virt and gpu support, and could maybe even virtualize a traditional oxide host. Like running multi-host vcenter clusters inside a single esxi, amazingly powerful lab scenarios, teardown is even fun
Switching to Linux and KVM would gain the ecosystem benefits but Oxide would lose control of the host OS which can impact our security stance. Better to tightly integrate at this layer and expose the primitives that are needed. It's not impossible, but we're already maintaining the existing Helios host OS. Adding another wouldn't be beneficial.
Yeah me too, unfortunately I cant justify huge 100K USD servers so ill continue with cluster api provider hetzner and metal3. The whole CAPI ecosystem is quite cool.
What if this project offered binaries from GitHub releases instead? How do you read those?
It's more realistic to decide if you trust the source, GitHub user yamafaktory in this case. Then you can ensure that your method of download uses TLS.
It already does. The releases page has prebuilt binaries for four targets plus a SHA256SUMS file. You can download the tarball straight from GitHub over TLS, verify it against the checksums, and run it. No script involved.
If you run an ubuntu image with a kata kernel, do you have difficulty installing some packages? For example if you need linux-headers-$(uname -r) or linux-modules-extra-$(uname -r), it won't exist in canonical apt repos?
I was pretty disappointed to discover DGX spark has no BMC. I was able to hook it up to pikvm by connecting leads to the power switch wires. It honestly works really well and I can provision it with MAAS
Shout-out to pikvm for their redfish support with the pikvm switch
Great writeup there are quite a few here that are new to me
I feel like the shape of the market right now for "home lab" inference is:
The sparks are good if your ultimate plan is to spend even more on NVidia hardware in future to run your dev setups at usable speeds. Or, you're developing for a work cluster.
If you mainly want to run local models at acceptable speeds portably, buy a mac with lots of RAM. If you’re happy with non-portable / racked, buy 3090s (dense) or mac studios (MoEs). Buy newer cards if you are restricted on power or slots. If you are rich, buy a6000 blackwells.
Has anyone that has set up microceph determined the overhead of the required multiple OSDs? The docs make it sound scary, but it's not clear if that's because people run it on a Pi with an sdcard for block storage or because someone once ran 18TB of OSDs in production that then fell over.
I do continue to be impressed/ over-awed by how effectively scared the Ceph docs are about just how many system resources you need. To run a mid tier not that fast storage cluster. Bother.
Impressive as hell software and I am so glad to have it. But man! The insistence on mountains of ram per TB, on massive IO is intimidating.
How is changing the architecture of a platform that only you make hardware for doing the impossible?
They could change the architecture again tonight, and start releasing new machines with it. The users will adopt because there is literally no other choice.
Every machine they release will be fastest and most capable on the platform, because there is no other option
Exactly this! Rosetta + the whole app developer community who really quickly released builds for M chips (voluntary or forced, but it did happen).
I had the initial m1 air, and it was remarkable how useable it was. You'd expect all sorts of friction and issue but mostly things just worked (very fast). Even with some Rosetta overhead it was still fast compared to intel macs.
Rosetta 1 delivered 50-80% of the performance of native, during the PPC->Intel transition. It turns out, you can deliver not particularly impressive performance and still not ruin your app ecosystem, because developers have to either update to target your new platform, or leave your platform entirely.
You can also voluntarily cut off huge chunks of your own app ecosystem intentionally, by giving up 32bit support and requiring everything to be 64bit capable.
...because users have no other choice when only one vendor controls the both the hardware+software. They can either use the apps still available to them, or they can leave. And the cost of leaving for users is a lot higher.
Yes. Apple put custom hardware support in the M series chips based on the needs of Rosetta 2. The x86_64 performance on Rosetta 2 was often higher at launch than the prior generation of Intel chips running those same binaries natively.
Microsoft and Qualcomm already knew the performance of x86 app emulation on windows was killing the ARM machine lineup, so Qualcomm was working on extensions to their chips and Microsoft on having Windows support them already, but ARM64EC and Prism didn't launch for two years after the M1 shipped.
