For those who a) use Proxmox to b) manage containers c) some of which are of ostype: unmanaged (e.g. those who run a virtual OpenWRT router, like me...) be warned that the current version of Proxmox will not start those containers due to a bug/an omission in the PVE::LXC::Setup package (/usr/share/perl5/PVE/LXC/Setup.pm) which makes it attempt to call a non-existing method to configure cgroupv2 support on such containers. The following patch will work for now while the problem has not yet been solved in the repo:
My absolute wish is to run Proxmox on a laptop and have a VM drive the laptop display.
Currently this is not possible.
There are many guides on passing through the integrated or dedicated GPU, but NOT using the laptop display in a guest VM. The host/Proxmox doesn't need the display. I want the guest to own it. No looking glass no spice no gl window no virt-gpu. The VM should directly own and exclusively operate the gpu and laptop display.
My dream setup is Proxmox being very minorly configured - a stable base that's easy to restore. Within it 5 VMs: firewall, primary-linux, rescue-linux, linux-for-containers, and Windows 10.
I recognize most folks are running Proxmox on a desktop but I have a very nice laptop. I want to isolate my firewall into a guest VM (opnsense). I want to run my primary Linux in another guest VM - with the Intel iGPU passed through and using the laptop display. I want to run Windows - when I need it - for games that intentionally disallow VMs. I want to run a separate Linux (Rancher) for containers/development. Finally I want a rescue-linux so if the primary fails for whatever reason, Proxmox would fail over by starting the rescue VM - just something with enough for me to remote into the host/proxmox and fix things. Windows would be viewable from RDP or looking glass. The container Linux would be ssh-administered. Same as firewall or web admin.
I LOVE the idea of Proxmox making it easy to backup/transfer/experiment with different OS's.
Because I can't get Proxmox to drive the laptop display - something not possible on AMD or Intel afaik - I settled on NixOS as the host with explicitly configured everything. Windows in a VM. Docker available on the host. It's the best I can do :(
Someday I hope hypervisors-against-metal on laptops is commonplace.
Seriously: If someone can get the laptop display to work from a guest VM on any laptop I would be so happy. I hope AMD gets this because I am so done paying for baby steps with Intel.
I think this is how Qube OS works, but I haven't tried it personally so can't be sure. Perhaps check it out - it might be what you're after? https://www.qubes-os.org/
I assume the intention is to run only a bare-bones OS + Proxmox directly on the laptop with all the other functionality virtualised or compartmentalised through KVM and LXC, managed by Proxmox. Running the desktop directly on the hardware does not fit this profile.
I'd say this - having nearly everything compartmentalised with access to the laptop display - might be possible by using containers instead of virtual machines. This would not work for Windows but it could probably be made to work for those mentioned Linux installs.
This sounds pretty cool, but for now the closest thing I’ve tried is Qubes :( being able to actually pass the GPU to a VM would be great. Certainly it is technically possible? Maybe not…
Running some games in a VM within VMware or Virtualbox is detected as such. Running a game in a qemu VM there are some options available to hide that the hardware is within a VM - like how the CPU identifies itself. I'm aiming for the setup where the GPU is passed through from the host for exclusive control by the VM. I honestly don't need this RTX2060 except for games - the Intel iGPU is excellent for webdev.
I have an Intel iGPU and a GTX 1070 myself, and currently just dual boot to separate SSDs (the larger SSD for Windows, the smaller for Linux).
When you switch to the Windows VM, do you intend to suspend the Linux VM, or you just want to let it run in the bg?
Also, couldn't you just run the VM in Linux and get the same benefits? Some people are passing through their graphics cards and reporting good performance IIRC.
You're not alone my friend, I also have my old but powerful gaming notebook as a Proxmox host, and I'd very much like to be able to bring it to LAN parties and just boot a Windows 10 and play Age of Empires II while maintaining all my VMs intact.
Can anyone comment on update/package stability when using the free version of Proxmox. It's my understanding that only the paid license allows access to stable repos. I find the community tier price a bit too expensive for my current home use case (mostly nas + plex).
Been running the free version for a couple of years now. No problems at all. I update and reboot it maybe once a month, or whenever there's a severe security update (that I don't miss). It's uneventful.
Same. I’ve been running it a little over 2 years for home/family services and it’s been entirely stable for me. I update slightly less often than parent comment, but have had zero issues from updates.
Likewise, have run it for years on a bunch of machines, never had a problem. I think I've got at least one that's been dist-upgraded all the way from a 4.x release to the current 7.0.
We run Proxmox for non-production-critical VMs. Nothing fancy, just Proxmox hosts, iSCSI storage and standard VMs on that (Windows and Linux).
Every now and then we update hosts using the No-Subscription Repository. No issues from that so far.
Maybe if we updated more frequently we would see more issues. I think we only do it once or twice between each new official version release. That's out of laziness/lack of time, not out of fear of issues from less tested packages.
Have been running it as a Debian system (realized I don’t need Proxmox, but already had some stuff set up) for about a year and had no idea. Absolutely no problems.
this seems like such a really good baseline for a home-cloud. there's such good host infrastructure here. i'm such a huge Debian lover but there's so much more required to turn it into a decent host environment, and Proxmox covers so much ground to do just that.
but i still want Kubernetes as my primary interface atop it.
since LXC is the primary container system on Proxmox (and a damned fine one at that), it seems like one might be able to make something like LXE[1] run, as a shim, to provide a Container Runtime Interface & start running kubelet (the thing that runs containers) atop that.
Just out of interest, why do you want to use Kubernetes in a home cloud environment? Given the limited scale of such an environment I don't see the need for something like Kubernetes here, just have those containers managed by Proxmox and be done with it. That is how I use it, with task-specific containers (auth, mail, (data)base, a few build environments, p2p, media, reverse web proxy, router, etc) directly managed through pct (Proxmox LXC CLI).
i.e. I can easily bring up a set of pods in kubernetes that have health checks, such that kubernets will kill a container and restart it automatically if the health check fails.
To do that in proxmox with VMs is a lot more work. In reality, at minimum, proxmox should have the ability to provision a kubernetes environment by provisioning a set of VMs and building a kubernetes cluster out of the VMs, even if it didn't provide a web UI to it. (think GKE or the like). It should enable you to scale up/down and use the proxmox storage for volumes. this would be a MVP for most people to be able to use it at a small scale. One would then want to start work on how one can control ingress into the kubernetes cluster in a standard way.
A lot of people don't see. I think that trying to turn a blind eye to complexity is a foolish mistake. There's so much you're given, that helps so much with out of the box Kubernetes. Logging & load balancing/services, and gateway/ingress, and daemon-sets and... how convinced are you that you understand these features well enough to confirm that you don't & wont need them?
It also sounds exhausting to me to try to start from the basis that's what's good enough for everyone is too much for me. That, because we are scared of complexity, we're going to go bake our own systems, go back & re-assess less-popular less common less comprehensive options.
And how happy are you knowing that the ops you do will be unintelligible to 90% of engineers? To me, getting to participate & share & work with other cloud native projects, to get to explore pull in the future, as it is happening, is an option I would give up for nothing.
I also highly recommend this other thread I have, on how home-computing has never ever had a basis to stand from before, and how now that the computing community finally has it's first chance ever to grow good at ops together[1].
But I see enormous vast huge waves of radical retro-conservatism. "I don't know if I need Kubernetes" is a critical mindset that I think is worth assessing, but honestly, 99.999% of the time, you should use Kubernetes. It works fantastically well at all levels of scale, you can decide as you go what pieces you're going to want to make use of, there are super easy get-going-get-containers-started resources, you will have far far far more chance to bring in & use potentially helpful emerging technology. Please everyone: don't fucking decide to hike the fuck off to your stupid ass cabin in the woods computing environment cause you're all "I don't think this is needed." You are punking yourself. Asking what you need is a question you can harp on for a long time, but the risk of starting with the good comprehensive, proven, well-integrated, robust, scalable (low and high), known, well used/well liked not hard to start with default option is- I think- a low enough that you should probably just give it a spin.
I was referring to your specific use-case. I can go to k8s’ website to check its features, no need to list them here. It seems you are trying to create your own SaaS/PaaS to manage your home infrastructure. However, I still don’t see k8s as proper tool for this, because the scale is not big enough for such. Remember, k8s was designed with the Borg philosophy. But maybe you have a special case, so by all means, go for k8s.
I don’t think anyone is scared of complexity. However, not everyone has easy horizontal scalable web applications. And most of time one just wants environmental organization and isolation, which containers/VMs brings. In home clouds I would see vertical scaling as a critical feature for efficiency in the limited infrastructure that cannot be easily scaled up. And today, k8s vertical auto-scaler isn’t anything but a disguised horizontal auto-scaler.
Proxmox isn’t competing against Kubernetes, because by design k8s doesn’t support containers that run anything other than a single application/process that is supposed to be easily killed.
If anything, LXD is the tool now competing with Proxmox due to its recent support to Qemu virtual machines. Although I don’t see what super advanced use case you have, I agree it is okay to run k8s on top of your Proxmox cluster. And you should check LXE limitations.
