I love the slightly weird learning curve on NixOS where initially it seems incredibly complex writing this weird functional programming language for every change you want to apply to your system, then over time it clicks and you end up in a place where any operating system that doesn't define everything in a weird functional programming language seems incredibly complex. I read blog posts about doing things on other distros now and I'm left thinking that an 800 word set of instructions would be 25 lines of Nix expressions.
When I started self hosting last year it seemed insane that hosting a service required separately managing the imperative firewall state, vpn, container runner, systemd, and reverse proxy.
Nixos is nice because it just works. When I do a rebuild switch it will take down my affected services gracefully, apply the updates, update my firewall, and start the container all automatically and declaratively.
Plus the owning account and file permissions are explicit so it’s impossible to forget you ran chmod/crown on some script and now your service is over-permissioned.
It’s not magic but it feels like magic because it works so easily and reliably.
I've find my email on simple-nixos-mailserver for almost a half decade... So easy, less than 25 lines of code and a command line to update every few months.
it's like reading instructions for using a GUI program (open this dialog, click this checkbox, fill in this value) vs copying and pasting a complex FFMpeg command
When I was in college in 2001, I went to the library and checked out Kaare Christian's book called "The UNIX Operating System". One of the early chapters covered vi - I'd telnet into the school's Sun server with a pretty early version of vi (one-level undo) and follow the examples. Never looked back!
I've been using daisyUI for a couple of years and really love it - it's a quick, reliable way to rig up a nice looking/functioning UI with minimal work, while making things appear reasonably uniform across pages/workflows.
The combination of Tailwind and daisyUI made it possible for me - a backend developer - to pretentd to be somewhat competent in frontendland, which has been incredibly handy, work-wise.
I've been using Azure's "Document Intelligence" thingy (prebuilt "read" model) to extract text from PDFs with pretty good results [1]. Their terminology is so bad, it's easy to dismiss the whole thing for another Microsoft pile, but it actually, like, for real, works.
I don’t quite understand this - by 30k pages, is this the number of entries in your index? Did you mean 30M?
At the <100k scale I just full compute / inner product directly, and I don’t mess with vector stores or added complexity. No ANN algo needed — they’ll all be slower than actual exact kNN re ranking. (10k7684 =30MB, a scan over it and a sort is on the ~100us or faster). frankly, I’ve even sent at the 5k scale to client and done that client side in JS.
Often, I find i use an ANN algo / index to get me my nearest 10k then I do final re ranking with more expensive algorithms/compute in that reduced space.
The original HNSW paper was testing/benchmarking at the 5M-15M scales. That’s where it shines compared to alternatives.
When pushing to the 1B scale (I have an instance at 200M) the memory consumption does become a frustration (100GB of ram usage). Needing to vertically scale nodes that use the index. But it’s still very fast and good. I wouldn’t call it “dangerous” just “expensive”.
Interestingly though, I found that usearch package worked great and let me split and offload indexes into separate files on disk, greatly lowered ram usage and latency is still quite good on average, but has some spikes (eg. sometimes when doing nearest 10k though can be ~1-3 seconds on the 200M dataset)
Hi, I'm the author of the article. Please check out our vector search extension in postgres, VectorChord [1]. It's based on RabitQ (a new quantization method) + IVF. It achieves 10ms-level latency for top-10 searches on a 100M dataset and 100ms-level latency when using SSD with limited memory.
You're dealing with much larger datasets than I have, so far - mine is only a few million vectors. I have a hard constraint on resources, so had to get things working quickly in a relatively gutless environment.
I ended up breaking up/sharding HNSW across multiple tables, but I'm dealing with many distinct datasets, each one just small enough to make HNSW effective in terms of index build/rebuild performance.
The article suggests IVF for larger datasets - this is the direction I'd certainly explore, but I've not personally had to deal with it. HNSW sharding/partitioning might actually work even for a very large - sharded/partitioned - dataset, where each query is a parallelized map/reduce operation.
Hi, I'm the author of the article. Please check out our vector search extension in postgres, VectorChord [1]. It's based on RabitQ (a new quantization method) + IVF. It achieves 10ms-level latency for top-10 searches on a 100M dataset and 100ms-level latency when using SSD with limited memory.
What are you using for HNSW? Is the implementation handwritten? I’ve seen people using it well over XXm at full precision vectors with real time updates
pgvector - I wasn't able to get HNSW to build/rebuild quickly [enough] with a few million vectors. Very possibly I was doing something wrong, but fun research time ran out and I needed to get back to building features.
