> Then you send the patch upstream, they incorporate and maintain it for you
Firing patches upstream is still adding burden to the (likely already over-burdened) maintainers.
In an ideal world, if you want a patch upstreamed, you would be contributing to upstream maintenance (or at least donating to the upstream maintainers)...
Fair, but it is less of a burden than just submitting a report with no proposed fix. Also, submitting quality patches regularly seems to be a good way to eventually become a maintainer, provided that both sides are interested (cURL generally is – at least that seemed to be the vibe at the last year's cURL Up event I attended).
I believe both are valid: sometimes upstreams are not set up for donations, and sometimes your org will make it easier to submit a patch or to financially sponsor a maintainer.
A cache line is simply the unit of data a CPU cache works with (generally 64 bytes, because someone somewhere has probably determined that that is the best line size for general use), much like there are units of data like bytes (8 bits nowadays, but there have been weird ones historically), pages (varies between hardware as well, and may be OS-configurable), etc.
As TFA mentions, a CPU does some predictions about what cache lines to prefetch, e.g. when you do sequential reads. Moreover, the x86_64 instruction set provides a prefetch instruction through which you are able to give the CPU a hint "hey, I'm gonna be using this soon, prepare accordingly, pretty please".
Still, the utility of prefetching is diminished if you only use a single byte from each cache line, because the mechanism generally depends on you doing other work while the next cache line is being fetched. So really the best case scenario is to take as much time as possible to work with what is already fetched, so that there is time for the next unit of data to be fetched in the meantime.
Pretty sure if you let only a handful of individuals from an almost-extinct species roam around freely in an uncontrolled environment, chances are pretty high something is going to kill them off before they reproduce, hence why they are almost-extinct.
The zoo provides a controlled environment needed to restore the species.
Also, careful breeding to retain as much genetic diversity as possible is important to avoid collapse in small populations. Even if small local pockets survive, if each pocket is only able to inbreed with itself that will cause problems.
Our local children's museum is part of a network of sites working to restore red wolf [1] populations. Every few years they get new wolves as the coordinators move young wolves around to optimize mating pairs.
Zig actually bundles LLVM's Clang, which it uses to compile C with the `zig cc` command. But the long term goal seems to not be so tightly coupled to LLVM, so I'm expecting that to move elsewhere. They still do some clever stuff around compiler-rt, allowing it to be better at cross-compilation than raw Clang, but the bulk of it is mostly just Clang.
There is also another C compiler written in Zig, Aro[1], which seems to be much more complete than TFA. Zig started using that as a library for its TranslateC functionality (for translating C headers into Zig, not whole programs) in 0.16.
I find that a very bold move, how will they reivent the wheel on the man-years of optimization work went into LLVM to their own compiler infrastructure?
Proebsting's Law: Compiler Advances Double Computing Power Every 18 Years
You need to implement very few optimizations to get the vast majority of compiler improvements.
Many of the papers about this suggest that we would be better off focusing on making quality of life improvements for the programmer (like better debugger integration) rather than abstruse and esoteric compiler optimizations that make understanding the generated code increasingly difficult.
No, the whole point is to eliminate dependencies that they have to maintain. "not obligate" really doesn't mean anything if it's available as a backend--the obligation is on the Zig developers to keep it working, and they want to eliminate that obligation.
And the original question was "how will they reivent the wheel on the man-years of optimization work went into LLVM to their own compiler infrastructure?" -- the answer is that Andrew naively believes that they can recreate comparable optimization.
There are a whole lot of misstatements about Zig and other matters in the comments here by people who don't have much knowledge about what they are talking about--much of the discussion of using low-level vs high-level languages for writing compilers is nonsense. And one person wrote of "Zig and D" as if those languages are comparable, when D is at least as high level as C++, which it was intended to replace.
First I was naive to believe I could make a new programming language, then I was naive to believe it would be anything but a toy project, then I was naive to believe that we could make our own backends for debug mode, now I'm naive to believe that we can add optimizations to the pipeline. It's getting old. Just because you lack the creativity, willpower, and work ethic to accomplish something, doesn't mean I do.
I admire your creativity, willpower, and work ethic, and a few other things about you, but I don't admire reactionary garbage like this ... I'm actually rather shocked by it and how it leans heavily on the strawman "I'm naive to believe that we can add optimizations to the pipeline" which is not the statement that was made, but I will maintain my high regard for you and your efforts despite it ... no human is perfect. I have a lengthy list of technical brilliancies in Zig that I admire that I won't bore you with but do often bore others with.
At least you acknowledge that I am correct about your belief, whereas someone else said I was exactly wrong.
As for me, while I had a successful software development career spanning 6 decades, received a mention in a two-digit RFC, and hold several networking patents, my best years are far behind me, but even in my heyday I couldn't hold a candle to your creativity, willpower, work ethic, or productivity ... but how is that at all relevant?
And that is grossly dishonest ... my original comment said nothing about what is true for me, and I never assumed (or assume) that what is true of me is true of others. In fact the whole point of my comment that you are responding to here is that I don't think that what is true of me is true of you--you are clearly better at these things than I. You are just being pointlessly and ungraciously belligerent. It's a hell of a way to treat consumers and admirers.
> zsf team is perfectly capable of implementing compiler optimizations.
Again with that strawman. The issue was about duplicating the years of effort put into LLVM. No doubt with ENOUGH effort that can be duplicated--logic dictates that ... but then it has to be continuously tracked. And when I said "recreate comparable", there is of course an implied time scale.
You're all in a huff because I used the word "naive". But then you made an argument that your naivety gets the damn thing done anyway because of will power, work ethic, etc. I grant that it's a good argument with results to prove it.
Enough ... I won't be provoked into responding again.
I think you are overestimating the impact of the long tail of compiler optimizations. You don't need to reverse-engineer every microarchitecture under the sun and optimize for their specific quirks like LLVM does to have useful code generation.
Just doing the basics goes a long way with a tiny fraction of the effort. Yes it will leave some percent on the table but this is hardly the end of the world. With their own compiler they have full control over the entire chain and might be able to make up for that with their own language-specific optimizations.
Go has a custom compiler, too, it's not as good as LLVM, so what?
To clarify, my statement was based on comments I have seen and heard from Andrew Kelley when discussing this subject. I can't locate those at the moment, but here is https://news.ycombinator.com/item?id=39156426 by mlugg, a primary member of the Zig development team (emphasis added):
"To be clear, we aren't saying it will be easy to reach LLVM's optimization capabilities. That's a very long-term plan, and one which will unfold over a number of years. The ability to use LLVM is probably never going away, because there might always be some things it handles better than Zig's own code generation. However, trying to get there seems a worthy goal; at the very least, we can get our self-hosted codegen backends to a point where they perform relatively well in Debug mode without sacrificing debuggability."
The current interim plan (which I think was developed after the comments that I heard from Andrew, perhaps in recognition of their naivete) is for Zig to generate LLVM binary files that can be passed to a separate LLVM instance as part of the build process. Is that "a first-class supported backend target for compilation"? I suppose it's a matter of semantics, but that certainly won't be the current LLVM backend that does LLVM API calls.
What do you mean by "interim"? As I explicitly stated in the comment you quoted, it has never, and likely will never, been planned for the Zig compiler to become incapable of using LLVM. The LLVM backend still sees plenty of active development by the core team [0]---that's perfectly compatible with improving the experience of users (including ourselves) by avoiding unnecessary uses of LLVM [1].
> ...is for Zig to generate LLVM binary files that can be passed to a separate LLVM instance as part of the build process. Is that "a first-class supported backend target for compilation"? I suppose it's a matter of semantics, but that certainly won't be the current LLVM backend that does LLVM API calls.
I think you are incorrectly assuming that we currently make heavy use of the LLVM API. As indicated by #13265 being closed, that is not true. The Zig compiler already generates bitcode by itself, without touching the LLVM API. The only thing we actually use the LLVM API for is feeding that bitcode to LLVM, which can easily be done by invoking a CLI instead. Users quite literally would not be able to tell if, for instance, we changed the compiler to pass the bitcode to Zig's embedded build of Clang over CLI.
> the answer is that Andrew naively believes that they can recreate comparable optimization.
That's exactly wrong.
> There are a whole lot of misstatements about Zig and other matters in the comments here by people who don't have much knowledge about what they are talking about.
as a comment about a particular project and its goals and timelines, this is fine. as a general statement that we should never revisit things its pretty offensive. llvm makes a lot of assumptions about the structure of your code and the way its manipulated. if I were working on a language today I would try my best to avoid it. the back ends are where most of the value is and why I might be tempted to use it.
we should really happy that language evolution has started again. language monoculture was really dreary and unproductive.
20 years ago you would be called insane for throwing away all the man-years of optimization baked into oracle, and I guess postgres or mysql if you were being low rent. and look where we are today, thousands of people can build databases.
I expressly said "not be so tightly coupled to LLVM" because I know they're not planning on dropping it altogether. But it is the plan for LLVM and Clang not to be compiled into the Zig binary anymore, because that has proven to be very burdensome. Instead, the plan seems to be to "side-car" it somehow.
How is it a hack? I mean, you may not like the fact that Zig does not provide syntax sugar for interfaces, but how exactly do you think they are implemented in languages that do?
First, if a performance optimization is a reliability regression, it was done wrong. A bounds check is removed because something somewhere else is supposed to already guaratee it won't be violated, not just in a vacuum. If the guarantee stands, removing the extra check makes your program faster and there is no reliability regression whatsoever.
And how does performance improve reliability? Well, a more performant service is harder to overwhelm with a flood of requests.
It does not need to be an explicit check (i.e. a condition checking that your index is not out of bounds). You may structure your code in such a way that it becomes a mathematical impossibility to exceed the bounds. For a dumb trivial example, you have an array of 500 bytes and are accessing it with an 8-bit unsigned index - there's no explicit bounds check, but you can never exceed its bounds, because the index may only be 0-255.
Of course this is a very artificial and almost nonsensical example, but that is how you optimize bounds checks away - you just make it impossible for the bounds to be exceeded through means other than explicitly checking.
Funny how you can tell a project is vibe-coded just from a first glance at its website. All these websites seem to somehow have the same visual style. Anyone noticed this?
Yep, my fans started revving as soon as I loaded it. Animations are out of control on the normal web as it is, but genAI sites take it to another level.
Yeah it was. I've been hoping to get the time to write a userscript to identify and stop animations like that, but I haven't been able to understand the whole requestAnimationFrame thing enough yet.
I'd given up on that a while ago because I basically had to disable it everywhere. But yeah this was custom inline js so my ublock filters didn't get it
Then you send the patch upstream, they incorporate and maintain it for you. Congratulations, you just FOSSed.