It's not completely impossible, depending on what your expectations are. That language model that was built out of redstone in minecraft had... looks like 5 million parameters. And it could do mostly coherent sentences.
Which is a lot more than 888kb... Supposing your ESP32 could use qint8 (LOL) that's still 1 byte per parameter and the k in kb stands for thousand, not million.
> But cutting down to 2 million 3 bit parameters or something like that would definitely be possible.
Sure, but there's no free lunch
> Hey look what I just found
I've even personally built smaller "L"LMs. The first L is in quotes because it really isn't large (So maybe lLM?) and they aren't anything like what you'd expect and certainly not what the parent was looking for. The utility of them is really not that high... (there are special cases though) Can you "do" it? Yeah. I mean you can make a machine learning model of essentially arbitrary size. Will it be useful? Obviously that's not guaranteed. Is it fun? Yes. Is it great for leaning? Also yes.
And remember, Tiny Stories is 1GB of data. Can you train it for longer and with more data? Again, certainly, BUT again, there are costs. That Minecraft one is far more powerful than this thing.
Also, remember that these models are not RLHF'd, so you really shouldn't expect it to act like you're expecting a LLM to work. It is only at stage 0, the "pre-training", or what Karpathy calls a "babbler".
A reminder that what I said was "not completely impossible, depending on what your expectations are"
And I was focused more on the ESP32 part than the exact number of bytes. As far as I'm concerned you can port the model from the minecraft video and you still win the challenge.
Also, that last link isn't supposed to represent the best you can do in 800KB. 260k parameters is way way under the limit.
I disagree, in the future it might be possible. But perhaps not in English, but in some more formal (yet fuzzy) language with some basic epistemology.
I mean, there is a lambda calculus self-interpreter in 29 bytes. How many additional logical rules are required for GAI inference? Maybe not that many as people think. Understanding about 1000 concepts of basic english (or say, lojban) might well be sufficient. It is possible this can be encoded in 800kB, we just don't know how.
Having worked at multiple companies and talked to multiple legal teams about this, they tend to be very conservative. So the guidance I've gotten is that if we store any information at all on the person's computer, even to know whether they've visited the site before, we still need a cookie banner.
Basically, the law created enough fear among the lawyers that software developers are being advised to include the cookie banner in cases where it isn't strictly needed.
Agreed! Many sites don't actually comply with the GDPR because they don't provide simple tools to control the cookies and instead force you through a flow. Part of my gripe with the law is the way those violations are not being systematically cited.
You'd have much better retention rates if you don't cover up the content the viewer is trying to view.
How would you like it if I shoved a banner in your face the moment you walked into a store and forced you to punch a hole in it in order to view items on the shelves?
> even to know whether they've visited the site before
So uh, don't do that.
You don't need to notify if you use cookies for required functionality like login sessions or remembering a functional setting.
If you're tracking whether they're returning or not your activity is exactly the kind of behaviour the rule is covering because, in legal terms, it's skeezy as fuck.
> You don't need to notify if you use cookies for required functionality like login sessions or remembering a functional setting
Nobody wants to be the EU test case on precisely how "required functionality" is defined. Regardless of what the plaintext of the law says, it should be self-evident that companies will be more conservative than that, especially when the cost is as low as adding one cooke banner and tracking one preference.
"Strictly necessary cookies — These cookies are essential for you to browse the website and use its features, such as accessing secure areas of the site. Cookies that allow web shops to hold your items in your cart while you are shopping online are an example of strictly necessary cookies. These cookies will generally be first-party session cookies. While it is not required to obtain consent for these cookies, what they do and why they are necessary should be explained to the user."
Right, and then the legal teams tell me they don't care, and we should put up the cookie banner anyway. I feel like you didn't read my original comment.
That just means your legal team is lazy or incompetent. I work for a massive company that handles extremely sensitive PII and we don't have a cookie banner, because we don't need to have a cookie banner. GitHub doesn't have one, Gitlab doesn't have one.
The problem is that I spend hours explaining the actual technical nature of what we're doing to the legal team and I feel that there's often some kind of breakdown in communication because they don't understand the underlying technologies as well as the engineers do. And I haven't had this experience at one company, I've had it at multiple companies, several of which folks in this thread will have heard of.
To put a finer point on some of this, in one instance, I was writing an application that would allow our customers to deploy their own website with content that they had created through the tool that my company had provided. My company wasn't adding any tracking whatsoever to these pages. We were simply taking their content, rendering it properly, and hosting it for them. We ended up enforcing a cookie banner on these pages because the lawyers couldn't guarantee that there wouldn't be tracking content on that page that was added by the customers. But the end result is that every page, the vast majority of which don't have any tracking, still have cookie banners.
In essence, the law created a new legal hazard, and people aren't sure when they're going to run into it, so they end up putting up fences all over the place. Between this and malicious compliance, the end user experience has suffered greatly.
That's super interesting, because the lawyers should know that under GDPR, consent needs to be specific.
So a generic cookie banner is actually going to make the legal case worse than not having one at all (because you've now demonstrated that you knew you should have explicitly declared usages, partners, and used opt-in consent, but you didn't).
I know that everyone wants to give me legal advice. Lawyers don't care about legal advice from engineers. That's the crux of the point I'm trying to make.
Laws should be evaluated on the effect they actually have on society, rather than the effect that we wish they had on society. I am very critical of laws that fail this test, and I think they should be updated to improve their performance. We want the right outcome, not the right rules.
I'm willing to argue that, sure (though it's purely a hypothetical point as I'm not a citizen of the EU and thus I don't and shouldn't have a voice in the laws there). I don't judge a law by a deontological measure of worth, but rather by whether it seems to be making things better or worse. The GDPR has overwhelmingly made my experience browsing the web worse, not better. Whether it should have resulted in that is beside the point: it has resulted in that, so that is what I judge it by. Therefore, I think it makes sense to get rid of the law as it seems that it is making things worse for people, not better.
> The GDPR has overwhelmingly made my experience browsing the web worse, not better.
From where I sit that's hard to evaluate since you cannot actually see most data abuses and privacy concerns, and you also don't know how it would have been without it. You also see the effects of various laws and regulations in combination, so the ones related to GDPR are not easy to be singled out. Are you thinking only of the cookie banners? Maybe sites would be plastered with even worse bullshit. Did you consider that GDPR also resulted in privacy policies that (if actually somewhat legal) are fairly easy to read and not just copy pasta but specific to the service(s), have proper contact information, you get some transparency about which data partners the sites work with, sites need to have full data export, right to be forgotten (removal of your data/contributions), and so on. I am certain you benefit from it often, potentially without realizing, and you wouldn't know what the world would be like without them today so it's not so straightforward to reason about.
>At GitHub, we want to protect developer privacy, and we find cookie banners quite irritating, so we decided to look for a solution. After a brief search, we found one: just don’t use any non-essential cookies. Pretty simple, really.
Go to that link, these are the cookies it writes (at least for me):
Some are from github.blog, some are from the cloudflare.com hosting. Not sure how the laws apply to that. But obviously there's several analytics cookies.
All the legal uncertainty problems the cookie law produces aside, the core problem with the law is that it's fundamentally stupid. Cookies are a client side feature: You store the cookie, not the server. If you don't want to store the cookie, complain to your browser, that's the software responsible here. But instead of fixing the issue in the one place actually responsible, we make laws that force millions of websites to adopt.
You only start to need the popups if you specifically put cookies on a visitor's browser to build a personal profile of them.
This can be for e.g. sales acquisition or marketing engagement, but also includes cookies to simplify login, so not everything is "stupid stuff." A cookie that stores "was here, skip the splash page" may already fall afowl, if you put any session metadata in it.
It is just bad UI. It could have been better implemented, such as with a browser-side opt out setting, for instance. Similar to what we have for permissions, for instance.
I happened to work with people who elaborated the GDPR rules and they knew very well that it would end with cookie banners everywhere, or mandatory logins.
if you don't track users you don't need GDPR consent dialogs
I think in the past you still needed some info box in the corner with a link to the data policy. But I think that isn't needed anymore (to be clear not a consent dialog, a informational only thing). Also you can without additional consent store a same site/domain cookie remembering you dismissing or clicking on it and not showing it again (btw. same for opting out of being tracked).
But there are some old pre-GDPR laws in some countries (not EU wide AFIK) which do require actual cookie banners (in difference to GDPR consent dialogs or informational things). EU want them removed, but politic moves slow AF so not sure what the sate of this is.
So yes without checking if all the older misguided laws have been dismissed, you probably should have a small banner at the bottom telling people "we don't track you but for ... reasons .. [link] [ok]" even if you don't track people :(. But also if they haven't gotten dismissed they should be dismissed very soon.
Still such a banner is non obnoxious, little annoying (on PC, Tablet, a bit more annoying on Phone). And isn't that harass people to allow you to spy on them nonsense we have everywhere.
The regulatory body could clarify that a DO NOT TRACK header should be interpreted as a "functional/necessary cookies only" request, so sites may not interrupt visitors with a popup modal/banner if it's set.
Having the EU decide on a technical implementation is more of a last ditch effort, like what happened with more than a decade of the EU telling the industry to get its shit together and unify under a common charging port.
I'm curious as to what your thought process is for suggesting "the app" (not sure what app you are referring to) as an alternative for someone who essentially rage quits when they see a cookie banner, given that apps on average are even more so an invasion of privacy and riddled with dark patterns.
But that would require directing the anger at specific companies (and their 2137 ad partners) rather than at an easy target of the banana-regulating evil authority.
Sadly whenever this kind of discussion pops up it's usually a very unpopular take.
1. GDPR consent dialogs are not cookie popups, most things you see are GDPR consent dialogs
2. GDPR consent dialogs are only required if you share data, i.e. spy on the user
3. GDPR had from the get to go a bunch of exceptions, e.g. you don't need permission to store a same site cookie indicating that you opted out of tracking _iff_ you don't use it for tracking. Same for a lot of other things where the data is needed for operation as long as the data is only used with that thing and not given away. (E.g. DDOS protection, bot detection, etc.)
4. You still had to inform the user but this doesn't need any user interacting, accepting anything nor does it need to be a popup blocking the view. A small information in the corner of the screen with a link to the data policy is good enough. But only if all what you do falls under 3. or non personal information. Furthermore I think they recently have updated it to not even require that, just having a privacy policy in a well know place is good enough but I have to double check. (And to be clear this is for data you don't need permission to collect, but like any data you collect it's strictly use case bound and you still have to list how its used, how long stored etc. even if you don't need permissions). Also to be clear if you accept the base premise of GDPR it's pretty intuitive to judge if it's an exception or not.
5. in some countries, there are highly misguided "cookie popup" laws predating GDPR (they are actually about cookies, not data collection in general). This are national laws and such the EU would prefer to have removed. Work on it is in process but takes way to long. I'm also not fully sure about the sate of that. So in that context, yes they should and want to kill "cookie popups". That just doesn't mean what most people think it does (as it has nothing to do with GDPR).
If you need it in order to do what the user asked, you don't need the popup. The user asking to do a thing is consent to do what is obviously necessary to accomplish that thing, and may be consent to do what is less obviously necessary. The user's click on "sign in with google" is consent to share data with Google as needed to complete the sign-in, but no more - it's not consent to Google Analytics.
Legal bases for processing: https://gdpr-info.eu/art-6-gdpr/ Everyone knows part A because that's a catch-all. If the user requested something, it's better UX to use Part B. Parts C and F apply sometimes. You still have to follow the rest of the GDPR, like letting them delete it.
Comparing Tahoe to Windows 7 is hyperbolic. 7 had Media Player, 8 had Groove Music. 7 was welcomed as a feature-rich upgrade, 8 was boycotted as a user-hostile downgrade.
I don't know what school of contemporary design you hail from, but you can't piss on my back and tell me it's raining. Liquid Glass needs an 8.1 update, at the very least.
The Windows 7 vs 8 analogy doesn’t support your argument, it undermines it. Windows 8 wasn’t disliked because it was “modern”, it was disliked because it broke core interaction models in a way that actively obstructed use. macOS Tahoe hasn’t done anything remotely comparable. No Start screen catastrophe, no forced touch-first UI on non-touch hardware, no fundamental workflow regression.
What you’re reacting to is aesthetic drift, not functional decay. Liquid Glass is a visual language experiment, not a UX rupture. You may dislike it, that’s fine, but equating it to Windows 8 is category error. One is a design iteration layered on top of a relatively stable interaction model, the other was a structural interface failure.
Also, invoking Media Player vs Groove Music as if those are meaningful historical markers of “user-hostile downgrade” is... generous to the point of fiction. Windows 8’s problem was input metaphors, not media apps.
This isn’t Apple becoming Microsoft. It’s Apple doing what Apple always does: overreaching aesthetically, then sanding it down in point releases until everyone forgets they were angry in the first place. Which, incidentally, is exactly what happened with Aqua, iOS 7, Big Sur, and every other supposed apocalypse.
Has MacOS ever been better than Windows for allowing fine grained control over system services?
I've been a Mac user for my entire life so maybe I didn't understand what things were like with Windows, but the fundamental problem identified by Howard, that there are many many system daemons and it is expected that the user not know what they are, or what they do, and to just leave them alone, has been the case for at least 20 years, I think.
The entire point of Macintosh is that you don't need to know anything about it (and Apple used to actively try to hide things you didn't need to know about). Or at least that is the user it has always been targeted at since the original Mac OS was released.
Windows used to be known as the OS you'd "have to" tinker with.
Early versions of OS X allowed more freedom in what you could do with the OS. As soon as SSV/SIP entered, that cut off a lot of freeform access.
Every login steam steals focus no less than two times. Steam is one of the few login items I'd choose to keep, but wasting the first 30 seconds of login is too heavy a price to pay.
I don't know if you've used Windows lately, but Windows is orders of magnitude less pleasant than MacOS (or even previous bad Windows versions like Vista).
So good to see a developer who actually cares about resources. Thank you!
I usually develop my apps on a 1300 MHz single-core Intel Atom mini PC from 10 years ago. When something like this goes wrong, the app becomes unusable and I immediately notice it.
A bit unrelated but I found this interesting: water is transparent only within a very narrow band of the electromagnetic spectrum, so living organisms evolved sensitivity to that band, and that's what we now call "visible light".
I like to joke that while nitrogen gas is the most common thing around us, we are blind to it. Of course, that's a feature, since it allows us to perceive everything else further away, instead of stumbling through a perpetual fog.
This location-dependent tradeoff is something to think about when it comes to "false color" images in astronomy. If some aliens described Earth as "a boring uniform nitrogen-colored ball", we'd probably be a little offended at their ophthalmo-centrism, and tell them that the fault lies in their eyes, not in our planet.
It would probably be some application of spectral imaging [0], highly dependent on what data you choose to capture based on your assumptions about how the aliens see.
Even if you have a mathematical "photo of a planet of nitrogen gas clouds", that leaves the problem of how to present it to humans, since we have no concept of what "nitrogen gas color" is supposed to be.
As OP said, most astronomical pictures, even in the visible, target specific wavelengths and are thus "converted" for human vision. It's also obviously the case of any picture in wavelength beyond visible light. Keep in mind that there is also massive processing to clean the pictures, not just a translation of the wavelengths
visible light is also the last octave before you hit ionizing radiation. it’s very energetic. good for harnessing in chemical processes. not so energetic that the electrons leave the party.
At least he gave credit to HN, so the diaspora could find the source. The article is interesting. I think more needs to be said about how our eyes perceive color w.r.t. led lighting.
The sun is very close to a black body radiator, so all wavelength. The atmosphere and water filters a lot.
It is actually quite strange that plants are green -- that's the wavelength the atmosphere lets through particularly well, so would be particular good to be absorbed instead of reflected, for energy production. It seems nature hasn't come up with a good, cheap way to move the absorption into that wavelength.
Well, black-body radiation is still peaked around a certain range of wavelengths depending on the temperature, it's not just equal power at all wavelengths.
Light visible to humans is at the peakiest bit of the sun's black body spectrum, see here image here: https://i.sstatic.net/kRUju.png
Green isn't just the wavelength the atmosphere lets through the best, or the wavelength humans are most sensitive to, it's also the peak of the sun's black body spectrum.
however, it is really not that impressive for just a client