Hacker Newsnew | past | comments | ask | show | jobs | submit | _venkatasg's commentslogin

I got this silly idea for a benchmark and decided to test how well models do. Repo here: https://github.com/venkatasg/fizzbuzz-bench


I was thinking about FizzBuzz and thought it might be cool to benchmark various LLMs to see the highest number they could go before they got it wrong. FizzBuzz is cool because you can test whether the model's can generalize to any other game (divisors of 3 and 7 instead of 3 and 5 for example).

Fun, short and sweet experiment to run over the weekend, with some mildly interesting results :)

https://github.com/venkatasg/fizzbuzz-llm


Inspired by Jonathan Hoefler's essay that illustrated the problems with pangrams as typeface proofs, I wanted to build a simple site that lets you proof any font from Google Fonts with his proof texts. Almost all(~95%) of the site was built with Gemini CLI


Doesn't work for me: I switch between Arial and Helvetica but the font doesn't change.


The differences are very subtle in lowercase, check uppercase? But maybe your system doesn't have Helvetica/Arial installed? I assumed its available on every OS, but that might not be the case.


I'm running a pretty standard Win 11. I looked at the page you made about the differences and looked for those and don't see them.

Pro tip: take a look at the home page vs /new -- people are pretty frustrated with all the "I vibe coded something that almost worked" posts and not voting them up.


Weird. I see the differences on mine across browsers and on my phone and desktop. And yeah I get it, I don’t claim this to be something amazing that should shoot up in ranking, just that it was my first experience vibe coding. But it certainly works on my end! What about if you search for a google web font?


That seems to work -- I can say though that off the top of my head I don't know the name of any of those fonts, though typing in "Tre" turned up a whole lot of them. Helvetica vs Arial is about as good as Batman vs Superman (though a New Yorker thought that my use of Arial to simulate signs from the New York Subway was an outrage) but you should have some example that just works for everyone.


SHould have double checked Windows default fonts — Windows doesn't have Helvetica ugh.


Just wanna say this is a really cool project, and I can't think of higher praise than me hoping I build something as cool as this some day! I've been meaning to learn Go for sometime now, and will be referring to Zasper for the future :)


Most sentences in the world are not about truth or falsity. Training on a high quality corpus isn’t going to fix ‘hallucination’. The complete separation of facts from sentences is what makes LLMs powerful.


Slightly related to the task, I wanna plug in my utility app for finding LaTeX commands for characters, DeTeXt: https://venkatasg.net/apps/detext

I've gotten a lot of requests to do whole equations, but I feel that would massively increase the complication of the app for not that much benefit? How often do people want to convert a whole bunch of equations into LaTeX? My use case is usually writing my own equations and forgetting the command for a specific symbol, or looking for a symbol that looks something like X.


Stealing formula. Sometimes one just want to be able to copy and paste equations in their own notes.


I've noticed that soundtracks show up in this Classical music app, so some (not all) of my Tamil songs (which are songs from movies because that's how most pop/commercial music in India is released) are listed there now.


I keep running into issues, even after installing Rust in my condo environment (using conda). Specifically the issue seems to be building wheels for `tokenizers`:

  warning: build failed, waiting for other jobs to finish...
  error: build failed
  error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib -C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/tokenizers.cpython-310-darwin.so'` failed with code 101
  [end of output]
 note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
  Failed to build tokenizers
  ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
Any suggestions?


I played around a bit and found out dropping the tokenisers version to 0.11.6 worked

`pip install tokenizers==0.11.6` first


That worked thank you!


LSP is game-changing. I had made a bunch of clippings to work with for TeX and Python, and this obviates the need for most of those :)


I'm beginning to think I can replace my Macbook Pro 2015 with the new Macbook Air...the numbers are so tempting.

Let's see how the reviews turn out, where people will use at real-world tasks.


I'm especially interested to see how it deals with heavy loads heat-wise. With no fan it could end up getting up to leg-roasting temps


It seems like Apple may err on the side of just letting it get painfully slow instead of crisping your legs. The primary differentiator between the Air and Pro is that the Pro comes with a fan and a $300 premium.


Don't forget the Touch Bar - that weird uncle in every family that we just have to put up with - the feature nobody asked for and every coder I know would so much rather be without.

It annoys me any my coworkers (we use 2017/2018 15" MBP's) every work day to no end. I hate it when when I'm accidentally logged out by feathering the 'lock' icon.

I'm aware I could move it - but ideally where are times that I'd want to use the feature as intended.

The lack of haptic feedback for the Touch Bar is a bizarre and unusual oversight for a company usually so detail-obsessed as Apple. We are not allowed to install anything but specifically approved software, so any third-party solutions are a no-go here. It should be built-in, it's a no-brainer.

I've personally taken to using a bluetooth Apple keyboard half the time with my work laptop just to avoid the Touch Bar as it honestly feels infinitely more comfortable for coding to not have it there.


The air is using binned chips where one core has failed (a very common practice that intel also does), so the pro is on a better processor


What is tempting about this? This benchmark shows that the M1 performs marginally worse than the low-end sku from 3 generations ago.


No, it shows it performs better? Go look again at the numbers in the linked comparison. Also calling it low-end is misleading, the 1050Ti was solidly mid-tier, and also a dedicated graphics card used in desktop machines. This is comparing it to integrated graphics in a lightweight laptop (the M1 in the recently announced MacBook Air).


I'm a graphics engineer. It's so low end at this point I honestly dread benchmarking on it.


If you really are a graphics engineer you'll know that the person you're replying to is absolutely right. The performance on this integrated gpu is outstanding and outperforms a dedicated desktop gpu that's ~2 years old on 10W of power.


1060 is budget mid-tier, I don't really see how you could say the 1050ti is anything but low-end when looking at either performance or price


We are talking about a half-inch thick ultrabook with 18 hours of battery life here.

The idea is that this can do the same as what was historically in a thick and heavy gaming laptop that got 4 hours of battery life in use..is very impressive. It firmly destroys all SKUs of the 15 inch MBP


You can run Witcher 3 on a 1050 Ti with a reasonably useful graphics level and framerate. You can't really do that with any other laptop integrated CPU/GPU.


For mac users these numbers are amazing lol


lol what's hilarious is the numbers probably go up just by installing windows and using the actual first-party nvidia driver.


I would wait some time until more software and os projects gets native support and kinks are ironed out.

I don't know your workflow but maybe some dependency in your pipeline has issues and then you are in a world of pain trying to figure out why.


I started using a pi for some personal projects and I concur. There are things that aren't supported or have to be build from source, and that is if there's support at all.

I'm sure the M1 is going to light a fire in getting ARM support on most projects, but it's not there yet.


I’ve gotten update notices for a large percentage of my apps where they call out that they are now compiled for M1 Macs. I think it likely that most actively maintained apps will publish M1 natives version in short order.


The idea of changing my 2015 MBP for something else is a tough one to accept, my biggest beef with the recent Air being the heat issues: the fan on the previous Air is so useless it's laughable[1] and they completely removed it on the new one, I remain skeptical.

[1] https://www.youtube.com/watch?v=iiCBYAP_Sgg


I just upgraded my 2015 MBP to a 16" 2019 MBP. I specifically bought it now so that I have an Intel chip because I don't want to deal with the headache while things slowly switch over.

I hadn't realized how much faster computers had gotten since 2015, this new machine runs circles around the old one. The keyboard is great and I actually like the touch bar. 0 regrets, it's an upgrade in every single way.


If your needs require high CPU utilization for extended time periods, you might be better off what the new MacBook Pro. The Air is intended for more causal use cases.


You have 2 weeks to return it ;)


In the US, you have even longer during the holiday period:

> Items purchased at the Apple Online Store that are received between 10 November and 25 December 2020 may be returned up to 8 January 2021.

https://www.apple.com/shop/help/returns_refund


I was tempted as well until I realized it's only 8GB of memory. There's no way i could survive on less then 16 currently.

edit - ignore that I'm an idiot :)


For $200 you can upgrade to 16GB during the order process.


Ouch, that's pretty steep price for additional 8gb ram. How does it compare to other devices in the same category? I'm out of the loop when it comes to laptops, for desktops iirc you can get one 32gig die of ram for cheaper than that.


It's definitely more expensive than simply buying a DIMM, but keep in mind that is "unified" memory and is part of the SoC, and is likely much faster than any old stick of RAM you'd normally put in a laptop. That, plus the Apple Premium.


Yup, thats what I did, also if you "work in education" it's $180.


This is exactly the camp I'm in. I won't touch the new MBPro because of the touchbar (and keyboard woes), but I'm willing to concede the peripheral inputs if it means the thing is much lighter/slimmer and has better performance.


FYI the keyboard has since been fixed and is, apart from some slight differences to key layout, the same as the air.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: