Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I mean, I think we're both in the realm of [citation needed] here. I would argue that people index into strings quite a lot--whether that's because we thought UCS-2 would be enough for anybody or UTF-8 == ASCII and "it's probably fine" is academic. The solution is the same though: don't index into strings, don't assume an encoding until you've validated. That makes any "advantage" UTF-8 has disappear.

If you really think no one made this mistake with UTF-8, just read up on Python 3.



The difference is that with UTF-8 you're much more likely to trip over those bugs in random testing. With UTF-16 you're likely to pass all your test cases if you didn't think to include a non-BMP character somewhere. Then someone feeds you an emoji character and you blow up.


Which is why you should be using a library for all this, that uses fuzzing and other robustness checks.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: