Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm going to stick with multiarm bandit testing.


This is a much better approach...


What tools/frameworks are you using for running and analysing results?


At my previous job when it was relevant, I wrote something in house.

A higher order component would pick which variant at runtime (cause we had problems with SSR, or that would be more appropriate). Cached the picks in cookies.

In house charting and probability calculations to determine what P(X>Y) was for each experiment pair. Then we'd just manually prune them occasionally (since the bad ones weren't being displayed, timeliness didn't much matter). Periodically re-introduce old variants by hand if we thought it was worth it.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: