I’m not sure what your professional experience is in, but as a counterpoint, I’ve never been in a situation where I hadn’t wished for a system I’m working with to already be in a Bayesian framework. Having said that, I only occasionally am building things from scratch instead of modifying existing systems, so I’m not always lucky enough to be able to work with them.
The pain points around getting a sampler/model pairing working in a reasonable timeframe is definitely a valid complaint. In my experience, inference methods in Bayesian stats are much less forgiving of poorly specified models (or said another way, don’t let you get away with ignoring important structural components of the phenomena of interest). A poorly performing model (in terms of sampler speed/mixing) is often a sign of a problem with the geometry of the parameter space. Frustratingly this can sometimes be a result of conceptually equivalent, but computationally different parameterizations (e.g. centered vs non-centered multi level effects).
The struggles are worth it IMO because it is helpful feedback that helps guide design, and the ease with which I can compute meaningful uncertainty bounds on pretty much any quantity of interest is invaluable.
My go to for teaching statistics is Statistical Rethinking. It’s basically a course in how to actually thing about modeling: what you’re really looking for is analyzing a hypothesis, and a model may be consistent with a number of hypotheses, figuring out what hypotheses any given model implies is the hard/fun part, and this book teaches you that. The only drawback is that it’s not free. (Although there are excellent lectures by the author available for free on YouTube. These are worth watching even if you don’t get the book.)
I also recommend Gelman’s (one of the authors of the linked book) Regression and Other Stories as a more approachable text for this content.
Think Bayes and Bayesian Methods for Hackers are introductory books from a beginner coming from a programming background.
If you want something more from the ML world that heavily emphasizes the benefits of probabilistic (Bayesian) methods, I highly recommend Kevin Murphy’s Probabilistic Machine Learning. I have only read the first edition before he split it into two volumes and expanded it, but I’ve only heard good things about the new volumes too.
Any time you start conditioning on something, i.e. selecting subsets of data to analyze. You can fool yourself quite often if you do something seemingly innocuous like select "everyone with at least one X" and compare expectations to what's true unconditionally (meaning not conditioning on anything, not "in all cases") with conditional computations.
Agreed wholeheartedly. I have argued with the VP of our department about this paper quite a few times.
I feel like Breiman sets up a strawman that I've never encountered when I work with my colleagues that are trained in the statistics community. That doesn't mean it didn't exist 25 years ago when he wrote it. I concede that we are sometimes willing to make simplifying assumptions in order to state something particular, but it's almost like we've been culturally conditioned to steep everything we say with every caveat possible.
Whereas I am constantly having to point out the poor feedback we've had about some of the XGBoost models despite the fact that they're clearly the most "predictive" when evaluated naively.
> I'm so slow it takes too much time for me to not think that it's a waste of time because I could have done something more meaningful instead.
That's the funny thing about the idea of meaningful things. It is solely determined by what you think is meaningful. Personally, just sitting and making something is an extremely meaningful activity to me.
Very neat, but audio doesn't work on desktop either. (Linux, Firefox) I had an old-school generated voice say "Chinese letter Chinese letter Chinese letter." It was definitely amusing, but I don't think that's what you were going for.
Wow. I thought you were being glib, but the average comment length is noticeably higher in the linked discussion. While length isn’t necessarily a valid proxy for meaningful conversation, this was definitely an eye-opening contrast to the current thread.
The pain points around getting a sampler/model pairing working in a reasonable timeframe is definitely a valid complaint. In my experience, inference methods in Bayesian stats are much less forgiving of poorly specified models (or said another way, don’t let you get away with ignoring important structural components of the phenomena of interest). A poorly performing model (in terms of sampler speed/mixing) is often a sign of a problem with the geometry of the parameter space. Frustratingly this can sometimes be a result of conceptually equivalent, but computationally different parameterizations (e.g. centered vs non-centered multi level effects).
The struggles are worth it IMO because it is helpful feedback that helps guide design, and the ease with which I can compute meaningful uncertainty bounds on pretty much any quantity of interest is invaluable.