It’s been around 2 months or so since my last blog post, and I was thinking of talking about something besides Gen AI or Machine Learning. Thus I figured that I should share something about A/B testing or Experiments, because this kind of knowledge can be required, especially in Analytics type Data Science roles.

Now, while there are plenty of academic resources on hypothesis testing and experiments, they typically don’t really discuss the nuances of online tests. That’s where this book, “Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing” really shines. It’s not a very technical book, but it does describe the problem of practical significance vs. statistical significance, which is an issue when dealing with large sample sizes. The only gripe I had with this book was that it was not so technical. The delta method was just barely mentioned, and so I had to look elsewhere for resources to thoroughly understand it.

That’s where this article and this GitHub repository come in. What I especially like about them, is that there is easy to understand code for simulating views, click and click-through-rates, and for those that prefer to read, the article does a great job explaining the simulations. Oh, and did I mention that it’s in Python. I spent weeks trying to find the GitHub repository because I thought it was coded in R. I finally found it earlier last week when I was searching for topics to post on.

Anyways, I hope you found this helpful. After New Years, I may talk about resolutions, or perhaps another Data Science topic. Until then, Merry Christmas!