Let's learn about Ab Testing via these 67 free blog posts. They are ordered by HackerNoon reader engagement data. Visit the Learn Repo or LearnRepo.com to find the most read blog posts about any technology.
A/B testing is a method of comparing two versions of a webpage or app feature to determine which performs better. It matters for data-driven decision-making, allowing businesses to optimize user experience, conversion rates, and product effectiveness through empirical evidence.
1. Feature Selection for Imbalanced Datasets Using Pearson Distance and KL Divergence
A model-free method using statistical distance metrics like Pearson chi-squared and KL divergence to identify important features in highly imbalanced datasets.
2. Using the Stratification Method for the Experiment Analysis
Learn how to improve experiment efficiency and metric sensitivity through stratified sampling in data analysis.
3. Using T-tests for Abnormal Data in AB Testing
Discover the truth about using t-tests in AB testing for abnormal distributions in the IT industry.
4. The Power of User Feedback in Product Development
The article describes using both direct and indirect feedback methods throughout the product development lifecycle.
5. Increasing the Sensitivity of A/B Tests
A statistical approach to determining whether A/B results are significant or random noise.
6. Size Does Matter: Global Control Group for a Bank
Learn how to approach data-driven measurement properly. See what unexpected results we got in a bank and get insights for your own data analytics journey.
7. Measuring Non-Linear User Journeys: Rethinking Funnels Metrics in A/B Testing
A deep dive into user reorders, hidden behavioral patterns, and how aggregated funnels improve A/B test accuracy in non-linear user journeys
8. When A/B Tests Aren’t Possible, Causal Inference Can Still Measure Marketing Impact
Learn how to measure marketing impact without A/B tests using causal inference, Diff-in-Diff, synthetic control, and GeoLift.
9. AB Testing on Small Sample Sizes with Non-Normal Distributions
In this article, we will explore the intricacies of AB testing on small sample sizes, which can be valuable in B2B settings or products with a limited user base
10. How to Run Impact Analysis Without an A/B Test?
A practical guide to Propensity Score Matching — learn how to estimate treatment effects without running a traditional A/B test.
11. Common Pitfalls in Setting up A/B Tests
Identify common A/B testing pitfalls and learn to over come them.
12. Causal Impact Analysis as an Alternative to A/B Testing
Causal Impact analysis is a valuable tool, but it comes with its set of limitations that practitioners need to be mindful of.
13. The Data Delusion: Why Brands Trust Dashboards More Than People - And Why That’s a Mistake
Why data alone misleads—and how emotion, feedback, and AI create better brand decisions.
14. Use Beta Distribution and Thompson Sampling to Beat The Multi-armed Bandit at the Casino
As a logical person at the casino. you want to put your money on the machine with the maximum expected return. This is the origin of the multi-armed bandit problem. We will cover the two most basic concept here: Beta distribution and Thompson sampling.
Beta Distribution
15. Hallway Testing [A Deep Dive Analysis]
So you’ve got a fantastic idea for improving your product’s interface. Problem is, it’s gonna cost time, money and energy to implement. You’re pretty sure it’s gonna be good, but how can you tell for sure? Simple. Use a “Hallway Usability Test”, which will help you find out early on whether you’re on the right track.
16. Beyond A/B Testing — Switchbacks and Synthetic Control Group
Experimentation designing in the marketplace without AB-Testing using Synthetic Control Groups and Switchbacks.
17. Waiting for your A/B Testing Results — Guide for Easy Acceleration
Explore techniques for accelerating A/B testing, including paired testing, covariance adjustment, stratification, CUPED, CUPAC, and Bayesian approaches.
18. AB Testing For Digital Products
An overview of AB testing during the design of digital products like UX, digital marketing advertisements and software development.
19. Paywall AB Testing for Subscription-Based IOS and Android Apps
AB testing is an essential aspect of mobile app development, especially when it comes to subscription-based iOS and Android apps.
20. A/B Testing was a Jerk, Until we Found the Replacement for Druid
The recipe for successful A/B testing is quick computation, no duplication, and no data loss. So, we used Apache Flink and Doris to build our data platform.
21. A/B Testing In Flutter With Statsig
Leverage Statsig to build Flutter apps FAST!
22. Tales of the Undead Salmon: Exploring Bonferroni Correction in Multiple Hypothesis Testing
Bonferroni correction as a solution for multiple comparisons problem in A/B tests. Here is an explanation of how it works with a simulation written in Python.
23. Spotify’s Secret to Smarter A/B Testing (Hint: It’s Not Just Statistics)
Discover how Spotify refines A/B testing with decision rules to improve product experimentation and reduce risks in multi-metric analysis.
24. Automated A/B Testing: 5 Ways To Use ML To Improve Your UX Design
Can you use machine learning to improve your UX design? Here are 5 ways to use ML when designing your website.
25. User Research in UI/UX Design is Not Always Required, Here's Why
Talking about the role of user research, examples of when you can skip it and how to know when to use it
26. How To Build Products Like a Scientist
Every decision and step we take at a startup is based on some belief. Here's how to validate those beliefs and build products like a scientist ;)
27. How to Improve Marketability of a Mobile Game Application with Concept Testing
28. A Well-intentioned Cashback Program Caused an Increase in Fraud—Here's What Happened
Discover how our cashback strategy unexpectedly led to an increase in fraudulent activities. Learn from our A/B test results and insights on preventing fraud.
29. Retrospective is A/B Testing for Teams
If you are new to Agility and Retrospective, I’ll offer in this post a novel introduction to it. We will explore how retrospectives and your team organization can take inspiration from A/B Testing.
30. How to Use A/B Testing for Product Design Improvement
This guide defines that approach, and how you can use A/B testing for product design improvement.
31. How To Nail Video Email Marketing With Seasonal Email Campaigns
Ah, so you've decided to use video marketing. A wise choice indeed, seeing as, according to Google, "6 out of 10 people prefer online video platforms to live TV".
32. Three A/B Testing Mistakes I Keep Seeing (And How to Avoid Them)
The three most common mistakes in A/B testing analysis involve the Mann–Whitney test, bootstrapping, and default Type I and Type II error rates.
33. Mastering Exposure Points for Accurate Mobile A/B Testing
Learn why exposure points can make or break your mobile A/B tests, common pitfalls to avoid, and practical tips to improve your app experimentation results.
34. More A/B Tests Won’t Fix Your Growth Problem
35. The Four Key Metrics in A/B Testing
Spotify standardizes A/B testing with success, guardrail, deterioration, and quality metrics to refine product experimentation and minimize risk.
36. How to Create an App Landing Page that Converts
Have you thought of building an app for your business? If so, this is the smartest business decision you can make. In today’s mobile-first world, you have to target potential customers on mobile. According to Techjury, there are 2.7 billion smartphone users around the world. The same report says 77% of Americans have smartphones. The time spent per user with digital media on mobile in the US daily in 2017 was 2.3 hours. This shows latent potential for your business in the digital market. A report by Statista says the total number of mobile app downloads in 2017 was 197 billion. This highlights the importance of mobile apps as a business tool. If your business doesn’t have an app yet, you risk losing out on this customer-rich market. It is important, however, to note that not every mobile app works. There are millions of apps on Google’s Android Play Store and the Apple app store. The competition is stiff, and for your app to stand out, you need the highest converting landing page.
37. Instrument Variables and AB Testing – Part 1
This article explores the Mathematical details of least squares estimator in an unbiased and biased settings due to model specification errors.
38. How to Build Connections for A/B Testing and Linear Regression: An Essential Guide
In a world of LLM and cutting-edge architectures, linear regression quietly plays a crucial role, and it’s time we shine a light on how it can be beneficial.
39. Measuring Product Impact When A/B Testing Is Not Available
How to evaluate product releases without an A/B test. A trustworthy framework using causal inference, Synthetic Control, and rigorous data guardrails.
40. Aisles of the Future: How PCIC’s Category-Item Blend Transforms Online Grocery Shopping
This article details PCIC’s deployment, A/B test lifts, virtual aisles impact, and future directions for combining category and item insights.
41. Spotify’s Approach to Multi-Metric A/B Testing Decisions
A decision rule framework improves A/B testing by balancing statistical rigor and practicality, ensuring reliable product decisions with controlled error rates.
42. The Next Evolution in Business Process Improvement
Combining AB testing and reinforcement learning empowers rapid, data-driven business process changes, addressing failures faster than traditional BPM.
43. What BPM Pros Really Think About AI and A/B Testing Process Change
Industry experts say AB-BPM’s DevOps-driven process improvements need human oversight, cultural fit, and platform integration for practical success.
44. How Spotify Standardizes Multi-Metric Experiment Analysis
Exploring decision theory, OECs, and clinical trial methods to improve A/B testing. Learn how Spotify standardizes multi-metric experiment analysis.
45. Why People, Platforms, and Process Drift Shape AB-BPM’s Future
Industry experts see AB-BPM as promising for structured, rapid process testing—if paired with impact forecasts, human oversight, and strong change management.
46. Evaluating A/B Testing Decision Rules with Monte Carlo Simulations
Monte Carlo simulations analyze the impact of alpha & power corrections in A/B test decision rules, optimizing error rates for better statistical reliability.
47. How Nyholt’s Method Makes Scientific Testing More Reliable
Nyholt’s method improves statistical efficiency by refining error rates and sample size calculations, offering an alternative to Bonferroni-type corrections.
48. These Six A/B Testing Mistakes Are Costing You Big Time
After examining thousands of experiments from top tech companies, I discovered six critical A/B testing mistakes that are squandering your team's hard work.
49. How Companies Decide Which Product Changes to Keep or Scrap
Learn how Spotify’s Decision Rule 2 integrates deterioration and quality metrics to improve A/B test validity and prevent regressions in online experiments.
50. Ensuring Reliable A/B Test Decisions with Guardrail Metrics
Learn how UI & IU testing principles, Bonferroni corrections, & power adjustments ensure accurate A/B test decisions with multiple success & guardrail metrics.
51. How a BPM Dream Team Ranked the Risks and Tools for AB-BPM
A grounded theory and ranking-type Delphi study with BPM experts captured qualitative insights on AB-BPM risks, adoption, and key tool features.
52. Evaluating False Positives and Sequential Testing in Experimentation
Analyzing false positive rates & impacts of sequential deterioration tests on statistical accuracy using Monte Carlo simulations and Group Sequential Testing.
53. A Detailed Guide on Conversion Rate Optimization in E-Commerce
Improve your online sales with my comprehensive guide on Conversion Rate Optimization in E-commerce. Learn about website design, UX, A/B testing, sales funnels.
54. Retain More Customers With these A/B Testing Tools
Here we have listed Best A/B testing tools to help you improve your digital marketing strategy, find solutions, and engage with your ideal customers.
55. No Testing Is (Sometimes) Better Than Some Testing
Explore why "some research" can harm innovation & sound design, plus tools & reads for better UX practices. Empower smarter research decisions today!
56. Balancing Type I and Type II Errors in A/B Testing Decisions
Learn how A/B testing decision rules use multiple-testing corrections like Bonferroni to balance Type I and Type II errors in multi-metric experiments.
57. Why Human Touch Still Matters in Automated Process Testing
AB-BPM is promising, but needs human oversight, transparency, and integration for success; expert input reveals new research paths and key tool priorities.
58. How to Improve Accuracy in Success and Safety Testing
Improving efficiency in hypothesis testing by minimizing overlap in rejection regions for success and guardrail metrics in superiority and inferiority tests.
59. Enhancing Experiment Sensitivity in B2C: A Robust Framework for Heavy-Tailed Metrics
Boost B2C experiment sensitivity with Cross-Fitted CUPED. Learn how to handle heavy-tailed metrics like ARPU without overfitting. Includes Python code.
60. 4 Lessons We Learned in 2019 (and How Marketers Can Apply Them in 2020)
Even if you’re still knee-deep in holiday and end-of-year promotions, it makes sense to take time to pause. Now’s the time to reflect on the challenges, opportunities, and accomplishments of 2019—before the crazy starts up again.
61. Frequentist Stats Are Failing Your UX Decisions—Here’s a Better Way
Learn why Bayesian A/B testing offers more intuitive insights than traditional stats, & get practical tips and tools for better UX decisions under uncertainty.
62. How to Run Tons of Experiments at the Same Time Using an Adaptive Control Group
Bandit algorithms solve some A/B testing complexity, but hide others. Here's a method to fix that.
63. Using A/B Testing For Your e-Commerce Business
The basic idea behind an A/B test is to present a change to a small segment of the overall audience, and see how it impacts their behaviour.
64. How a No-Code Tool Changed Our Testing Process
Jooble introduced a no-code approach that made testing faster, more cost-effective, and scalable.
65. The HackerNoon Newsletter: What If There Was a Right Way to Argue Online? (3/31/2025)
3/31/2025: Top 5 stories on the HackerNoon homepage!
66. How to Apply A/B Split Testing to Marketing
A/B Split Testing takes the guesswork out of optimization.
67. Running A/B Tests You Can’t Measure
Thank you for checking out the 67 most read blog posts about Ab Testing on HackerNoon.
Visit the /Learn Repo to find the most read blog posts about any technology.