Multi-Armed Bandit

The Multi-Armed Bandit (MAB) is a feature within Optimizely's experimentation platform that leverages a machine learning algorithm to optimize the allocation of traffic among different variations in real-time. 

Multi-Armed Bandit

Instead of statistical significance, the MAB results page focuses on improvement over equal allocation as its primary summary of your optimization's performance. 

MABs do not show statistical significance

• MABs aggressively push traffic to whichever variations are performing best, because the MAB does not consider the reason for that superior performance to be very important.

Why MABs Don’t Show Statistical Significance

In traditional A/B testing, the goal is to gather data and determine if a variation performs better or worse than the control, usually through statistical significance. This helps you know if the change had the desired effect and allows you to improve future variations. Fixed traffic allocation speeds up the process of reaching statistical significance.

With MAB (Multi-Armed Bandit) testing, the focus is on directing more traffic to the best-performing variations without considering why they perform better. Because of this, MABs don’t show statistical significance on the results page, which helps avoid confusion and keeps the focus on optimizing for performance.

Best Use Cases for Multi-Armed Bandit (MAB) Optimization

Here are scenarios where MAB optimization may work better than traditional A/B testing:

• Promotions and Offers: For limited-time promotions, MAB optimization quickly shifts traffic toward the most successful variations, helping maximize conversions while the offer is active.

• Headline Testing: Headlines often have a short lifespan. Instead of waiting for an A/B test to reach statistical significance, MAB optimization ensures you get the most impact while the headline is still relevant.

• Webinars and Events: Test multiple call-to-action variations for event signups, like webinars, to quickly identify and promote the most effective one.

For more on how MAB works in Optimizely, check their end-user documentation.