Prioritization

Step 2: Prioritization

In Optimizely, prioritization refers to the process of determining the order or sequence in which experiments should be implemented. It involves evaluating and ranking potential experiments based on various factors to allocate resources effectively and focus on initiatives that are likely to have the greatest impact.

Key aspects of prioritization in Optimizely include:

1. Impact:

   • Assess the potential impact that each experiment could have on your key performance indicators (KPIs) or business goals. This involves estimating the degree to which a proposed change is likely to influence user behavior or improve the desired metrics.

2. Ease of Implementation:

   • Consider the complexity and resources required to implement each experiment. Some experiments may be simple and quick to execute, while others might involve more significant development efforts. Prioritization often involves finding a balance between impact and ease of implementation.

3. Confidence Level:

   • Evaluate the level of confidence you have in the proposed changes. This can be influenced by factors such as the amount of data supporting the hypothesis, the reliability of the testing methodology, and the degree of uncertainty associated with the experiment.

4. Strategic Alignment:

   • Align experiments with broader business or marketing strategies. Prioritize experiments that are in line with the overall goals and priorities of the organization.

5. Resource Availability:

   • Consider the availability of resources, including time, personnel, and budget. Prioritize experiments that align with the resources currently available or can be realistically allocated.

Another aspect that is very important to consider is the amount of traffic going to the desired page. It will help you identify how long it will take for the experiment to run to reach statistical significance. You can use Optimizely sample size calculator to understand sample size needed to run your experiment.

Here is the sheet that you can fill out with the team to help prioritize your experiments:

• Experimentation prioritization sheet

By systematically evaluating and prioritizing experiments, Optimizely users can optimize their testing roadmap and focus on the initiatives that are most likely to deliver meaningful results. This helps in making informed decisions about where to allocate resources and efforts for ongoing optimization and experimentation efforts.

Before moving to the next stage you will also need to fill out experimentation template for the experiments you would like to proceed with. 

People and Resources

• Program Manager (responsible for final scoring)
• Developer (responsible for estimating effort)
• Designer/UX (responsible for estimating effort)
• Executive sponsor (review, approve, and provide strategic alignment)

Test Plan & Design

Once a test has been scoped, prioritized, and approved by the CRO Lead, the Program Strategy Lead will put together an individual Test Plan. This PowerPoint deck will be used to define the test details and later on as a place to document the test’s results, recommendations, and next steps. A template of the Test Plan is available on SharePoint.

Once the test overview and visualizations sections are finalized, they will be reviewed by the Program Lead for feedback and/or approval.