Start your first A/B testing #

Welcome to Ptengine A/B Testing!

You are now entering Ptengine's A/B testing feature, a powerful capability designed to improve and optimize your website experience. A/B testing allows you to simultaneously present two (or more) versions of a page and determine which version performs more effectively by analyzing user responses to each.

Ptengine offers:

• Module-Level A/B Testing
• Page-Level Redirect Testing
• Intelligent Traffic Bandit Testing（Allocation for Short-Term Campaigns ）

Getting started with A/B testing #

In this section, we will guide you through the setup process for A/B testing. Whether you are a beginner or an experienced user, this section will quickly get you started and on your way to becoming an A/B testing expert.

Prior Reading Recommendation: Before diving into this section, we strongly recommend that you first review Pop-up Building & Page Editing.

Why Conduct A/B Testing? #

A/B testing is a methodology that uses two or more variations (e.g., page hero images, section content, or other page elements) to determine which version has the greatest impact on user behavior and contributes most effectively to your ultimate conversion goals.

By conducting A/B tests, you can make data-driven decisions, grounded in scientific algorithms, rather than relying on intuition or guesswork, when optimizing your website.

If you require more flexible or comprehensive creative testing, beyond simple block-level testing, consider using Redirect Testing instead.

1. Create an A/B Test

Within Experience, you can create new A/B tests or add test variations to existing Experiences. Follow the steps below to create a new testing:

1.1. Create A/B testing in an existing experience
If you want to turn on A/B testing in your existing experience, you can follow this step.

2. Specific Settings

In the Settings, you can make detailed Settings, including traffic weight, minimum testing ****duration, minimum probability to be best score, and anti-flicker functions.

2.1. traffic weight

The default setting is that the traffic weight is evenly split, but you can also customize the percentages

• Evenly split: The traffic is evenly distributed to each group.
• Custom percentage distribution: Please make sure the sum is 100%.
  After A/B testing for a while, you can also use the custom option if you want to increase traffic to a well-performing group.

2.2 How ptengine deliver the results

Ptengine A/B test runtimes are generally determined by the "Probability of Being Best" (Win Rate). By setting thresholds for "Probability of Being Best" and "Minimum Test Duration," you can control the accuracy of your tests and determine when a test should conclude or continue.

2.2.1 Probability of Being Best (Win Rate)

"Probability of Being Best" (Win Rate) refers to the probability that one variation is statistically determined to be superior to other variations. The default setting is (95% or higher). When the "Probability of Being Best" exceeds 95%, you can be confident in declaring a winning variation, and, considering the Minimum Test Duration, the test results will be displayed in your reports.

The confidence interval for "Probability of Being Best" can be adjusted to "90%" or "98%".

A lower percentage results in a shorter test duration but lower accuracy. Conversely, a higher percentage results in a longer test duration but higher accuracy.

2.2.2 Minimum Test Duration

"Minimum Test Duration" is set to ensure that test results are not evaluated prematurely within a certain period after the test begins.

Due to cyclical influences (e.g., weekly patterns in user behavior), a default setting of 7 days is generally recommended.

The "Minimum Test Duration" can be adjusted within a range of 3 to 14 days.

Tip: Practical Examples

<Quick Result (Time: Short / Accuracy: Low)>

• Probability of Being Best: 90%
• Minimum Test Duration: 3 days

<Week-Based Testing for Balanced Accuracy (Time: Default / Accuracy: Default)>

• Probability of Being Best: 95%
• Minimum Test Duration: 7 days

<Highly Accurate Results with Extended Testing (Time: Long / Accuracy: High)>

• Probability of Being Best: 98%

• Minimum Test Duration: 14 days

  2.3 Anti Flicker
  

Anti-flicker functionality is a feature designed to mitigate the phenomenon where the original page is briefly visible when editing page content and displaying Experiences. In Ptengine, anti-flicker functionality is enabled by default.

To ensure correct operation of the anti-flicker feature, you need to paste the Ptengine Basic Tag directly into the page's HTML source code. If the tag is installed via Google Tag Manager (GTM) or other Tag Management Systems, the anti-flicker feature may not function properly, potentially impacting user experience. If you encounter any issues during the process, please contact our technical support team.

3.Set the goal

After editing the testing, we can display for different pages and different user groups.
This means that in Ptengine, you can easily personalize the user experience!