Documentation Index
Fetch the complete documentation index at: https://docs.orderprotection.com/llms.txt
Use this file to discover all available pages before exploring further.
Setting Up Your Test Settings in Shopify
Before you can start your A/B tests within the Order Protection app, you need to configure a few settings.Enable a New Custom Pixel
From Settings, click on 
Once in the Customer Events settings, click 
Customer Events.
Add custom pixel. Name it “Order Protection A/B Testing” or something similar to know the testing is originating in the OP app.
Add the Code Script
Once the pixel is added, add the custom script below.It should look something like this.
Before you save the file, you need to swap out the following in quotations

const STORE_URL = ‘kysonteststore4.myshopify.com’;
with your Shopify store URL. The store URL can be found at the top of the left navigation in settings as well as at the top of the current custom pixel page you are populating.
This URL must be your myshopify URL for it to work properly. The Shopify URL will be something similar to: {store_name}.myshopify.com. Save the custom pixel and you’re done! 🙌Starting an A/B Test in the Order Protection App
Now that you have configured your custom pixel, you’re ready to start A/B testing. Log back in to app.orderprotection.com, navigate to to the
A/B Tests tab
in the left navigation and click on Start New Test to get started. Session data will start rolling into the test analytics within a few minutes. Expect 1-2 weeks for results to
reach statistical significance depending on your store traffic volume.What You Can Test
When creating a new A/B test, you’ll choose from three test types. Each type isolates a different variable so you can measure its impact on conversions and revenue.
Widget
Test whether the Order Protection widget defaults to on or off for customers at checkout. This is the most common test type for measuring opt-in vs. opt-out behavior.
Price
Test different pricing structures to find the optimal balance between customer conversion and revenue. Compare your current pricing against a variant with adjusted rates.
Widget Style
Test different widget designs and layouts to see which style drives the highest engagement and attach rate.
Configuring Your Test
After selecting your test type, you’ll configure the test details. Each test type has its own configuration options.Pricing Tests
When you select Price as the test type, you’ll see your current pricing displayed as the Original variant. This is pulled directly from your store’s existing price settings and cannot be edited within the test — it serves as your control group. Create a Variant with adjusted pricing to test against the original. You can modify both the flat rate and percentage-based pricing tiers for different order value ranges.
Widget Style Tests
Widget style tests let you compare two different widget designs to determine which performs better. Before setting up this test type, you’ll need at least two widget configurations created in the Customizations section. Navigate to Customizations > Widgets to view your available widget configurations. You’ll need both a published widget (your control) and at least one additional widget to use as a variant.


Checking Your Results
Once your test is running, you can monitor results in real time from the A/B Tests tab. Click on any active or completed test to view the results dashboard.
All metrics are calculated starting from when the A/B test was launched — not from when your store was created or when Order Protection was installed. This ensures a clean comparison between your test variants.
Understanding Your Metrics
The results dashboard shows a summary row at the top with key winner indicators, a time-series chart for tracking trends, and a detailed comparison table. Here’s what each metric means:| Metric | What It Measures |
|---|---|
| Views | The number of times the Order Protection widget was displayed to customers in each variant. |
| Conversions | The number of completed checkouts per variant, regardless of whether Order Protection was purchased. |
| CVR (Conversion Rate) | The percentage of widget views that resulted in a completed checkout. Calculated as Conversions / Views. |
| Attach Rate | The percentage of orders that included Order Protection, per variant. Calculated as Protected Orders / Total Orders. |
| Profit | Estimated profit generated from Order Protection fees based on your revenue share agreement. |
Views
Views
Views represent the total number of times the Order Protection widget was rendered and visible to a customer during their shopping session. Each unique session where the widget loads counts as one view.The results table breaks views down per variant, so you can see whether traffic was split evenly. A significant imbalance in views between variants may indicate that the test needs more time to normalize.
Conversions
Conversions
Conversions track the number of completed purchases (checkouts) within each test variant. This metric captures all orders — both those that included Order Protection and those that did not.The CVR (Conversion Rate) is calculated as:
CVR = Conversions / Views x 100The CVR Compare to Original column shows the percentage-point difference between each variant and the original, making it easy to spot whether a variant is helping or hurting overall checkout completion.Attach Rate
Attach Rate
Attach rate is the most important metric for most A/B tests. It measures the percentage of total orders in a variant that included Order Protection.
Attach Rate = Protected Orders / Total Orders x 100For example, if 3,409 out of 5,171 orders in the Original variant included Order Protection, the attach rate is 65.93%. A higher attach rate means more customers are opting in to protection, directly increasing revenue.The Attach Rate Winner in the summary row highlights which variant achieved the highest attach rate.Profit Calculation
Profit Calculation
Profit is calculated based on your revenue share agreement with Order Protection. This represents your store’s estimated earnings from Order Protection fees collected during the test period.The calculation considers:
- Total Order Protection fees collected per variant (based on the pricing tier and order values)
- Your revenue share percentage — the portion of each fee that goes to your store
- Claim costs are not factored into this metric; it reflects gross revenue share only
Variant Details
At the bottom of the results page, you’ll find Variant Details showing the exact configuration for each variant. This includes:- Whether Order Protection was on by default
- Whether the No Order Protection option was provided
- Any widget style differences between variants
- For Price tests, the full pricing tiers and rates for each variant (flat rates, percentage-based rates, and order value ranges)

