Setting up a Cross-Site A/B Test

Run a true end-to-end A/B test by comparing entire Shopify themes and their connected checkout experiences

May 7, 2026

A Cross-Site A/B Test compares the performance of different Shopify themes against each other. Traffic is split between your themes, and PDQ tracks how each performs against your goal metric. It's the right call when you're testing a redesigned theme against your current live one before fully committing.

For the why-and-when explainer, see Introducing: Cross-Site Testing.

Step 1: Create the test

In the left sidebar, go to Optimize > A/B Tests. Click New Test and select Cross-Site Test from the dropdown.

Step 2: Fill in the details

• Test Name — short and clear (e.g. "New Theme Redesign vs Current Live Theme").
• Hypothesis — what you expect to learn (min. 100 characters). Be specific: "The new theme redesign will improve checkout conversion by providing a cleaner, more streamlined experience for our customers."
• Goal Metric — Checkout Conversion, Gross Profit, AOV, Shipping Revenue, or Net Revenue.

Click Next.

Step 3: Allocate traffic

You'll see your available Shopify themes. For each theme you want to include:

• Set the traffic percentage directed to it.
• Assign a Checkout Group — this determines which PDQ extension setup is served alongside that theme.

Percentages across all included themes must add up to 100%.

Make sure the checkout group assigned to each theme matches the experience you want to serve. This controls the full checkout that shoppers in that group will see.

Step 4: Install the PDQ script

For PDQ to route traffic correctly, you need to install a small script in each theme being tested. In the test, go to the Installation tab and follow these steps for every theme:

1. Copy the provided script snippet.
2. In Shopify, go to Online Store > Themes.
3. Click ... next to the theme and select Edit code.
4. Open the theme.liquid file.
5. Paste the script just before the closing tag and save.

Repeat for every theme included in the test.

Step 5: Launch the test

Once traffic allocation is set and the script is installed in all themes, go back to PDQ. Find your test in the Drafts tab, click ⋯, and select Start Test to go live.

Managing your test

• Editing a draft — Find it in the Drafts tab, click ⋯, select Edit.
• Launching — Click ⋯ on the draft, select Start Test. Goes live immediately.
• Monitoring — A Running tab appears between Drafts and Concluded once at least one test is active. Performance data lives in Optimize > Analytics Dashboard, available the day after launch.
• Ending — In the Running tab, click ⋯, select End Test. View final results in Optimize > Analytics Dashboard, or click ⋯ in Concluded and pick Get Analytics.

Happy testing! Reach out to support@prettydamnquick.com if you need help.