Setting up an Extension Groups A/B Test

Compare two full extension stacks against each other, or test a stack against showing nothing

May 7, 2026

An Overview of A/B Testing

An A/B test compares two experiences side by side to see which performs better. Instead of guessing, you split real customer traffic between versions and let the data tell you which drives more conversions, revenue, or whatever metric matters most. PDQ runs the test automatically — set it up, launch it, and check the results.

An Extension Groups A/B Test compares two extension groups against each other. An extension group is a collection of multiple extensions that run together on your checkout. Use this test type to compare entire checkout experiences — for example, an upsell + trust badge stack against a free shipping bar alone.

You can also test nothing vs. an extension group — one group sees no extensions at all, the other sees your chosen group. Useful for measuring the true incremental impact of your extension setup on checkout performance.

Note: Display rules on individual extensions within each group still apply during the test.

Step 1: Create the test

In the left sidebar, go to Optimize > A/B Tests. Click New Test and select Extensions Groups Test from the dropdown.

Step 2: Fill in the details

• Test Name — short and clear (e.g. "Full Upsell Stack vs Free Shipping Bar Only").
• Hypothesis — what you expect to learn (min. 100 characters). Be specific: "Showing a full suite of checkout extensions will increase net revenue more than a single free shipping bar by creating multiple conversion touchpoints."
• End Test After X Checkouts — optional. Leave blank to run until you stop it manually.
• Goal Metric — Checkout Conversion, Gross Profit, AOV, Shipping Revenue, or Net Revenue.

Click Next.

Step 3: Set rules (optional)

Add conditions to limit which shoppers enter the test. Leave empty to run for everyone. Click Next.

Step 4: Allocate traffic

Two ways to run this test:

• Extension Group vs. Extension Group — pick a Control group and a Test group. Set the traffic split (typically 50/50). Click + Add Test Group to run a multi-variant test.
• Nothing vs. Extension Group — set Control to NULL (no extensions), Test to your chosen group. Measures the true impact of the group vs showing nothing.

Click Create draft test.

Step 5: Connect to Shopify and launch

Once the draft is created, a Token for Shopify tab appears inside the test. How you use the token depends on test type:

• Nothing vs. Extension Group — copy the token and add it to the desired Shopify checkout block (Settings > Checkout > select your checkout > Add block > paste the token). PDQ serves the extension group to the Test group and nothing to Control.
• Extension Group vs. Extension Group — paste the same test token into both extension groups' Shopify checkout blocks. PDQ uses the token to determine which group to serve to each shopper.

Once Shopify is connected, find your test in the Drafts tab in PDQ, click ⋯, and select Start Test to go live.

Managing your test

• Editing a draft — Find it in the Drafts tab, click ⋯, select Edit.
• Launching — Click ⋯ on the draft, select Start Test. Goes live immediately.
• Monitoring — A Running tab appears between Drafts and Concluded once at least one test is active. Performance data lives in Optimize > Analytics Dashboard, available the day after launch.
• Ending — In the Running tab, click ⋯, select End Test. View final results in Optimize > Analytics Dashboard, or click ⋯ in Concluded and pick Get Analytics.

Happy testing! Reach out to support@prettydamnquick.com if you need help.