Setting up a Shipping Rates A/B Test

Compare two or more Shipping Rate Groups to find the rate strategy that converts best

May 7, 2026

Table of contents

• Step 1: Create the test
• Step 2: Fill in the details
• Step 3: Set rules (optional)
• Step 4: Allocate traffic
• Managing your test

An A/B test compares two experiences side by side to see which performs better. Instead of guessing, you split real customer traffic between versions and let the data tell you which drives more conversions, revenue, or whatever metric matters most. PDQ runs the test automatically — set it up, launch it, and check the results.

A Shipping Rates A/B Test compares two or more Shipping Rate Groups against each other. Test whether free shipping above $50 converts better than $5 flat rate, or any other rate strategy you can structure into a group. PDQ splits checkout traffic and tracks the results.

Step 1: Create the test

In the left sidebar, go to Optimize > A/B Tests. Click the purple New Test button in the top-right and select Shipping Rates Test from the dropdown.

Step 2: Fill in the details

• Test Name — short and clear (e.g. "Free Shipping vs Flat Rate").
• Hypothesis — what you expect to learn (min. 100 characters). Be specific: "Offering free shipping above $75 will increase checkout conversion by reducing payment-step drop-off."
• End Test After X Checkouts — optional. Leave blank to run until you stop it manually.
• Goal Metric — Checkout Conversion, Gross Profit, AOV, Shipping Revenue, or Net Revenue.

Click Next.

Step 3: Set rules (optional)

By default the test runs for everyone. To limit it to a segment (specific country, cart value, etc.), click + Add rule group. Click Next.

Step 4: Allocate traffic

You'll see your existing Shipping Rate Groups. Check the ones to include and assign each a traffic percentage. All percentages must add up to 100%.

• Standard A/B test — check two groups and split 50/50.
• Multi-variant test — check three or more groups (e.g. 33/33/34). The more variants you add, the more checkout volume you need to reach statistical significance.

Click Create draft test.

Managing your test

• Editing a draft — Find it in the Drafts tab, click ⋯, select Edit.
• Launching — Click ⋯ on the draft, select Start Test. Goes live immediately.
• Monitoring — A Running tab appears between Drafts and Concluded once at least one test is active. Performance data lives in Optimize > Analytics Dashboard, available the day after launch.
• Ending — In the Running tab, click ⋯, select End Test. The test moves to the Concluded tests tab. View final results in Optimize > Analytics Dashboard, or click ⋯ in Concluded and pick Get Analytics.

Happy testing! Reach out to support@prettydamnquick.com if you need help interpreting results.