PDQ Analytics Terms 101

Understanding Metrics and Statistical Methods in PDQ Analytics

July 29, 2025

PDQ’s analytics framework is built to help you make data-informed decisions about your checkout performance and A/B tests.

This article walks through the key concepts, metrics, and statistical tools used in PDQ’s dashboards and experiments.

What is OEC (Overall Evaluation Criterion)?

The Overall Evaluation Criterion (OEC) is the primary metric used to evaluate experiments and overall checkout performance in PDQ. It ensures we are measuring what truly matters.

Why it matters:

Traditional metrics like:

• AOV (Average Order Value),
• ASR (Average Shipping Revenue), and
• CVR (Checkout Conversion Rate)

are useful but often conflict. For example:

• Raising AOV by increasing free shipping thresholds can reduce CVR.
• Increasing CVR through discounts might lower AOV.

To resolve this, PDQ recommends using ARPC (Average Revenue Per Checkout), a unified metric that balances both revenue and conversion.

Primary Metric: ARPC

ARPC (Average Revenue Per Checkout) is the default and recommended OEC in PDQ.

Formula:

ARPC = Total Revenue / Total Checkout Sessions

This metric effectively combines AOV, ASR, and CVR into one:

ARPC = (AOV + ASR) × CVR

When profit data is available, you can also use:

• Gross Profit Per Checkout = (Revenue - COGS) / Checkouts
• Direct Profit Per Checkout = (Revenue - COGS - Shipping Costs) / Checkouts

Checkout Component Metrics

PDQ’s checkout components (like upsell modules and delivery date pickers) generate their own metrics.

Key Metrics:

• Checkout Component Impressions
  Unique checkout tokens (users) who viewed a component each day
  → Impressions = Distinct Tokens per Day

• Digital/Upsell Revenue & Added Items
  Tracks how many additional products (physical or digital) were added and purchased via PDQ components
  → Added Items = Sum of Products Added
  → Revenue = Sum of Upsell Revenue

📦 Track360 Metrics

Track360 helps merchants drive post-purchase engagement by bringing shoppers back to the brand’s site via a PDQ-hosted tracking page.

Key Metrics:

• Total Revenue
  Revenue from orders placed after a customer visits the PDQ tracking page
  → Revenue = Orders from Tracking Page Visits

• Visits & Avg. Visits per Order
  → Visits = Total Page Loads
  → Avg. Visits = Visits / Orders

• Orders Stuck in Transit
  → Orders where tracking = "InTransit" for > 7 days

• Order Failures
  → Orders where tracking = "Failed"

🧮 Core Metric Formulas

Measuring Statistical Significance with a t-test

To determine if a checkout A/B test had a meaningful impact, PDQ uses a t-test for differences in means.

Step-by-step:

1. Calculate ARPC for both groups
   → ARPC = Revenue / Checkout Sessions

2. Estimate variability
   → Use standard deviation of ARPC in each group

3. Compute standard error (SE)

SE = sqrt[(sA² / nA) + (sB² / nB)]

4. Calculate t-statistic

t = (ARPCB - ARPCA) / SE

5. Find p-value
   → Use degrees of freedom and t-distribution to find it

6. Interpret results
   If p < 0.05, the difference is statistically significant

Measuring Conversion Rate Significance with a z-test

When comparing conversion rates, PDQ applies a two-proportion z-test.

Step-by-step:

1. Calculate CR for both groups:
   → CR = Orders / Checkouts

2. Pool the CR across both groups
   → p = (ordersA + ordersB) / (checkoutsA + checkoutsB)

3. Calculate SE and z-score

SE = sqrt[p(1-p) × (1/nA + 1/nB)] z = (CRB - CRA) / SE

4. Check significance
   If p < 0.05, the difference is statistically significant

⚠️ SRM: Sample Ratio Mismatch Check

SRM occurs when the actual sample split doesn't match the expected allocation (e.g., 50/50 or 80/20).

Step-by-step:

1. Use Chi-squared formula:

χ² = ((OA - EA)² / EA) + ((OB - EB)² / EB)

2. Compare result to critical value (3.841 at 95% confidence for 1 degree of freedom)

• If χ² > 3.841 → SRM detected
• If χ² ≤ 3.841 → sample split is fine

In summary

• Use ARPC as your primary metric—it balances revenue and conversion.
• Track engagement with checkout and post-purchase components using built-in PDQ metrics.
• Validate A/B tests using t-tests (for ARPC) and z-tests (for CR)
• Watch for SRM and ensure test samples are valid.

If you’d like help running a test or interpreting the results, reach out to your CSM!