# PDQ Analytics Terms 101

Understanding Metrics and Statistical Methods in PDQ Analytics

July 29, 2025

PDQ’s analytics framework is built to help you make data-informed decisions about your checkout performance and A/B tests.

This article walks through the key concepts, metrics, and statistical tools used in PDQ’s dashboards and experiments.

---

## What is OEC (Overall Evaluation Criterion)?

The **Overall Evaluation Criterion (OEC)** is the **primary metric used to evaluate experiments** and overall checkout performance in PDQ. It ensures we are measuring what truly matters.

## Why it matters:

Traditional metrics like:

- **AOV** (Average Order Value),
- **ASR** (Average Shipping Revenue), and
- **CVR** (Checkout Conversion Rate)

are useful but often conflict. For example:

- Raising AOV by increasing free shipping thresholds can reduce CVR.
- Increasing CVR through discounts might lower AOV.

To resolve this, PDQ recommends using **ARPC (Average Revenue Per Checkout)**,
a unified metric that balances both revenue and conversion.

---

## Primary Metric: ARPC

**ARPC (Average Revenue Per Checkout)** is the default and recommended OEC in PDQ.

## Formula:

```
ARPC = Total Revenue / Total Checkout Sessions
```

This metric effectively combines AOV, ASR, and CVR into one:

```
ARPC = (AOV + ASR) × CVR
```

When profit data is available, you can also use:

- **Gross Profit Per Checkout** = (Revenue - COGS) / Checkouts
- **Direct Profit Per Checkout** = (Revenue - COGS - Shipping Costs) / Checkouts

---

## Checkout Component Metrics

PDQ’s **checkout components** (like upsell modules and delivery date pickers) generate their own metrics.

## Key Metrics:

- **Checkout Component Impressions**  
  Unique checkout tokens (users) who viewed a component each day  
  → `Impressions = Distinct Tokens per Day`

- **Digital/Upsell Revenue & Added Items**  
  Tracks how many additional products (physical or digital) were added and purchased via PDQ components  
  → `Added Items = Sum of Products Added`  
  → `Revenue = Sum of Upsell Revenue`

---

## 📦 Track360 Metrics

Track360 helps merchants drive post-purchase engagement by bringing shoppers back to the brand’s site via a PDQ-hosted tracking page.

## Key Metrics:

- **Total Revenue**  
  Revenue from orders placed after a customer visits the PDQ tracking page  
  → `Revenue = Orders from Tracking Page Visits`

- **Visits & Avg. Visits per Order**  
  → `Visits = Total Page Loads`  
  → `Avg. Visits = Visits / Orders`

- **Orders Stuck in Transit**  
  → `Orders where tracking = "InTransit" for > 7 days`

- **Order Failures**  
  → `Orders where tracking = "Failed"`

---

## 🧮 Core Metric Formulas

|     |     |
| --- | --- |
| **Metric** | **Formula** |
| **CVR** | Orders / Checkouts |
| **AOV** | Revenue / Orders |
| **ASR** | Shipping Revenue / Orders |
| **ARPC** | Total Revenue / Checkouts = (AOV + ASR) × CVR |
| **Gross Profit per Checkout** | (Revenue - COGS) / Checkouts |
| **Direct Profit per Checkout** | (Revenue - COGS - Shipping Cost) / Checkouts |

---

## Measuring Statistical Significance with a t-test

To determine if a checkout A/B test had a meaningful impact, PDQ uses a **t-test for differences in means**.

## Step-by-step:

1. **Calculate ARPC for both groups**  
   → ARPC = Revenue / Checkout Sessions

2. **Estimate variability**  
   → Use standard deviation of ARPC in each group

3. **Compute standard error (SE)**

```
SE = sqrt[(sA² / nA) + (sB² / nB)]
```

4. **Calculate t-statistic**

```
t = (ARPCB - ARPCA) / SE
```

5. **Find p-value**  
   → Use degrees of freedom and t-distribution to find it

6. **Interpret results**  
   If `p < 0.05`, the difference is **statistically significant**

---

## Measuring Conversion Rate Significance with a z-test

When comparing **conversion rates**, PDQ applies a **two-proportion z-test**.

## Step-by-step:

1. **Calculate CR** for both groups:  
   → `CR = Orders / Checkouts`

2. **Pool the CR** across both groups  
   → `p = (ordersA + ordersB) / (checkoutsA + checkoutsB)`

3. **Calculate SE and z-score**

```
SE = sqrt[p(1-p) × (1/nA + 1/nB)] z = (CRB - CRA) / SE
```

4. **Check significance**  
   If `p < 0.05`, the difference is **statistically significant**

---

## ⚠️ SRM: Sample Ratio Mismatch Check

SRM occurs when the actual sample split doesn't match the expected allocation (e.g., 50/50 or 80/20).

## Step-by-step:

1. Use Chi-squared formula:

```
χ² = ((OA - EA)² / EA) + ((OB - EB)² / EB)
```

2. Compare result to critical value (3.841 at 95% confidence for 1 degree of freedom)

- If χ² > 3.841 → **SRM detected**
- If χ² ≤ 3.841 → sample split is fine

---

## In summary

- **Use ARPC** as your primary metric—it balances revenue and conversion.
- **Track engagement** with checkout and post-purchase components using built-in PDQ metrics.
- **Validate A/B tests** using t-tests (for ARPC) and z-tests (for CR)
- **Watch for SRM** and ensure test samples are valid.

If you’d like help running a test or interpreting the results, reach out to your CSM!

---