AI search control group calculator
Compare how often AI answers mentioned your brand before and after a change. Use an unchanged set of prompts to check whether mention rates also shifted elsewhere.
Gap between the changes
+12.5
percentage points
- Test prompts
- 37.5% to 62.5% +25.0 points
- Comparison prompts
- 25.0% to 37.5% +12.5 points
Answer counts changed between rounds. Check which prompts are missing before interpreting the gap.
Build a fair comparison
Choose both sets before the change. Keep the exact wording, model, search mode and market fixed across rounds. Run the two sets on the same dates. Look at several earlier rounds to check whether their rates usually move together.
Read the gap
Subtract the comparison group’s change from the test group’s change. The result is descriptive. If the sets already had different trends or the change affected both, the gap can mix those effects. World Bank explanation of the method.