Customer intelligence case study
Why prediction is not proof of incremental impact
A financial-services organization needed to know which personalized campaigns warranted further investment. Quantum compared targeted members with eligible members who were not targeted.
Talk to an ExpertClient and business question
Which personalized treatments should be scaled, stopped or revised?
A financial-services organization was using propensity models to select members for acquisition and retention/up-sell campaigns. Before committing more resources, its leaders needed to determine whether the targeted treatments were associated with better customer response.
Quantum evaluated campaign outcomes against an eligible non-targeted comparison group. The analysis gave the organization a practical basis for deciding where further investment was supported and where the treatment required reconsideration.
What the analysis found
Campaign outcomes provided a clearer investment signal
Comparison logic
Targeted members were evaluated against eligible members who were not targeted, giving leaders an outcome comparison rather than a model score alone.
2 groups
Targeted vs. eligible non-targeted
Strongest observed signal
The line-of-credit acquisition segment produced the clearest positive comparison in the analysis.
≈3×
Observed response in one segment
Decision value
Mixed results helped leaders determine which treatments merited further investment and which should be stopped, revised or tested again.
3 actions
Scale, stop or revisit
Why the experiment mattered
The client needed evidence for campaign decisions
A propensity score can identify members who are likely to respond, including members who may have responded without receiving a campaign treatment. The comparison helped the organization assess the treatment itself, not only the model used to select the audience.
-
Set the business question
Quantum focused the analysis on acquisition and retention/up-sell outcomes that could inform campaign investment.
-
Created an eligible comparison
Compared targeted members with eligible non-targeted members so observed response could be examined by objective and segment.
-
Supported the next campaign decision
Positive, weak and negative comparisons identified what to scale, stop or revisit.
Evidence and results
| Evaluation | Observed result | Decision implication |
|---|---|---|
| Line-of-credit acquisition segment | Approximately three times the observed response for targeted members relative to the eligible non-targeted comparison. | A strong signal worth further validation and carefully controlled scaling. |
| Other evaluated objectives | Weaker or negative target-versus-comparison results. | Do not assume a high propensity score means the intervention created incremental impact. |
Key lesson
Prediction guides whom to target. Comparison reveals whether the intervention is working.
Pairing predictive models with outcome measurement gave the client a more reliable basis for campaign decisions. A model may accurately identify people who are already likely to respond; an appropriate comparison is needed to assess whether the intervention is associated with changed behaviour.
Evidence boundaries
What the evidence supports
- Observed lift based on a targeted-versus-eligible non-targeted comparison.
- Approximately three times the observed response in one line-of-credit acquisition segment.
- Not all targeted treatments outperformed their comparison group.
- The analysis informed what to scale, stop or revisit.
Related Quantum capabilities
Services relevant to this business question
Customer Intelligence & Personalization
Shape behavioural and lifecycle signals into profiles, decision cues and more relevant customer experiences.
Explore this capability →Customer Segmentation
Identify groups with distinct behaviours, value patterns and needs to guide targeting and journey design.
Explore this capability →Data Mining
Reveal drivers, patterns and anomalies that strengthen understanding and inform future analysis.
Explore this capability →Let's collaborate for success
Turn customer and operational data into insight, decisions and action with evidence you can trust.
Contact Us
