More volume. Less quality.
One metric grows while the commercial outcome deteriorates.
Marketing Analytics
More leads.Better business?
When one metric improves and another gets worse, I analyze what changed, what the evidence shows and what is still a hypothesis. I organize it into a diagnosis, prioritized hypotheses and a plan to validate them before deciding.
I apply it to questions about investment, performance, audiences, messaging and measurement, using SQL on GA4, ad-platform and CRM data.
SpendUnchanged
Enquiries+33%
Qualified contacts−56%
Betterbusiness?
Qualification rate30% → 10%
Cost / qualified+125%
Revenue?
01 / When data does not give one clear answer
The data changed. The decision is still unclear.
One metric can improve while another gets worse. Deciding requires separating what is known, what is still assumed and what needs checking.
Conflicting signals.
Platforms, CRM data or periods can tell different stories.
Several possible explanations.
Audience, messaging, measurement or context could explain the change.
My starting point: separate evidence, hypotheses and questions that remain open.
02 / What the analysis produces
From a change to a defensible next step.
How I structure the work: a clearly framed decision, comparable evidence and a concrete way to reduce uncertainty.
01
Decision brief
The pending decision, the alternatives and the result that could change the action.
Explore an example
- Example
- Increase investment, hold it steady or first review Paid Social lead quality?
- How it is validated
- The question identifies an action, an owner and the evidence that would justify reconsidering it.
02
Measurement contract
KPIs with definitions, formulas, sources, populations, observation windows and limitations.
Explore an example
- Example
- Qualified: meets the CRM’s agreed commercial criterion within seven days of creation.
- How it is validated
- Two people can recalculate the rate and cost using the same rules and mature cohorts.
03
Evidence-based diagnosis
What changed, by how much, where it is concentrated and what the data can establish.
Explore an example
- Example
- 300 → 400 enquiries; 90 → 40 qualified contacts, with US$10,000 spent in each period.
- How it is validated
- Denominators, sources, follow-up and discrepancies remain visible, without inferring causality from association.
04
Hypothesis map
Possible explanations ranked by evidence, relevance and feasibility of validation.
Explore an example
- Example
- Messaging, the audience or the commercial definition recorded in the CRM may have changed.
- How it is validated
- Each hypothesis lists initial signals, alternatives and the evidence that could support or contradict it.
05
Validation plan
What evidence to gather next, how to obtain it and which decision criteria to use.
Experiment design, when appropriate.Explore an example
- Example
- First validate CRM consistency and review changes. If appropriate, compare two messages with the same offer.
- How it is validated
- The plan specifies a metric, comparison, prior criterion, owner and conditions. It does not presume a favorable result.
Volume is not quality.
Paid Social · May → June 2026
- 01Decision
- Should we increase Paid Social investment?
- 02What changed
- Enquiries rose from 300 to 400, while qualified contacts fell from 90 to 40.
- 03Evidence
- With US$10,000 per period, the qualification rate falls from 30% to 10% and cost per qualified contact rises from US$111.11 to US$250.
- 04What we do not know yet
- The comparison alone cannot identify whether the change comes from audience, messaging, CRM or measurement.
- 05Next evidence
- Validate consistent CRM qualification and review audience and messaging changes before scaling.
- 06Decision criterion
- Reconsider scaling when quality can be compared under the same definition and observation window.
The selected piece highlights the parts of the brief it feeds.
03 / Interactive diagnosis
Change the lens. Change the reading.
The same campaign can tell different stories depending on the metric you observe. Change the lens and see how the decision evolves.
Interactive demo · simulated dataPaid Social · May → June 2026
Would you invest more?
Unchanged spend
Spend per period: US$10,000 · same channel
Unchanged spend
Start with a lens.
Observe one metric, then add more evidence before deciding.
Qualified within seven days of each enquiry. Descriptive comparison: it does not identify a cause or demonstrate a realized improvement.
Question: Start with a lens.Experiment design, if applicableOne hypothesis. A way to test it.
Clarifying qualification requirements may improve the proportion of visitors who generate a qualified lead.
A / CurrentCurrent message
B / VariantClearer qualification requirements
- Primary metric
- Visitors with at least one qualified lead / all assigned visitors.
- Guardrails
- CPL · lead volume · form errors.
- Assignment
- Random and persistent per visitor · same offer.
- Decision rule
- Defined before launch: meaningful effect, sample, duration and guardrails.
Proposed test design. A favorable result is not assumed. Acquisition and follow-up are planned separately; seven days of follow-up do not mean a seven-day test.
04 / How I approach it
From a question to the evidence it requires.
One decision. Five steps.
- 01
Frame the decision
I start from what needs deciding and which outcome actually matters.
I document the pending action, alternatives, decision owner and constraints. A broad question becomes a comparison the data can answer.
- 02
Define the measurement
I define KPIs, sources, populations and windows so the comparison is valid.
Formulas, exclusions, identifiers and cohort maturity are documented before comparing.
- Outcome
- Agreed commercial criteria recorded in the CRM
- Cohort
- Unique enquiries created during the reporting period
- Follow-up
- 7 days from each enquiry’s creation
Define before counting
I define with the sales team what qualifies an enquiry, where that decision is recorded and which identifier prevents duplicates. I document the source, formula, time zone and exclusions so two people can reproduce the same KPI.
Allow equal follow-up
In this example, each enquiry has seven days to qualify. I compare cohorts that have completed that follow-up; recent records remain pending, so incomplete observation is not mistaken for lower quality.
Review criterionSpend must cover the same scope. With no qualified contacts, cost per qualified contact is undefined and is never reported as zero.
Inspect a KPI definition
cohort = unique enquiries from the period qualified = meet criteria within 7 days mature = latest enquiry + 7 days ≤ cutoff rate = qualified / enquiries cost = spend / qualified denominator = 0 → undefinedIllustrative window. Commercial criteria and follow-up are agreed with the team.
- 03
Diagnose the evidence
I locate what changed, by how much and where the variation is concentrated.
Every figure retains its denominator and source; breakdowns are used only where coverage and follow-up are comparable.
- Qualification
- 30% → 10% · −20 percentage points
- Cost per qualified
- US$111.11 → US$250
- Spend
- US$10,000 in each period
Read the whole funnel
Enquiries rise from 300 to 400, while qualified contacts fall from 90 to 40. I show the rate, its percentage-point change and cost per outcome, retaining the denominators needed to check each figure.
Check competing explanations
I review changes in coverage, qualification criteria and channel mix. Breakdowns are used only where the data support a valid comparison. A change over time suggests hypotheses but does not establish that a campaign caused it.
Review criterionI check definitions, currency, complete follow-up and comparable coverage. Unreconciled differences are documented before recommending an intervention.
Inspect the synthetic calculation
Period A: 90 / 300 = 30% Period B: 40 / 400 = 10% Change: 10% − 30% = −20 pp Cost A: 10,000 / 90 ≈ US$111.11 Cost B: 10,000 / 40 = US$250 Readout: observed decline; cause unresolvedSynthetic data. These totals describe the change without establishing its cause.
- 04
Prioritize explanations
I separate findings from hypotheses and prioritize those worth investigating.
I rank hypotheses by evidence, potential impact and feasibility. For each, I record what observation would support it, what could refute it and which alternative remains open.
- 05
Design the next evidence
I define what to observe, measure or test to reduce uncertainty.
Experiment design, when appropriate.With enough sample and traceability, I design an experiment; otherwise, I use another validation path and state its limitations.
- Change
- Current message versus explicit requirements
- Assignment
- Random and persistent per identifiable visitor
- Outcome
- Visitors with a qualified lead / all assigned visitors
Preserve the comparison
I propose stable assignment and one binary outcome per visitor: at least one qualified lead. Intent-to-treat analysis includes everyone assigned, including visitors who never submit the form. The offer and qualification rules remain consistent.
Agree how to decide
I plan the minimum detectable effect, power, sample and duration before starting. The final assessment includes the difference and confidence interval, form errors and cost. A favorable interim result does not trigger an early finish.
Review criterionI check assignment imbalances and missing data before interpreting results. Without traceability or sufficient sample, the hypothesis remains unresolved.
Inspect an evaluation protocol
A = current message · B = explicit requirements unit = visitor with persistent assignment outcome = ≥1 qualified lead per visitor denominator = everyone assigned to the variant plan = minimum effect + power → sample readout = difference + interval + guardrails close = planned sample and durationAcquisition and follow-up windows are specified separately: seven days to qualify a lead does not mean a seven-day experiment.
05 / Method limits
What it covers and what it does not claim.
The method focuses on one concrete question and the evidence available to answer it. What it cannot establish is written down.
In every analysis
- Decision and alternatives
- Measurement definitions
- Evidence-based diagnosis
- Hypothesis prioritization
- Validation plan
If the question requires it
- Experiment design
- Audience or market signals
- Additional data collection
What it does not claim
- Causality from an association
- Representativeness from partial signals
- A winner from a one-off improvement
- A favorable result before measuring it
06 / Technical questions
Questions about the method.
Doesn’t a dashboard solve this?
Not on its own. A dashboard shows changes; diagnosis explains what changed, against which denominator and what remains to validate. A dashboard is one input to a diagnosis, not the diagnosis itself.
Does analysis prove what caused a change?
Not always. Comparisons identify changes and associations; establishing a cause requires an appropriate design. I distinguish observed evidence, hypotheses and outstanding validation.
What if there is not enough volume for an experiment?
First I estimate the difference that matters and the sample required. Improving measurement, gathering further evidence or exploratory research may be more useful, with explicit limits.
Is an A/B test always needed?
No. I experiment when it is feasible and addresses the decision. Validation can also mean reconciling sources, reviewing cohorts or investigating changes.
How is a hypothesis evaluated?
I set the expected evidence, metric, comparison and criteria before looking at the data. The data can support, contradict or leave a hypothesis unresolved; a temporary improvement alone does not establish a winner.
What role do market or audience signals play?
They add context when they help answer the question. Social listening, search trends and audience signals have their own coverage and biases; I document them and do not treat them as representative of the whole audience.
Why count qualified contacts at seven days?
Because a recent cohort has not had time to qualify yet. I compare mature cohorts with the same window; otherwise the latest period looks worse simply because it is newer.
What is automated and what remains human judgment?
Calculations, joins and consistency checks can be automated and re-run. Framing the decision, ranking hypotheses and judging when the evidence is enough remain human calls, documented so others can review them.
Contact
Interested in the topic?
Measurement, hypotheses and experimentation.
If you work in marketing measurement and want to compare approaches, message me. The data and examples on this page are synthetic; the method is the one I apply.
Message me on LinkedInExplore the Labs