Evaluating podcast brand lift without overstating certainty

Judge a podcast brand lift result by how the study formed its groups, what it called exposure, who could answer its survey and how uncertain the difference is. A difference in answers can inform a campaign decision.

It does not automatically show that every reported impression was heard or that the result applies to all Australian consumers.

Set the brand question first

Choose a primary outcome that matches the campaign, such as brand recognition, ad recall or consideration. Fix the audience, question wording and survey timing before results arrive. A stated intention to buy is a survey answer, not a count of future purchases.

Define the decision the study should support. A study of recognition may help assess an introductory campaign; a study of message recall answers a different question. Keep secondary measures labelled as secondary.

Examine the comparison

Ask how the provider selected and, if applicable, assigned eligible people to treatment and control groups. A randomised holdout with comparable groups gives a stronger basis for an effect estimate than an observed exposed-versus-unexposed comparison. Check whether people could receive the campaign outside the study or appear in both groups.

Establish exactly what “exposed” means. It may refer to a platform’s ad-serving signal, a matched delivery record or another rule. The IAB Tech Lab’s Podcast Technical Measurement Guidelines address measurement of downloaded media and the ads included in those downloads. Describe the result using the study’s actual exposure rule rather than calling all exposed respondents confirmed listeners.

A before-and-after survey without a suitable comparison can show a change in answers. It cannot by itself separate the campaign from other advertising, news or a change in respondent mix.

Check coverage and precision

Request completed surveys in each group, recruitment method, Australian locations represented, survey timing, weighting and exclusions. Ask whether the study covers the full podcast buy or only inventory the provider can identify and survey. A result for one platform or subset of shows should not be applied to every placement.

Read the absolute difference in positive responses in percentage points, alongside an uncertainty interval where available. Check whether the sample can distinguish a difference large enough to matter for the decision.

A subgroup needs enough responses of its own; a campaign-wide result does not make every show, age group or state estimate reliable.

Conclude at the level the design supports

A well-controlled study can support an effect estimate for the population and inventory it actually covers. An observational comparison supports a more cautious statement because differences between groups may also explain the result. If the interval is wide or groups differ materially, call the result inconclusive for the proposed decision.

Keep the campaign’s media delivery beside the study to show what was assessed. Do not turn a survey difference into an exact count of people whose minds changed by multiplying or dividing it by impressions. State other activity that may have affected the comparison, and carry the study’s limits into the final conclusion.