The problem a bandit solves
You have three channels and a fixed budget. One has produced the most pipeline so far, another looks promising on little data, and the third had one good month. Put everything on the leader and you never find out whether the others are better. Split evenly and you keep paying for channels the data already argues against.
This is the multi-armed bandit problem, named after a row of slot machines with unknown payouts. Each channel is an arm. Every euro you spend is a pull that pays out and also tells you something about that arm. The aim is to earn the most over the whole period while still learning enough to change your mind when the evidence turns.
Thompson sampling, first described by William R. Thompson in 1933, answers it with one rule: give each arm a share of the traffic equal to the probability that it is the best arm, given everything observed so far. Arms you are unsure about still get tried, because their uncertainty leaves room for them to be the best. Arms that are clearly worse fade out by themselves.
How it works
-
Describe each channel with a distribution
For every channel, keep a probability distribution over its true rate of return, for example opportunities per €1,000 of spend. A channel with plenty of data has a narrow distribution; a new one has a wide one.
-
Draw once from each
Take one random sample from every channel’s distribution. A wide distribution sometimes produces a high draw even when its average is low, which is how uncertain channels get their chance.
-
Back the highest draw
The channel with the highest draw wins the round. Repeat the draw thousands of times and count how often each channel wins: that share is its probability of being the best.
-
Allocate, spend and observe
Split the budget in line with those probabilities, spend it, record what came back, and update each distribution with the new results.
-
Repeat
As data accumulates the distributions narrow and budget concentrates on the channels that keep producing. If a leader starts to slip, the others win more draws again.
The distributions behind it
For rates such as click-through or conversion rate, the usual choice is a Beta distribution: start from Beta(α, β), add each success to α and each failure to β. For counts per unit of spend, such as opportunities per €1,000, a Gamma distribution plays the same role: its shape grows with every conversion and its rate grows with every unit of spend. Both updates are simple additions, so the model stays cheap to run and easy to audit.
To weight by money instead of counts, multiply the rate by a second distribution for the value of one conversion, such as pipeline per opportunity. A channel then wins a draw when its sampled rate times its sampled value is the highest.
The core idea in code
// Each arm has a Gamma posterior for opportunities per €1,000
// and a Gamma posterior for pipeline value per opportunity.
function probabilityBest(arms, draws = 10000) {
const wins = arms.map(() => 0);
for (let d = 0; d < draws; d++) {
let best = 0;
let bestValue = 0;
arms.forEach((arm, i) => {
const rate = sampleGamma(arm.shape, 1 / arm.rate);
const value = sampleGamma(arm.valueShape, arm.valueMean / arm.valueShape);
if (rate * value > bestValue) {
bestValue = rate * value;
best = i;
}
});
wins[best] += 1;
}
return wins.map((w) => w / draws);
}
// Every arm keeps a floor so nothing stops learning;
// the rest of the budget follows the probability of being best.
function shares(pBest, floor = 0.05) {
const spare = 1-floor * pBest.length;
return pBest.map((p) => floor + spare * p);
}Simplified from the allocation code in Demander. sampleGamma(shape, scale) draws from a Gamma distribution; any statistics library has one.
A worked example
Three channels and the last 30 days, with opportunities and their pipeline value taken from the CRM. The figures are an example, chosen to show the effect of value weighting.
| Channel | Spend | Opportunities | Pipeline | Pipeline per €1 |
|---|---|---|---|---|
| €12,000 | 6 | €180,000 | €15.00 | |
| Google Ads | €9,000 | 5 | €100,000 | €11.11 |
| Meta | €4,000 | 1 | €15,000 | €3.75 |
Prior: 0.4 opportunities per €1,000, worth two opportunities of evidence. Posteriors: LinkedIn Gamma(8, 17) with a mean of 0.47, Google Ads Gamma(7, 14) with 0.50, Meta Gamma(3, 9) with 0.33. Pipeline per opportunity is shrunk toward the pooled average of €24,583 the same way.
What 200,000 draws say
| Channel | Chance of being best, by opportunities | Chance of being best, by pipeline | Budget share with a 5% floor |
|---|---|---|---|
| 38% | 57% | 53.5% | |
| Google Ads | 47% | 30% | 30.8% |
| Meta | 15% | 13% | 15.7% |
Counted by opportunities, Google Ads looks strongest because it has the highest rate. Weighted by pipeline, LinkedIn wins more than half the draws because its opportunities are worth more. Meta has one opportunity and still wins about one draw in eight: with so little data it could be the best channel, and the floor keeps it funded enough to find out.
Choosing priors
The prior is what the model believes before your own data arrives. It matters most in the first weeks and for small channels.
-
01
Start from a benchmark
A flat prior treats a 50 percent conversion rate as just as plausible as 1 percent. Centre the prior on what the channel usually achieves for this kind of campaign. Demander derives it from platform benchmarks: the click-to-lead rate divided by cost per click, adjusted for the funnel stage.
-
02
Keep it light
Give the prior the weight of a few observations. Demander’s budget allocator uses two pseudo-conversions, so about a week of real data outweighs it. A heavy prior protects you from noise but makes the model slow to notice a channel that really is good.
-
03
Shrink new ads toward the campaign
For ads inside one campaign, a prior at the campaign’s pooled rate means a new ad starts at the average instead of at an extreme, and a lucky first few hundred impressions does not crown it. Demander uses one pseudo-success at the pooled rate.
Why weight by pipeline value
Ad platforms optimise toward the conversions they can see, usually form fills. In B2B, cost per lead and value per lead often point in different directions: the cheapest leads tend to come from audiences that rarely buy. A bandit that counts leads rewards whichever channel produces the most form fills, whatever happens to them in sales.
Counting opportunities is better, and weighting each one by its pipeline value is better still, as the example shows. Opportunity creation is a good event to learn from: it happens weeks after the click, carries a value from the CRM, and is frequent enough to steer weekly budgets. Closed-won revenue is the truer signal, but it arrives too late and too rarely to steer anything week to week.
Value is lumpy. One large deal can make a channel look brilliant for a quarter. Modelling value per opportunity as its own distribution, shrunk toward the pooled average, keeps a single outlier from taking over the budget. When no value has been recorded yet, optimise for conversions until it has; Demander switches between the two automatically.
What the production version adds
The textbook version gives each draw’s budget to one winner. Real budgets have diminishing returns: doubling spend on a campaign rarely doubles what it produces. Demander’s allocator models each campaign’s daily outcome as θ × spend^0.7, so doubling spend yields about 1.62 times the outcome. For each draw it works out the best split of the whole budget under those curves, and averaging the splits over thousands of draws gives the target.
Each campaign then moves only part of the way toward its target, in proportion to how one-sided the evidence is, so two campaigns that cannot be told apart do not swap budget on every run. Changes below 5 percent, or below one unit of currency, are skipped. The daily change limit (20 percent by default) caps every move, campaigns with under seven days of data keep at least 5 percent of the total, and if you set a maximum cost per acquisition, no campaign gets more budget while its cost per acquisition is more than 80 percent likely to be above it.
Every proposal carries its numbers: the expected conversions or value at the current budget, the marginal return compared with the rest of the portfolio, and the probability that more budget helps. It then goes through the guardrails like any other change, and your autonomy level decides whether it runs or waits for you.
Where Thompson sampling stops working
-
01
Delayed feedback
B2B outcomes arrive weeks after the spend. The model learns only from what has been recorded, so a channel whose opportunities are still forming looks worse than it is. Learn from the earliest stage that predicts revenue, and judge long-cycle channels over longer windows.
-
02
Small numbers
With a handful of opportunities a month, the distributions stay wide for a long time. The method still behaves sensibly; it moves slowly. Minimum spend and impressions before an ad is judged (Demander’s defaults are €30 and 1,000 impressions) stop it acting on noise.
-
03
A market that moves
The basic model assumes each channel’s true rate stays put. Seasonality, a competitor’s push or creative fatigue break that assumption. Using a recent window instead of all history, and checking for fatigue separately, helps.
-
04
Attribution decides what it learns
The bandit optimises whatever credit you feed it. If last-touch attribution hands every opportunity to branded search, the bandit funds branded search. See attribution models explained.
-
05
Lift needs a test
A channel that reaches people who would have bought anyway looks good to any method that learns from observed results. Only a holdout test, where part of the audience is kept out, measures what the channel added.
Questions people ask
Is Thompson sampling better than splitting budget evenly?
On average, yes: it spends less on channels the data already argues against while still testing the uncertain ones. In the first week or two, a light prior and an exploration floor make it behave much like an even split anyway.
How much data does it need before it moves budget?
It moves from the first day, but cautiously. With wide distributions every channel wins a fair share of draws, so the split stays close to even. In Demander, budget moves are also limited by the daily change limit, and an ad needs €30 of spend and 1,000 impressions before it is judged against the others.
Can I run it on conversions instead of pipeline?
Yes. Leave out the value distribution and the method counts conversions. Demander does this when no conversion value has been recorded yet, and uses value as soon as there is some.
Does Demander replace the platforms’ own bidding?
Thompson sampling decides how much each campaign gets; bidding decides how that money is spent inside the campaign. Per campaign you can keep the platform’s automated bidding or let Demander set bids from pipeline value. Either way, the offline conversions Demander sends back teach the platform which clicks became pipeline.