Lucky Dip, Golden Tickets and chasing
How the sample is drawn and chased · 13 November 2026
Prepared by Mark Ainsworth with Claude (Opus 5.5). Mark made the decisions, with advice from the analysis team; Claude drafted the text, which Mark reviews.
This page sets out how the census draws its random sample of towers, who chases them, and the rules that keep the measurement honest. It is a companion to the implementation design, which describes the census as a whole and points here for this part. The two will eventually become chapters of one design document (see Part 3).
The advice behind these decisions came from the analysis team: Richard Heys, Matthew Greenaway and Ed Gosden, who between them have run large survey field forces and clinical studies. Their views are attributed where they shaped a decision, including where they differed. The decisions are Mark’s.
In short. We draw 1,000 towers at random, balanced so that every association, kind of place and size of ring gets its fair share, and chase every one until we know its state, because the towers we fail to reach are exactly the silent ones the sample exists to measure. Two reserve boxes of 500 are drawn at the same time, and opened at the 9am cut-off on 30 November only if a lot of towers reply on their own. Towers are shared out by the time each enumerator offered and how near they are to where they ring. Each enumerator’s job fits on a postcard: here are your towers; get the tower survey filled in; if there’s no band, fill it in yourself; and help one band, your Golden Ticket, get everyone to reply.
- Frame. Every tower in Dove’s Guide with four or more bells, about 5,700. Every tower and every ringer is invited to take part.
- The two surveys. The tower survey (one return per tower, from the tower rep) has a Core of about 5 minutes (the state of ringing, the band or roster, counts for census week) and an optional Tail. The individual survey is for every ringer.
- Voluntary response. Towers and ringers who reply unprompted are not typical: keener, better connected, more often thriving. Counting only them gives the wrong total and the wrong mix.
- Lucky Dip. A random sample of towers that enumerators chase until each is accounted for. Comparing the ones that replied on their own with the ones that needed chasing shows how voluntary replies are skewed, so the model can correct for it.
- Golden Ticket. One of each enumerator’s Lucky Dip towers, where they help the captain get every band member to do the individual survey. A complete band shows how voluntary ringers differ from the rest, on every question at once.
- In 1988 the sample replaced the population; in 2026 it calibrates it.
- Enumerators. About 235 volunteers, who chase through the tower rep or clergy, never individual ringers. Coordinators support them in an area.
- Dates. Census week is 16-22 November. The surveys open on 23 November. Chasing starts at 9am on 30 November. The published deadline is 6 December, and chasing carries on after it.
- One source of truth. A tower’s state is whatever QuestionPro holds for it. Whatever an enumerator finds out, they record by filling in the tower survey for that tower themselves.
Where things stand
| # | Point | Status | In one line |
|---|---|---|---|
| 1 | What “the sample” means | Decided | The towers drawn, not the towers chased |
| 2 | Size | Decided | 1,000 core towers, chased to completion |
| 3 | The reserve | Decided | Two boxes of 500, drawn with the core; opened at the cut-off if spontaneous response reaches 40% and 60% |
| 4 | How to draw | Decided | The cube method, balanced nationally and within each association; overseas its own stratum; core and boxes nested (D-074) |
| 5 | When enumerators get towers | Decided | 23 November; look up contacts, don’t make contact |
| 6 | Who gets towers | Decided | Enumerators who have checked in, by 19 November |
| 7 | First allocation | Decided | Every core tower, by time offered and nearness to people’s ringing (section 6) |
| 8 | Moving towers when someone drops out | Open | When to move is settled; who takes them is not |
| 9 | Golden Tickets | Decided | One per enumerator, from their own list, chance in proportion to bells |
| 10 | Completeness | Decided | Aim for 95% of chased towers, 90% the floor |
| 11 | Emails to enumerators | Decided | One each morning |
| 12 | Own-band towers | Decided | Go to their member, released on 29 November |
| 13 | Coordinators | Open | Groups of whole associations, about √E; the role to settle with Elva |
| 14 | Naming nearby enumerators in other associations | Open: minor | Committee, 13 November |
The flow at a glance
Part 1. The argument
1. The Lucky Dip calibrates, so completeness is everything
Anyone can answer the census, and it costs nothing when they do. But the towers that reply unprompted aren’t typical. A thriving twelve with a keen secretary replies in the first week; a village six that rings once a month may never reply at all. The Lucky Dip measures that skew. We draw towers at random, chase each one until we know its state, and compare the ones that replied on their own with the ones that needed chasing. That comparison, made by tower type, is what lets the model correct every other tower in the frame.
Two things follow. First, the sample is the towers we drew, not the towers we chased. A tower that replies on its own is still in the sample, and its reply is half of the comparison (Richard’s point, and the logic of Hansen and Hurwitz’s 1946 follow-up design). Second, a tower we drew but never reached is not just a missing return. It is a hole in the very thing we use to correct for missing returns, and nothing downstream can fill it.
2. A core of 1,000, chased to completion, and a reserve for a good problem
Enumerator effort can go on chasing more towers or on chasing the ones we have more thoroughly. The two kinds of error behave very differently. Chance variation shrinks as the sample grows. Bias from towers we drew but never reached doesn’t shrink at all, and it grows as soon as people have more than they can finish.
The core is 1,000 towers, about one in six, double 1988’s 500. That is roughly six towers for each enumerator, the committee’s rule of thumb for a job that feels worth doing without feeling like too much, and it is already enough for the national figure: if about half need chasing, the correction for non-responding towers is pinned down to within a couple of per cent. A bigger sample would mostly buy sharper association-level figures, and those will lean on pooling across associations whatever we do. Ed, by email, favoured a larger fixed sample as the simple compromise; Richard noted that around 1,300 is where sampling theory is comfortable, and that if enumerators manage five each rather than six we still end up with about 1,100.
The biggest unknown is how engaged ringers will be, not any one enumerator. If the publicity cuts through and most towers reply in the first week, many enumerators will be left with nothing to do. If it doesn’t, 1,000 may already be a stretch. We will know which within a few days of the surveys opening. Richard and Matthew both argued for a reserve to cover the first case: “a second bite of the cherry”, in Matthew’s phrase.
So two reserve boxes of 500 are drawn with the core, and there is one decision, at the 9am cut-off on Monday 30 November: open neither, the first, or both. One box is the expected case if engagement is high; the second is there for a runaway success, which seems unlikely.
- Which towers are released is decided by chance alone. A box is opened whole. Picking particular reserve towers because there is spare capacity nearby would tie the sample to where enumerators live, and towers in areas with no spare capacity would have no chance of release at all.
- When to open them is set in advance. Box A opens if the core’s spontaneous response has reached 40%, box B as well at 60% (D-072). That keeps between roughly 600 and 1,000 towers to chase per 1,000 core, whatever the response.
- Who chases them is where we can be clever. Released towers are shared out on top of what people still have to chase, by the same rules as the first allocation (section 6): nobody’s existing towers move, and the people with the most time left take most, as close by as possible. A box of 500 leaves plenty of room to keep that local. Who holds a tower doesn’t change whether it is in the sample.
- One decision, at the cut-off. By then general engagement will be clear. Deciding then means reserve towers share the core’s cut-off, so “voluntary” means the same thing for both, and nothing about what enumerators are asked to do changes at or after the published deadline.
- Enumerators hear about it once, up front. Their first email says that if a lot of towers reply on their own, we may give them a few more to chase from 30 November (Richard’s suggestion). Ed had already heard sharp comments about the sample size seeming to change; saying it once, early, is the cure.
How complete is complete enough
The aim is 95% of chased towers accounted for, and 90% the floor.
Reference points. Hansen and Hurwitz’s method assumes every unit in the follow-up sample responds; it is only unbiased if it does. US federal standards require a formal check for non-response bias whenever a survey’s response falls below 80%, and our follow-up sample is our check, so it deserves a higher bar. In 1988 the census got band counts for 471 of its 500 sampled towers (94%).
A rough worked case. Suppose towers that didn’t volunteer hold about 40% of all ringers, and the chased towers we never reach have half the ringers of the ones we do. The national total is then out by roughly 40% × 50% × the unreached share. Chance variation with 1,000 towers is around ±2%.
| Chased towers accounted for | Worst plausible bias | Against chance variation (±2%) |
|---|---|---|
| 80% (the US federal trigger) | about 4% | Twice as big: bias dominates |
| 90% (the floor) | about 2% | About the same: the balance point |
| 95% (the aim; 1988 got 94%) | about 1% | Half as big: chance dominates |
If the unreached towers mostly have no band at all, every bias figure doubles, which is why we aim at 95%. An enumerator’s own return helps twice over: a tower they have returned as having no band is accounted for, and it is exactly the kind that would otherwise go missing.
Matthew’s caution: don’t be slavish about the number. A random 10% missing matters far less than 10% that is mostly small rural towers. So the outstanding towers are watched by place type, ring size and association, not just counted.3. How the Lucky Dip is drawn: random, and fair to every association
Decided (Mark, 13 November, D-074). The Lucky Dip is a random draw, balanced so that it is fair in every way that matters, with none of them put first:
- every combination of place type and ring size gets its fair share nationally;
- every association gets its fair share of towers;
- within each association, its own mix of villages and hamlets, towns, urban centres and urban suburbs, and of rings of 8 or more, gets its fair share.
Within those rules, which towers are drawn is pure chance, and every tower in a stratum has the same chance. The method is the cube method (Deville and Tillé 2004), which INSEE, the French statistics office, uses to draw the rotation groups of its census. The core and both reserve boxes are three nested balanced draws, so whichever boxes open, the towers chased are a fair sample. The seed, 5040, was fixed before any draw.
Two strata. Overseas towers (Australia, the USA, Canada and New Zealand; D-075) are drawn as their own stratum, inside the 1,000 core, with twice their proportional share. That is Mark’s 9 November doubled chance, expressed as a fixed number: each overseas tower’s chance, and so its weight, is the same either way, but the stratum fixes the overseas count exactly and keeps those towers out of the main balancing. Whether the island of Ireland belongs with them is open; for now it is drawn with Great Britain.
What the advisers said, and how the decision answers them.
- Matthew proposed explicit strata of urban or rural by number of bells, a systematic draw sorted by postcode within each, and no balancing on association. His strata become balancing cells, so each combination still gets its fair share without fixed cells to merge. Balancing within associations gives each association a fair mix of its own places and rings, much as his postcode order would.
- Richard agreed, warning that association is tangled up with ring size (East Anglia’s medieval wool wealth left it more eights and tens than the West Country), so balancing on association alone could skew the draw. Balancing association, place and ring size together, including each association’s own share of large rings, answers it: East Anglia gets its eights and tens, the West Country its sixes. His check afterwards, that each association’s share is about right, is cheap and stays in.
- Ed, by email: bite the bullet, use the cube method, and explain it carefully. He also flagged towers that belong to two associations (Surrey and the Guildford Guild; Lancashire and Carlisle in Cumbria). The frame gives each tower one association for this purpose.
- Mark wanted ring size and place type to count without the draw looking skewed by association, because associations are how ringers see the census and how enumerators organise, and their own association’s figures are the ones people will look at most.
Why this method. Eight methods were compared by simulation in the analysis chapter Choosing a sampling method, on two stand-in populations built from 1988’s tables and one shaped by an association’s register. This one was the best of all for association figures and for balance within associations, best or near best on every national fair share, and within noise of the best for national figures. It does not sort or spread the towers, so the draw neither seeks nor avoids tower clusters. Its accepted cost: on the register-shaped stand-in, the sorted list logged earlier (D-042) was more precise for national figures. That list also missed ring-size shares by about 9 towers, put the three factors in a fixed order, and kept neighbouring towers apart. The code is in R/draw_designs.R.
4. Keep the enumerator’s job small, and the same for everyone
Towers go out on Monday 23 November, as promised. The first week is for looking up contacts (Dove’s Guide, the association, the church website), not making contact, because a word in the wrong ear during the voluntary week turns a voluntary tower into a chased one. Enumerators are told plainly that some towers will reply on their own and drop off their list, which is good news. Richard’s framing: a provisional list on the 23rd, no chasing until the 30th, and a win for everyone. Ed, by email: stick with what was promised.
Only enumerators who have checked in get towers. A tower given to someone who never reads our emails loses a week before anyone notices, and those are the towers most in need of chasing. The check-in confirms the person is active, that our emails reach them, where they are, and that they agree to their details being shared with other enumerators. You can’t board without checking in. Messages ask for check-ins by 13 November, to create some urgency; the working deadline for the first allocation is Thursday 19 November. People who check in later can take towers when some need moving.
Every core tower is allocated on 23 November to checked-in enumerators, with lists sized to the time each offered, as near as possible to where they ring, one of them the Golden Ticket (section 6). Where enumerators are thin, the nearest are further away. That works, because chasing is mostly by phone and through people who know people. The one exception to the timing is a tower whose band the enumerator belongs to: it goes to them, but is released on 29 November (D-070). Every tower has one accountable enumerator, but finding a way in is a job for the whole group: if nobody lives near Swansea, the question becomes “does anyone know someone in Swansea and Brecon?”. Richard added that an association with no enumerators, or one that has lost them, can be asked to find some, and that an association that can’t is itself telling us something about the health of ringing there.
The cut-off is 9am on Monday 30 November, when someone is at a desk to act on it. It counts Sunday-evening returns as voluntary, which is when many captains will sit down to do them. By then most enumerators should have two or three towers left to chase. In 1988, 77 agents covered 500 towers, about six and a half each, so this is well within what volunteers can do. The problem won’t be average capacity but the spread: some people will do a lot, some nothing.
Everything an enumerator needs comes in one email each morning, sent when there is something to do, and every Monday regardless. It lists each tower still without a return, what to do about it today, and who to call for help. The advice moves on with the days since chasing started: encourage the captain to fill in the tower survey (about 5 minutes) or pass it to whoever can; then try another route, such as the vicar or a churchwarden; then, if there is no band or nobody to reach, fill in the tower survey for that tower yourself. A second daily email would double the spam exposure and, as Richard put it, double the workload for no gain. Every word of every version is in config/email_text.md, and the email gallery script shows each one as it would be sent.
5. Four rules keep the measurement honest
- The sample is the towers drawn, not the towers chased. Voluntary towers stay in; they are half of the comparison.
- Golden Tickets are drawn in advance, from each enumerator’s own list. See below.
- A tower never moves because it is hard. A reluctant captain is the measurement, not a problem to route around. Towers move only when an enumerator steps down, goes quiet, or has a conflict, and every move is recorded.
- No contact with listed towers before chasing starts, and we check rather than trust it. The tower survey asks self-completers what prompted them, separating general reminders from someone contacting them about their tower in particular.
And one point of morale, from Ed: plenty of towers have no correspondent and ring only now and then. Pinning down “no band” is a finished job, not a failure, and once the enumerator has filled in the tower survey to say so, the emails count it as done. Association lists of non-practising towers are useful leads, but only to confirm: marking a tower silent without contact would take it out of the sample on the strength of its own status.
Golden Tickets: a sample of the sample
Each enumerator’s Golden Ticket is one of their own core Lucky Dip towers, drawn with chance in proportion to bells (our best guess at band size before we know it), whether or not the tower has already replied. It is on their list like any other tower; once its tower return is in, the aim there becomes a reply to the individual survey from every member of the band. Captains are told totals (“5 of 11 have replied”), never names.
The alternative was to wait for towers’ returns and draw Golden Tickets from those with big bands. Matthew and Richard both advised against it: the selection would then depend on which towers reply early, risking the very bias the Golden Tickets exist to measure.
| Situation | Swap the Golden Ticket? |
|---|---|
| No band, bells unringable, or a roster under 3, as shown by the tower return (including one the enumerator completed) | Yes, to the next tower in that enumerator’s ordered draw |
| Captain reluctant, band slow, enumerator struggling | No. That is non-response, which is what we measure |
| Enumerator steps down | No. The Golden Ticket moves with the tower to whoever takes it on |
The draw within each list is made as an ordered sequence, with chance in proportion to bells, so “skip the ineligible, take the next” keeps every tower’s chance of selection known. A replacement is recorded as a second-round Golden Ticket, so the analysis can check whether swapping biased anything (Richard). It never comes from the reserve, so the reserve stays clean.
Lists now vary in length with the time people offered and the towers near them, so a tower’s chance of being a Golden Ticket does depend on its list: a tower in a list of three has a better chance than the same tower in a list of eight. That is fine for the analysis as long as it is recorded. Each tower’s chance is known exactly from the list it was drawn from (its bells over the list’s total), own-band towers included, and the analysis weights by it.
Part 2. Still open
Moving towers when someone drops out
When to move a tower is settled:
| Situation | Move it? |
|---|---|
| Enumerator steps down, is ill, or is away for a week or more | Yes, all their open towers |
| No return after about a week, and the enumerator doesn’t answer their coordinator | Yes |
| A conflict: they know the captain or incumbent personally, or it is their family’s church | Yes |
| Captain reluctant, no answer, band in difficulty | No. Keep going; fill in the tower survey yourself if need be |
| The enumerator would prefer an easier or closer tower | No. That is how a random sample stops being random |
Who takes them on is not. Local relationships are how chasing works, and the census is also about activating associations, not just measuring them, so the lean is towards keeping a tower within its association. The worry is that reassignment becomes a route for bias if the towers that need moving aren’t a random set. Mark’s working view is that moves will come in a few small waves, leaving room to place towers without skewing anything. Ed’s email favoured longest-waiting first for thin areas, from the same concern about bias. Matthew suggested being pragmatic near the end, using counts of outstanding towers by area.
How a tower is moved. Few moves are expected: when someone’s towers are used up, they are given fresh towers from a reserve box, not someone else’s. When a tower does have to move:
- The census team or the area coordinator changes the tower’s Enumerator in the Census towers List. If someone drops out, setting their Status to Withdrawn in the Census enumerators List puts all their open towers under “To be reassigned”, to be given out the same way.
- The same day, the person who made the change emails both enumerators (the one losing the tower, and the one gaining it). Nothing is sent automatically to the person losing it; the tower simply drops off their next morning’s list.
- The next morning’s email to the person gaining it marks the tower “(added today: reassigned to you)”, in case step 2 didn’t reach them. R reads this from the List’s version history, so nobody keeps a separate record.
- The census team checks the “To be reassigned” towers each day until none are left.
The List’s version history records who changed what, and when. Separate evening emails to both people were considered and set aside (9 November): more emails and one more routine, for something rare.
Smaller points on sharing out
- What coordinators look after: their enumerators, the towers in their group, or both (with Elva).
- The settings themselves. The effort curve, the baseline, the hours cap, the 1.25 for experience and the 10% slack are judgements. The simulations show how each moves list lengths and distances; the analysis team may want to review them before 23 November.
Minor: what counts as “no band” for a Golden Ticket
The working line is a roster under 3. Two people “is not a band at all”.
Part 3. Reference
Timeline
| When | Central team | Enumerators |
|---|---|---|
| By 19 Nov | Check-ins in | Check in |
| Before 23 Nov | Draw the core and both reserve boxes (cube method, seed 5040; D-074) and each enumerator’s Golden Ticket order; commit the code first | Nothing yet |
| Mon 23 Nov | Surveys open; lists out by email | A list sized to the time they offered, including a Golden Ticket; distant towers marked. Look up contacts; don’t make contact. Reply “got it” |
| Sun 29 Nov | Own-band towers released (D-070) | Nothing yet: they appear in the next morning’s email |
| Mon 30 Nov, 9am | Cut-off: returns so far are voluntary, fixed for good. Open box A at 40% spontaneous response, B at 60% | Chasing starts. Voluntary towers shown as done |
| From 1 Dec | Released reserve towers shared on top of existing lists; towers moved when someone drops out | One email each morning |
| Sun 6 Dec | Published deadline; may be extended | Chasing carries on |
| End of chasing | When completeness reaches the aim, or by 21 December | “You’re done, thank you” |
Where every sampled tower ends up
| Outcome | Defined as | What it gives the analysis |
|---|---|---|
| Voluntary | Core return by the cut-off | The unprompted response rate, by tower type |
| Chased | Return after the cut-off | What the voluntary returns are missing |
| Enumerator’s return | The enumerator fills in the tower survey: no band, nobody reachable, or what they have established locally | The silent towers voluntary returns never show |
| Unaccounted | None of the above when chasing ends | A hole in the calibration: drive it towards zero |
What the emails never do
| The emails never… | Because |
|---|---|
| Prompt contact with a listed tower before chasing starts | The voluntary week must stay clean |
| Push the whole band anywhere except the Golden Ticket | Golden Tickets only tell us something if the others weren’t pushed |
| Show names, ages, or anything about under-18s | Safeguarding and data protection |
| Suggest contacting an individual ringer | Contact is through the tower rep or clergy only |
| Mention another enumerator’s towers | Privacy |
If the returns data is stale, nothing goes out: telling someone to phone a tower that has already replied is worse than silence.
Ideas noted for later
| Idea | From | Where it goes |
|---|---|---|
| BellBoard activity (peals, quarters, set-piece events such as VE Day and the Coronation) as a measure of tower activity | Mark, Richard | The model: a covariate known for every tower |
| Church of England attendance statistics by church | Mark | The model, if access is agreed |
| A weekly response thermometer, published in the Ringing World | Richard | Built from the same code as the emails |
| Repeating the census every four or five years, to see which towers grow | Richard | The final report |
| Code review of the draw | Matthew | Matthew to be added to the GitHub repository |
Considered and set aside
Ideas that seemed sensible and were cut to keep the census simple are in the register of ideas set aside, each with what would bring it back. Those that bear on this page include: extending the sample in blocks down an ordered list; regional queues for moving towers; withholding lists until after the cut-off; drawing Golden Tickets only from towers that have replied, or replacing them from the reserve; weighting the Lucky Dip towards small rural towers to find the true zeros (Richard: it answers a different question, and BellBoard activity in the model gets at it without bending the sample); two emails a day; and SharePoint workbooks for every enumerator; barring enumerators from their own band (D-069, replaced by D-070); measuring distance from an average of each enumerator’s towers, or from a shape drawn round them; a fixed cap on how far a coordinator group may spread; and choosing coordinators and their enumerators by an integer programme.
Towards one design document
This page and the implementation design overlap, and the overlap is deliberate for now. The plan is to fold them, with the question wording page, into one design document in chapters, each saying what was decided, why, and what else was considered, so that someone retracing our steps, or a critic, can see the reasoning as well as the result. The to-do is in design/_consolidation-todo.md.