Transparency Lab

What the official data actually says

This section holds the difficult, derived and fragmented public data: figures that exist only in a workbook footnote, only after joining two official datasets, only across many authorities, or not at all. Everything here is reproducible from first-party sources. Where we calculate a figure ourselves it is labelled a Civometry calculation and shows its numerator, denominator, formula and vintage; it never looks like a government-published figure.

The product standard

Every statistic on this site can be followed along one trail:

  1. Map
  2. area
  3. statistic
  4. raw count
  5. rate
  6. numerator
  7. denominator
  8. formula
  9. official dataset
  10. exact source
  11. methodology
  12. limitations

Don't trust us. Check us.

Every controversial statistic resolves to one of four outcomes:

  1. the official figure
  2. a reproducible calculation from official figures
  3. the data is incomplete and here is what is missing
  4. the government does not publish enough to answer

We never claim that a figure is withheld on purpose. We classify how accessible it is and say what is missing.

Areas

  • Ethnicity and criminal justiceIn progress

    Exact criminal-justice measures by ethnicity (stop and search, arrests, prosecutions, convictions, custody, prison population, use of force, reoffending) from Home Office and Ministry of Justice tables, with rates per 100,000 calculated against Census 2021 ethnic-group populations and every numerator, denominator and vintage shown.

  • Religion and demographic changePlanned

    Census 2011 and 2021 religion and ethnic-group counts for every local authority, MSOA and LSOA, from the ONS Nomis API, with change shown in people, in percentage terms and in percentage points of share, and the absence of any official religion series between censuses stated.

  • Places of worshipPlanned

    Planning applications for places of worship read from the MHCLG Planning Data platform and local planning authority registers, classified only from the official proposal text, with application, permission, refusal and confirmed completion kept separate and the authority coverage shown.

  • PlanningPlanned

    Planning application records by authority from planning.data.gov.uk, with the number of authorities supplying data, the share of England covered and the latest update per authority, so that no figure is extrapolated to a national total.

    • Planning applications by authority (planned)
    • 1 data gap
  • Government data gapsLive

    Every question this site has been asked of official data, classified by what is actually published: available, derivable, partial, fragmented, FOI required, not published, not collected, restricted for disclosure, or unknown, with what exists, what is missing and which authority holds it.

  • FOI-derived datasetsIn progress

    Datasets assembled from Freedom of Information responses where no publication exists, with each request, authority, deadline, response and released file recorded, and coverage shown by authority so that a partial set of responses is never presented as a national figure.

  • Cost of living and household budgetsIn progress

    Household spending averages by household type, income decile and region from the ONS Living Costs and Food Survey, local rents from the Price Index of Private Rents, Band D council tax by billing authority and average house prices, assembled into Civometry estimates that state the geographic level of every input and are labelled as estimates, never as official figures about any household.

Source accessibility levels

Each registered source carries a level from 1 to 7 describing how its figures reach us. The level says nothing about why a publisher chose a format. Levels 4 and above describe figures rather than files: a level 4 figure is one we calculate from two or more official datasets and document as a data recipe; levels 5 to 7 are recorded in the data-gaps register, not against a source.

LevelMeaningExampleSources
1Official API. Published through an official programmatic interface that returns the figures directly, with codes and periods.ONS Nomis API for Census 2021 tables22
2Official machine-readable download. Published as CSV, JSON or a zip of such files that can be read without locating cells in a layout.data.police.uk monthly archive4
3Spreadsheet requiring extraction. Published in a workbook (XLSX or ODS) where the figure must be located by sheet, table, row label and column header.ONS Crime in England and Wales appendix tables; Home Office and Ministry of Justice ODS tables71
4Multiple official datasets joined. No single publication gives the figure; it is calculated by joining two or more official datasets (for example a count and a population).Recorded crime per 1,000 residents by local authority
5Fragmented across authorities. Held by many separate authorities in different formats with no national compilation, so coverage must be tracked per authority.Planning applications for places of worship
6FOI required. Held by a public authority but not published; obtainable, if at all, through a Freedom of Information request.Requests recorded on /foi
7Not published. Not published by any authority, and in some cases not collected.Ethnicity of the offender for police-recorded offences with no identified suspect
Level 1 sources (22)
Level 2 sources (4)
Level 3 sources (71)

Local recorded-crime rates per 1,000 residents are a level 4 figure: data.police.uk street-level counts (level 2) joined to the ONS mid-year population estimates (level 3) through the ONS LSOA lookup. The population year and the coverage of every force are shown on each local page.

Data gaps

47 questions are classified in the data-gaps register:

Freedom of Information

Where a figure is held but not published, we ask for it and record the request, the deadline, the response and every released file. No requests have been sent yet; the candidates are listed on the FOI page. See /foi.

Challenges and corrections

Any statistic can be challenged on methodological grounds: the official source, the numerator, the denominator, the geographic or category mapping, the period, the formula or the interpretation. Accepted challenges produce a public correction that records the original value, the corrected value, the affected geography and the reason, and the original is never silently replaced. 1 correction published. See /transparency/challenges.