The Office for National Statistics (ONS) is making major changes to how it gathers some of the UK’s most important data. These figures shape decisions on wages, benefits, and public spending.

One of the biggest shifts involves how inflation is measured, which is changing on Tuesday. The Consumer Price Index (CPI), which tracks the cost of everyday essentials like food, energy, and transport, is being updated with a new system that aims to capture price changes more accurately.

This matters because inflation figures influence the Bank of England’s decisions on interest rates, which in turn affect the cost of borrowing, savings, and even rent.

For workers, inflation also plays a role in wage negotiations. This is because when prices rise, there’s often pressure on employers and the government to increase salaries, pensions, and benefits.

The ONS will continue sending researchers to shops to check prices and speak to retailers, but from this month, a new digital system will speed up how the data is processed.

It’s also testing a new method using real checkout data from supermarkets. Instead of just recording shelf prices, it will track what people actually pay, including discounts from loyalty schemes like Clubcard and Nectar. This should give a more accurate picture of real spending habits, with full rollout expected by 2026.

The change has been brought about over concerns the previous method measured price changes but failed to capture how consumers changed what they buy as a result.

Take the example of butter, which has gone up in price by 18% in the past year. That increase was reflected in the CPI, influencing the overall inflation figure. However, many consumers will have switched to a dairy spread or margarine rather than keep paying for the more expensive butter.

How is inflation data changing?

While this should improve inflation accuracy, tracking individual product prices may become harder. Sky News’ Spending Calculator, which helps users track price changes, will need updates and won’t be refreshed this month.

An ONS spokesperson said: “From next year we will be replacing much of the physical price collection in supermarkets with information from supermarket tills. While we won’t know what each consumer has bought, we will know both the price and quality of items sold in shops up and down the country, marking a step-change in our understanding of inflation and consumer behaviour.”

Data reliability concerns prompt changes

While these changes to the inflation data are intended to better reflect consumer behaviour, other changes are being introduced due to concerns over reliability.

One of the most affected datasets is the Labour Force Survey (LFS), the UK’s largest household study, which measures the state of the labour market and helps shape decisions on interest rates and employment. However, plummeting response rates mean its reliability is now in question.

“I think policymakers just don’t have as much trust or confidence in the LFS, so they have to find other ways to get the clear insights they used to rely on the LFS for,” said Michael McMahon, professor of economics at Oxford and former Bank of England economist.

“The Bank of England has a set of regional agents who will go out and speak to businesses. They’ll speak to local bodies and even in some cases do citizens’ panels. They were doing that before the LFS issue. It’s just they have to rely on these alternatives more, because they can rely less on the LFS.”

The pandemic accelerated these issues when face-to-face LFS interviews were replaced with phone surveys, causing a sharp drop in participation.

Internal ONS emails, revealed by the Financial Times, showed how one key estimate’s sample size had “collapsed to only five individuals” — too small for reliable statistics.

Resolution Foundation analysis shows that HMRC payroll and self-employment data aligned with LFS estimates before 2020 but after the pandemic began to diverge.

To address this, the ONS is developing the Transformed Labour Force Survey (TLFS), using shorter questionnaires and shifting primarily to online responses, with some face-to-face interviews remaining.

Survey issues aren’t just affecting job figures, they’re also complicating GDP estimates.

The Living Costs and Food Survey (LCF), which tracks incomes and spending and is used for GDP estimates, has seen a sharp drop in response rates, with fewer than one in five forms completed as of December 2024. The survey is particularly time-consuming, requiring participants to log spending for two weeks. A new digital tool allowing receipts to be scanned is in development but won’t launch until late 2025.

For policymakers, these delays are frustrating. “It’s certainly a moment of embarrassment: the idea that the chancellor and the governor [of the Bank of England] go to G7 meetings, talk to other advanced economies, and explain why we don’t know how our labour market is doing with any great confidence,” said McMahon.

Flawed migration data

Recent improvements to the way the ONS gathers migration data also highlight significant failings in the recent past.

Long-term migration estimates are a vital part of public debate and key policy decisions.

Before COVID, migration estimates relied on traveller surveys at airports and ports. These surveys frequently underestimated migration levels, as they depended on people’s own predictions about how long they would stay in the UK.

Under this method, net migration was seen to have peaked in December 2022, at 764,000.

Now, the ONS has shifted to using visa records, higher education statistics, and tax data to provide a clearer picture. Under this new method, it has become clear that net migration has been much higher than previously thought, peaking at 906,000 in June 2023.

Overall, the ONS increased its estimate for net migration in 2023 by more than 25%.

However, while these changes are making migration data more reliable, they also highlight how much of the political debate on immigration in recent years has been built on incomplete figures. The transition to administrative data is a step forward, but further refinements will be needed to ensure long-term accuracy.

The Data and Forensics team is a multi-skilled unit dedicated to providing transparent journalism from Sky News. We gather, analyse and visualise data to tell data-driven stories. We combine traditional reporting skills with advanced analysis of satellite images, social media and other open-source information. Through multimedia storytelling we aim to better explain the world while also showing how our journalism is done.