How to stop data from driving government mad

Patryk Kosmider – stock.adobe.co

The Institute for Government has recommended reforms to the way Number 10 operates – and a new way of managing and using data in the AI age is central to its proposals

Matthew Juniper

Published: 19 Mar 2024

Spare a thought for Dominic Cummings – it was data that drove him mad. Data drives us all mad. For a field to be successful today it must curate, store, and share data across disparate teams, which is maddeningly difficult. How might governments learn from this?

The Institute for Government’s Commission on the Centre of Government has revealed how underpowered Number 10 is, not least in data analysis. Its report into reforming the institutions at the centre of government recommends creating a Department for the Prime Minister and Cabinet and, within that, a Joint Analysis and Assessment Centre (JAAC) to assess and analyse data for better decision-making.

This would be a start, but everybody in large organisations knows that top-down initiatives from the centre rarely work well at the coalface. If the JAAC is to be effective at converting data into information, what insight could it glean from structures that have evolved to do this? And what could it learn from scientific fields that manage this successfully?

First, deep neural networks learn by repeatedly passing information back and forth until every neurone is tuned to achieve the same objective. Information flow in both directions is the key.

The free flow of data and information between the centre and departments will be key

Matthew Juniper

Neil Lawrence, DeepMind professor of machine learning at the University of Cambridge, notes that in government, “People at the coal face have a better understanding of the right interventions, although not what the central policy might be; a successful centre will have a co-ordinating function driven by an AI strategy, but will devolve power to the departments, professions, and regulators to implement it.”

Or, as Jess Montgomery, director of AI@Cam says: “Getting government data – and AI – ready will require foundational work, for example in data curation and pipeline building.” The free flow of data and information between the centre and departments will be key. This is an obvious point, which is far easier for computers than for people, but it must not be forgotten.

Second, science and technology are team sports played with equipment developed over generations. Thousands of years of human endeavour are embedded in a computer chip and in the algorithms that run through it. As this knowledge accumulates, so does the value of the expertise required to navigate it. In science, as in sport, successful teams contain players with different but overlapping skills. No individual can win alone.

Deep expertise

The implication for JAAC is that deep expertise in several areas needs to be at or behind the table where decisions are made. Talented generalists who can turn their hand to anything have their place, but recruitment and career progression in JAAC must be strongly informed by experience and expertise accumulated over years or even decades.

During our interviews at the Institute for Government we heard that government departments sometimes pay the National Cyber Security Centre to recruit employees with the expertise they require and then second them back. This is so that the departments can avoid civil service recruitment rules, which seem to prevent experience being valued.

Just as a Masters in international relations does not make you a diplomat, a Masters in machine learning does not make you an AI expert. When people are evading HR’s recruitment rules, it is time to change those rules. Perhaps the biggest test for the next government is not what it does immediately, but how it recruits for the long term.

It is over four years since Dominic Cummings called for “weirdos and misfits” to join him at Number 10. Post-Covid, and with a potential new government, we have the desire to bring data to the heart of governance.

This will be unglamorous work, requiring skills different from those at DeepMind – human problems with fuzzy objectives rather than toy problems with precise objectives. The UK will be good at this, but it will not be showbiz for geeky people. That’s what SpaceX is for.

Dr Matthew Juniper is a professor in the engineering department of the University of Cambridge. He was a commissioner on the Institute for Government’s Commission on the Centre of Government.

Read more on IT for government and public sector

CDDO will assess delivery of government digital projects