Audit’s existential crisis

Published by Anonymous (not verified) on Fri, 25/05/2018 - 2:20am in



Audit’s existential crisis

Richard Murphy

Professor of Practice in International Political Economy, City, University of London

An address to the

British Accounting & Finance Association

28th Audit & Assurance Conference, Dublin, 24-25 May 2018



Many thanks to the organisers for asking me to speak today. I appreciate the chance to do so.

Recent audit failures in the UK have left the audit profession reeling, politicians baying and even the profession’s own UK based regulator asking about the future of the profession.

But it’s hardly surprising that this has happened. The recent history of auditing is one of a collective retreat from responsibility as auditors, and their captive regulators, have sought to ring-fence their risk in pursuit of their own self-interest. In the process they have undermined their whole social purpose, and so their reason for being.

Audit is now in existential crisis. It has to face that fact or it is at risk of ceasing to be, with potentially catastrophic consequences for the world’s markets, both financial and tangible.

The time for audit caution is over: it’s reform or bust now. The only questions left are about how to achieve that goal.

These are my themes this afternoon.

A matter of perspective

What I stress is that I have been asked to speak as a practitioner today. Some of you will be aware that I now have an academic role and title to go with it, but I have had a practicing certificate as a chartered accountant for a lot longer - since 1985, in fact. Please don’t doubt it - accountancy and audit practice is something to which I am dedicated.

That’s why it troubles me to tell you that in my opinion - and I won’t be beating around the bush here - accountancy and audit in the UK face an existential crisis. No other description is appropriate.

I want to make clear I am not just talking about some recent failing here: the succession of firms that KPMG seem to have audited that have gone to the wall after their audit reports have been issued; the disaster of Carillion, and the dismal failure of the Financial Reporting Council to show any initiative at all in the face of such issues arising are not what has created a crisis. Important as all these things are I suggest to you that they are symptomatic of a much greater malaise that is what really needs to be addressed.

The greater issue is one that I suggest that anyone who has interest in saving the market economy should rally to support. That issue is that without data; without is verification; and without trust that the data supplied is both useful and reliable, then markets as we know them will cease to exist.

What do I mean by that? I suggest that:

  • People will stop saving - at least in shares;
  • The cost of bond finance will rise, not because the risk free rate will rise but because the risk premium does;
  • People will not trust the assurances of big business on almost any issue - and so real, on the ground investment will be harder to deliver;
  • Existing employment structures - which depend on a high degree of trust that people will be paid - will be strained;
  • The long term ability of markets to meet need will be challenged because faith in the durability of companies will be shattered;
  • Tax revenues will be imperiled.

In that case audit assurance matters. But that means we must understand just what we expect from both accounts and audit and then work out, afresh, how to deliver against that expectation.

What is accounting?

In principle accounting is simple: it relates to the stewardship of funds. It does not matter why the funds were entrusted - accounting principles are unrelated to the profit motive or markets. What accounting data must do is supply is information that says:

  • Who is accountable;
  • With what they have been entrusted;
  • What they did with it;
  • With what outcome;
  • And for whom that has consequence.

Note that this says nothing about numbers: the narrative of accounting is at least as important as any data. That is because almost any accounting has these stakeholders:

  • The providers of funds;
  • Those from whom purchases are made;
  • Those to whom services are supplied;
  • The people engaged in the process;
  • The communities of which they are a part;
  • The governments that those communities choose to represent them and their:
    • Regulators;
    • Law enforcement agencies;
    • Tax authorities.

Accounting is the process of reporting relevant, reliable, complete, comparable and comprehensible data of use to these stakeholders. Ignore the technicalities: this is what it’s really about.

What then is audit?

If accounting is reporting audit is the process of testing the data to be supplied by the person preparing an account to ensure that it truthfully and fairly reflects the actions of the person reporting in a fashion that meets the needs of the likely user of the data

What has gone wrong with accounting?

None of that is rocket science, so what has gone wrong?

There are a number of completely straightforward answers to this question. Let’s not spend hours agonising on this: the answers are almost as simple as saying the Emperor is wearing no clothes.

First, accounting has restricted the stakeholders to whom it has responsibility to shareholders alone. Everyone else has been abandoned: they are told they must make do. The social foundation of accounting was abandoned at that moment and the consequences are now clear.

Second, accounting has denied access to data: the profession has cheered as the amount of accounting data available to stakeholders has been steadily reduced. Around the world opacity is becoming the norm, not just in tax havens but in the company registries of Europe. A clearer exercise in self destruction of professional relevance is hard to imagine: the small business community has already been forced to work in the dark, and so too have far too many tax authorities.

Third, accountancy has resorted to faux science. When economists copied physicists they tried to build financial models. When accountants copied economists they delivered what I call CRAP. That, I assure you, is a technical term: it stands for Completely Rubbish APproximations to the truth. And that, as we now know all too well, is what a great deal of accounting is.

Fourth, in the process accounting forgot the thing that made it a profession: that is, its ethics. Rules were substituted and in the process the loss of a single world changed everything: prudence went out of the window and with it went any sense of judgement, responsibility and duty to engage with the task in hand. Purely mechanical approaches were considered sufficient. We have all paid the price for that.

So how did auditing fail?

Audit has failed because, unsurprisingly given their common commercial roots at present, it has followed a similar path to accountancy.

In particular, auditors have sought to ring-fence their liability liability so that very few stakeholders had any chance of holding an auditor accountable for their opinion.

And auditors also abandoned a key principle: in their case it was true and fair that went out of the window. What was substituted was compliance with an accounting framework: a rule based audit. The law might have said true and fair was required: I put it to you that the profession substituted its own inappropriate, and quite probably illegal, interpretation of its duty that has failed society.

In a nutshell

To put it in a nutshell, all branches of this profession have sought to act in what they see as their own rational self-interest, which has been achieved by transferring much, if not all, of their risk arising from their duty to stakeholders back onto those stakeholders, to whom they then deny any duty of care. The profession has, as a result, effectively ceased to be. The Big 4 firms, the tax havens that they promote and underpin, the professional bodies that they dominate and the regulators that they populate with their alumni; all of them have played their part in this wilful act of professional self-destruction.

What is left is just a husk: a body of technicians without an ethic and now without a social purpose.

The question is, what are we to do?

What are we to do?

I could spend all evening discussing this so let me offer highlights instead, and then leave time for questions.

First, we have to say that accounting is about meeting the needs of all stakeholders. And I mean all. And just to make sure my point is really heard I stress again that by this I mean:

  • Customers;
  • Suppliers;
  • Employees;
  • The communities that host the activities that companies undertake;
  • The governments and regulators of those communities;

And, last of all – because they are the most transient of stakeholders in a great many of our corporations and therefore those with the lowest priority:

  • The suppliers of capital, and their advisers.

In other words, accountancy has to be turned on its head as to priorities.

Second, accounting has to be about delivering decision-useful information - not technically fancy data that only a tiny number of people understand and the rest have to take on trust - which they are no longer willing to do. Forget IFRS: if anything is destroying this profession, it is.

Third, accountancy has no right to ever again ask directors to sign accounts that they cannot understand - which is what we do from top to bottom of this profession right now, from the FTSE 100 right down to the SME because of some of the absurd requirements of new accounting standards.

Fourth, the idea that we are delivering information for investment appraisal when producing audited annual financial statements has to go, once and for all: what is apparent is that in terms of the market worth of most companies accounting data makes little impression on value. Broader sentiment does that. In which case accounting data needs to meet other needs.

Fifth, it has to be appreciated that all these other needs all come down to durability, cash flow, ethical conduct and sustainability.

These, then, are the questions accounting data has to answer if it is to be credible once more:

  • Will this company survive in the long term and meet its obligations to me as a customer?
  • Will this company pay me as a supplier, in the country I am in?
  • Will this company pay me fairly as an employee, wherever I might be?
  • Will this company pay my pension?
  • Will this company pay its taxes in all the jurisdictions in which it works?
  • Is this company’s dividend legal, and sustainable?
  • Has this company the resources it needs to survive?
  • Is this company adding value to my community as a consequence of us hosting it?

What this means

What we need then is a new accounting framework to answer these questions - which IFRS does not - and make this new framework the accounting norm.

And then we must require that the accounts of all companies – whatever their size - comply with its requirements, even if we permit some, albeit minor, adaptations to suit scale.

And we must ensure that those accounts are available on public record – it’s our job to demand that, in fact.

And we must require then that auditors use their judgement to ensure that the accounts do not just comply with this new framework – but do so prudently so that the interests of creditors of all forms – which most stakeholders are – might be protected as an issue of paramount importance if the abuse of human rights that is implicit in limited liability is not to become actual abuse.

How will we know a new framework is relevant?

We will know if the new accounting and auditing framework is relevant if:

  • It provides the data to answer the questions stakeholders have. To make it clear this requires full information for all companies on public record:
    • What does this company do?
    • Where does it do it?
    • Who owns it?
    • Who manages it?
    • Is it solvent?
    • Is it generating or using cash, so potentially leaving its creditors risk? So, yes – I make clear – the cash flow has to be at the heart of accounting because our decision to ignore it is one of reckless irresponsibility.
    • How does it treat its staff?
    • How much are they paid?
    • Are their pensions funded?
    • What is the gender pay gap?
    • Is this company investing in people and technology for the future?
    • Is this company profitable?
    • Is it paying its taxes, by which I mean all its taxes, and if not at the expected rate, why not?
    • If it is operating in more than one country, which are those other countries and how much this activity takes place in each jurisdiction?
    • What subsidiaries does it have?
    • Are those subsidiaries viable?
    • How much of the group’s trading is between related companies, and does this resulting profit shifting, or the prejudice of creditors?
  • Auditing is restored as an essential protection for all companies employing 50 or more people, with appropriate financial considerations as an alternative, in the interest of protecting stakeholders and society at large.
  • The primary duty of auditors is stated to be to all the stakeholders of the company, with the restriction to shareholders alone being removed.
  • Auditors must be appointed by a new statutory Audit Agency who will be responsible for both regulating and paying them as well;
  • Private sector companies will still be able to do audits but with all audits being put out to tender subject to:
    • a limitation on the proportion of the market that any auditor might secure: I suggest a maximum of ten percent of any part of the market;
    • mandatory audit rotation after five years;
    • a restriction on auditors providing any other service apart from the filing of statutory documentation;
    • auditors to be paid from a levy on the turnover all companies subject to audit with additional charges for high-risk sectors;
    • an obligation being imposed upon auditors to whistleblow with regard to fraud, tax abuse and other abuse of regulation.

How will we know things have changed?

We will know that the existential crisis in auditing is over when truly independent auditors are required express opinion upon account information that is of real use to the stakeholders who must depend upon it.

What will indicate when that has happened? Try this list:

  • There will be an end to the hegemonic thinking that accounting is all about the interests of a tiny proportion in society, which belief has brought the whole profession to its knees and which has rightly shredded our reputation;
  • We will have had the reform of company law to:
  • set up the new accounting framework;
  • establish a new, government backed, method of creating accounting standards;
  • create a new, independent, accounting regulator;
  • create the new statutory audit requirement managed by a new statutory Audit Agency to oversee, appoint and remunerate all company auditors;
  • There will be a reform in professional culture so the accountants finally understand that it is their duty to communicate useful information;

Only these changes can make accounting and audit relevant again.

Our existential crisis demands radical reform. I strongly suggest that we go and deliver it.

And one final thought: I am open to offers from anyone who wants to help the revolution that we need.

Jersey is going to lose on beneficial ownership: it really should have the sense to do so gracefully

Published by Anonymous (not verified) on Thu, 24/05/2018 - 3:17pm in

As Channel Islands newspaper The Bailiwick Express reported yesterday:

Jersey's financial services sector is being thrust back into the international spotlight with the publication of new report on how Russian money is invested overseas.

The report had been written by the Foreign Affairs Committee of the UK's House of Commons, and is entitled: Moscow's Gold: Russian Corruption in the UK.

It calls on the UK Government to set a deadline for the Crown Dependencies to make their registers of who actually owns local companies publicly available by 2020, and says that the government should be able to legislate for the islands in this area, ripping up the current constitutional position.

There are around 20,000 properties owned by Jersey-registered companies in the UK, and changing the rules would reveal the names of the individuals ultimately benefiting from those ownerships.

That's an issue for the Foreign Affairs Committee as they claim that some individuals who purchase UK property through offshore shell companies, do so to disguise their identities and potentially corrupt sources of their funding.

What can I say, except to note that as the Foreign Affairs Committee knows, we can legislate for the Crown Dependencies on issues relating to foreign affairs? In that case its threat is not idle; it is more like giving notice.

Jersey is going to lose this one: it really should have the sense to do so gracefully.

Beneficial ownership for the Overseas Territories is just the start: there is a long way to go to get corporate transparency

Published by Anonymous (not verified) on Thu, 10/05/2018 - 5:33pm in



My apologies: it's been a slow start to the day. It was, for me, a late evening. There was a small celebration of the victory in imposing beneficial ownership registers on the British Overseas Territories in Westminster last night and I arrived back at home realising that I may have been over the limit and in charge of a bicycle.

The celebration included the usual NGO culprits, but also a good selection of MPs and peers, and both Andrew Mitchell and Margaret Hodge had a few words to say. But the discussion was, of course, the point of interest.

First, I sense that MPs were misled on whether they could legislate for the Crown Dependencies. Let me be unambiguous: they can. The Supreme Court has said so and the Supreme Court, sitting as the Privy Council is these places ultimate court of appeal: this, therefore is binding. If in doubt the wording used is this:

The Channel Islands are not part of the United Kingdom but as Crown Dependencies enjoy a unique relationship with the United Kingdom through the Crown, in the person of the Sovereign. The UK government is responsible for their international relations and for their defence. The UK Parliament has power to legislate for the Islands but Acts of Parliament do not extend to the Islands automatically. Usually, the Act gives power to extend the application of the Act to the Islands by Order in Council, which will be preceded by consultation. For the most part the Islands legislate for themselves.

I added the emphasis. And we already know that we can legislate for the Isle of Man: it has been done.  It worries me that the chance to bring good order to the Crown Dependencies was lost on the basis of misinformation.

Second, I was heartened to hear that the campaign to get the Crown Dependencies to comply is already in full swing. In my opinion this just adds to the pressure on these Islands. I explained what this might be here, last December. It is the threat of EU sanctions that will now require the UK to bring them into line, Brexit or not.

Third, there was widespread awareness that bringing the Crown Dependencies and OTs into line with the standards at Companies House would be a hollow victory: Companies House is nearly universally accepted to be dire. There was much discussion on what to do about that. It's a theme I will be returning to.

Honest business has no problem with disclosure

Published by Anonymous (not verified) on Fri, 04/05/2018 - 4:46pm in



Two tales from the Crown Dependencies this week.

One is that friends in those islands report that there is some disappointment on the ground that the House of Commons did not enforce beneficial ownership on them. Ordinary people - real islanders and not the incomers who populate the senior echelons of the finance industry - are fed up with the occupation of their islands by finance. The message was clear. 'Keep going' they have said.

And then an amusing tale. I was working in a coffee shop by the river in Ely an afternoon this week (it keeps my work tempo up to get out of a late afternoon) and fell into conversation with a couple of about my age. It turned out they were from Guernsey, here to visit grandchildren. I revealed my familiarity with the island, both geographically and politically.

What was the first thing they did? They checked me out on the web. As if evidence was needed that people need to know who they are dealing with, this was it.

Then they continued the conversation. They could see no problem with Guernsey being required to have a public register. They use Companies House. They are really in business. They have no problem with disclosure. They clearly could not see what the fuss was: honest business should have no problem with it, they thought.

And that's always been my point. Honest business does have no problem with disclosure. They know it reduces risk, and that's good for everyone.

Secrecy is all about increasing risk for others. And that is precisely why it has to be tackled. It is abusive. And that abuse has to be stopped.

The UK has an accounting regulator that refuses to be accountable

Published by Anonymous (not verified) on Thu, 12/04/2018 - 8:55pm in



The FT has reported this morning that:

The UK accounting regulator has been accused of a “shocking” lack of transparency after it emerged that it has avoided giving full responses to nearly 90 per cent of the Freedom of Information requests it has received since 2013. The Financial Reporting Council has answered just six of the 51 FOI requests it has received since 2013, according to its own disclosure log.

Amongst the questions it has refused to answer are ones on whether any of its staff have been seconded to the “big four” accounting firms and vice versa and its investigation into the role of KPMG in the collapse of the defunct lender HBOS.

My friend and occasional contributor to this blog, Atul Shah, who is professor of accounting at the University of Suffolk, was reported by the FT as saying that the number of FOI requests that are deemed to be outside the scope of the regulator’s remit was “shocking” and was noted as adding:

This shows there is a real problem within the soul of the FRC. It is a public regulator and not a private members club, and it [has] clear duties of transparency, accountability and reliability which it has been avoiding over many years.

A public regulator should have duties to respond to public queries. They have been fobbing them off, not just once or twice but over a long period. That is really shocking. It beggars belief really.

I agree with Atul. We are in the quite absurd position that in this country the regulation of accountancy has been outsourced to those being regulated and (you couldn't make this up) those responsible for delivering accountability are refusing to be accountable for doing so.

It really is time for the UK to have a proper, state-run, accounting regulator that is wholly independent of the profession and accountable to both a minister and the public who depend upon it.

Tax transparency can go backwards, as Belgium is proving

Published by Anonymous (not verified) on Fri, 06/04/2018 - 1:27am in



The OECD has just published its new peer review on the state of automatic information exchange with Belgium. As the OECD says:

In its new peer review report, the Global Forum found Belgium to be Largely Compliant with the international standard on transparency and exchange of information upon request over the review period (1 October 2013 to 30 September 2016). This deterioration compared to the previous peer review report – Belgium had been rated Compliant in 2014 – mainly results from an unfinished implementation of the strengthened international standard on beneficial owners and reversals of case law regarding some features of the access powers of the tax administration. These led to legislative changes, the implementation of which should be monitored to ensure full availability and access to information to be exchanged. That said, Belgium remains a country widely open to exchange of information and that provides timely replies and qualitative information to its many partners.

The report is important for a number of reasons.  First, it shows that the OECD is willing to record a failure to progress.

Second, it shows that promises are not enough: action must follow.

Third, it shows the vital importance of beneficial ownership data, which is now a European Union requirement meaning Belgium should be getting its act together and not going backwards.

Fourth, it shows the importance of getting legislation on this issue right so that courts cannot over-rule intention.

The report is useful for all these reasons. And Belgium needs to take note. It's in amongst some tax havens right now.

Beneficial ownership registers are not enough for tax justice: full accounts on public record are required as well

Published by Anonymous (not verified) on Wed, 04/04/2018 - 6:12pm in

There was an article in the Guardian yesterday under the headline:

Oligarchs hide billions in shell companies. Here's how we stop them

Written by Frederik Obermaier and Bastian Obermayer, who are investigative journalists with the German daily Süddeutsche Zeitung and initiated the 2016 Panama Papers as well as 2017 Paradise Papers revelations, they argue for:

[P]ublic beneficial ownership registries: Databases in which citizens can easily access and explore the owners of companies. Not the nominee director, not the fake shareholder – the real owner. The person at the center of the matryoshka-like corporate structures, or, as experts refer to them: the ultimate beneficial owner of a company.

A database of actual owners would enable companies to check with whom they are actually doing business. It would enable activists, journalists and skeptical citizens to investigate the individuals running dubious companies which earn millions in alleged “consulting contracts”, which are in many cases nothing more than concealed payments of corruption money. It would also give prosecutors the opportunity to follow dark money without having to rely on nerve-racking, time-consuming legal maneuvers with foreign governments.

I agree. But I have to say that I agree only in part. And that is because such a register is a list of data for the idly curious unless - and I cannot stress that unless enough - it leads to accounts on public record as well.

I confess to bemusement that there are very large numbers of tax justice campaigners who are now engaged in calling for beneficial ownership registries and yet knowing who owns a company when you have no idea what it does is of very little use at all. It's not as if beneficial ownership data even tells half the story. It does not even do that.

The reason should be obvious. I could, for example, own a company in the BVI. The beneficial ownership register would reveal that. If the company was dormant the answer would be 'so what?' If, however, it was turning over £50 million and owned properties in a range of jurisdictions the answer would be very different, indeed. A whole host of questions would follow. Without accounts no one would know the difference. That would mean the beneficial ownership register would rapidly prove to be of little meaning - precisely because people would realise that no answers of consequence could be provided with such data.

Don't get me wrong here: I want beneficial ownership registers. But I also want full accounts on public record which means  they must include a summary of the trade and location(s) of the business; details of management; details of all owners of more than 10% of the entity; an income statement, a balance sheet; a cash flow or statement of source and application of funds; notes to explain the accounts; details of related party transactions, in full, plus country-by-country reporting if working on more than one jurisdiction. Then I might know what is really going on. Beneficial ownership registers simply are not enough.

The UK cannot afford Sir James Dyson’s anti-competitive aspirations

Published by Anonymous (not verified) on Thu, 01/03/2018 - 5:14pm in

In an extraordinarily mistimed move the strongly pro-Brexit billionaire Sir James Dyson has made a demand that the UK  join the world's tax havens in abolishing the requirement that private companies (like his) must place their accounts on public record. Little could better indicate the planned tax haven, race to the regulatory bottom, trajectory of those supporting Brexit than such a demand.

As the FT has reported:

James Dyson has slammed UK rules that force privately owned companies to file accounts that can be viewed by the public, describing them as “anti-competitive” and handing an advantage to overseas rivals.

It's true that the US does not require this. Nor do many tax havens. But the EU does, of course. The FT added:

The billionaire inventor, famous for his bagless vacuum cleaners, said he saw “no reason” why private concerns such as his own should have to make the financial disclosures. “It’s not so much the cost of it — although there is a huge cost — it’s the fact that our competitors in foreign countries can see exactly what we’re doing and we have no sight of what they’re doing,” said Sir James.

So let me remind Sir James why we might require this. The list is not hard to understand.

First, it's because of limited liability. No one is asking for the accounts of unlimited companies to be placed on public record, just as I do not ask for private tax returns to be put on public record. So if Sir James wants privacy he has the option of enjoying it. All he has to do is agree to pay all his debts in full, come what may.

But, if he wants protection from his creditors if, for example, his £2 billion electric car scheme fails, which limited liability would most likely provide, then he has to make use of a limited liability company.

That choice is, however, not costless. It imposes a real burden on society. BHS proved that, as have so many others, including no doubt Toys'r'Us and Maplin, only yesterday. And the cost falls on a wide variety of people, including providers of loan capital, suppliers, customers, employees, tax authorities, regulators, host communities in the form of local authorities and civil society at large who bear the externalities of business activity. All these groups need to know the risk they take when dealing with a business that may not pay its debts, and require the information to make sure that their down side is managed as far as possible.

I would argue that stakeholders do not get all the information they need to do this at present. Country-by-country reporting is one obvious case where there is an exception. Sir James clearly thinks they get too much. But this is not the point. The point is to ask why Sir James thinks he should get all the gain and those who pay the price get all the cost? The answer is not clear.

Nor is it clear that, secondly, Sir James is really that clued up when it comes to what is an anti-competitive measure. If he had the slightest understanding of economics  then he would know that competitive markets are only efficient at allocating resources when all those working in them have perfect knowledge of what their competitors are doing.

Now, I know that perfect knowledge is not possible, of course. But we need to approximate to it. Unless we do the claim that markets allocate resources efficiently really cannot be sustained. What we get instead is market abuse, monopoly, and anti-consumer activity as well as practices where large companies (of which Sir James' is one) tend to exploit smaller ones because there is an unlevel playing field, not least with regard to information. This is what anti-competitive means. Being exposed to the risk that the market might not decide to deal with you because they do not like the risk of doing so, which is what Sir James is worried about, is about creating a competitive environment. Maybe, and I can only guess, it is that which he does not like?

Whatever the reason though, what Sir James does is reveal the true reason why those business persons backing Brexit did so. They wanted to race regulation to the bottom for their own gain at cost to others. This is, of course, the classic tax haven model.

Sir James is, of course, welcome to go to such a place. But he will find there are problems. There will be almost no trained employees, and little industrial infrastructure. There will be barriers to trade. And there are remarkably few customers. That's because the places that permit the kind of secrecy to which he thinks the UK should aspire are far removed from the realities of commerce. Theirs is a world of make believe where all that matters are the rights of the wealthy and their companies to hide from the view of those on whose backs their wealth is made.

I'm sorry Sir James, but that's not the type of country that I, or the vast majority of the people in the UK, want to live.

As I have pointed out, you have options to secure your privacy, and you can afford to take them. But we cannot afford to grant them. You may leave if you wish. But we want a strong, competitive and informed economy, which is not what you would appear to aspire to. If that means there has to be a parting of the ways, so be it. But the UK cannot afford your deeply anti-competitive aspirations.

How transparent is government about the evidence behind policy?

Published by Anonymous (not verified) on Thu, 08/02/2018 - 7:50am in



Richard Clubbe discusses the mixed findings of Sense about Science’s spot check on how transparent government departments are about their evidence use.

Last week Sense about Science published a report on how transparent the UK government has been about the evidence behind its policies. Transparency of Evidence: A spot check of government policy proposals July 2016 to July 2017 looks at 94 policy proposals from 12 departments, each scored according to the ease with which readers could tell how evidence played into the decision. The project was supported by funding from the Alliance for Useful Evidence and the Nuffield Foundation, and was conducted in partnership with the Institute for Government.

When the evidence behind policy isn’t shared, citizens are unable to understand the motivations behind policy decisions, decide whether they agree, or contribute to constructive discussion around the issue. On top of this, researchers and analysts find it harder to contribute to the evidence base in a meaningful way, and the government is less able to build on its own past work.

Transparency is a prerequisite for assessing the quality of evidence behind a policy proposal. Our report doesn’t look at the merits of a particular policy or the quality of the evidence considered, but rather whether a reader can identify what evidence was used and how it was assessed.

In 2015, the Institute for Government, Sense about Science and the Alliance for Useful Evidence established an evidence transparency framework, designed to be rapidly applied to any policy proposal, which broadly asks:

‘Can someone outside government see what the government is proposing to do, and why?’

With this question at the core of our assessment, Sense about Science led a citizen-centred review that rewards policy proposals for clarity of reasoning and consideration of the audience. For our spot check, we recruited a group of 27 volunteers to score policy proposals against the transparency framework. No prior knowledge or experience of the policy area was required. Scorers approached the task as if they were a member of the public motivated to learn more about the policy.

For this spot check we defined policies as ‘specific interventions intended to change the status quo’. We assessed the documents publicly available at the point when the policy was first set out substantively. The time frame for our assessment was 13 July 2016, when Theresa May became prime minister, to the end of July 2017. From a (very!) long list of potential policy proposals announced during this period, we randomly selected six to eight policies from each of 12 departments for our spot check. The scoring group then read the policy documents and scored each policy against the transparency framework. Scores were verified by the research team from Sense about Science and the Institute for Government.

The framework breaks down policy assessment into four sections – diagnosis, proposal, implementation and testing and evaluation – with possible scores on a scale from 0 (indicating very little mention of evidence and explanation of its use) to 3 (indicating a consistently well-referenced discussion of the evidence behind the policy, including acknowledgment of uncertainties and gaps in the evidence base).

Reviewers found that departments were capable of attaining a high level of transparency in diverse policymaking situations – showing, importantly, that evidence transparency is achievable. The most consistently high scoring departments were the Department for Transport (DfT); Department for Business, Energy and Industrial Strategy; Department of Health; Department for Environment, Food and Rural Affairs; and Department for Work and Pensions. DfT produced the highest scoring policy, ‘Cutting down on noise from night flights’, which received a 3 in every section of the framework. This was a strong example of how it’s possible to not only share the evidence being discussed, but also explore its limitations in the context of the policy area.

Sense about science image

Overall, the review revealed several areas of improvement since our preliminary assessment in 2016, which highlighted good and bad practice across government. One of the shortcomings revealed in that report was that often departments had compiled research during policy development but failed to clearly share this with the reader, either through poor referencing or complete omission. This year we observed more sharing of the research done by departments. Analysts should welcome this finding – they are seeing more of their good work being published by their departments, a trend that reflects well on their policies.

The least transparent area identified was testing and evaluation. To score well in this section of the framework departments are required to set out what they will measure in order to determine whether an intervention has been successful, and clear plans for the publication and use of those results. Our review revealed very few occasions in which this was discussed in detail. Of the 94 policies assessed, only four scored a 3 for testing and evaluation, while 64 policies scored a 0 or 1. Testing and evaluation is about considering what is working and helping to produce evidence that informs future policy decisions. Without measurable outcomes, it becomes very difficult to build upon past experience to create improved policy solutions in the future.

The transparency framework developed with the Alliance for Useful Evidence and Institute for Government could be used outside reviews such as this one. Over the last year we have used the framework to engage with departments through talks and workshops with senior civil servants, analysts and policy professionals, many of whom are keen to use the framework to ‘mark their own homework’ and improve their department’s scores in time for future reviews. Outside Whitehall, the framework has been shared through facilitated workshops at the away days of nine parliamentary select committees, a meeting of the European Commission’s Regulatory Scrutiny Board and international policy conferences like the What Works Global Summit. It has even been translated into Spanish for use in the Peruvian parliament.

We are currently planning to conduct a further spot check in two years’ time. The time period to cover and scope of departments, agencies and devolved governments to which we might extend the review will be decided over 2018.

Richard Clubbe is research and policy officer at Sense about Science and contributed to the research for this report. 

This project was funded by the Nuffield Foundation, but the views expressed are those of the author and not necessarily those of the Foundation, or the Alliance.

‘Promised’ Charity Money to Be Investigated

Published by Anonymous (not verified) on Tue, 10/03/2015 - 11:36am in