innovation

Error message

Deprecated function: The each() function is deprecated. This message will be suppressed on further calls in _menu_load_objects() (line 579 of /var/www/drupal-7.x/includes/menu.inc).

Playacting government: Victoria’s COVID response

Published by Anonymous (not verified) on Tue, 13/10/2020 - 5:41pm in


Dan Andrews said that his ‘Road Map’ for easing the lockdown is not a doctoral thesis – a proposition that’s hard to argue with. Further propositions will be offered at subsequent press conferences.

Life in the West is increasingly reminding me of the old Soviet joke. “We pretend to work, and they pretend to pay us”.

 

Herewith an email I received this morning.

Nick I am going troppo about the Vic policy response

– still denying aerosol transmission and its implications
– still rationing N95 masks which is contributing to these hospital outbreaks
– still haven’t got fit testing mobilised and its roll out scope is limited to covid interacting HCWs

We are counting the costs of pennies in the PPE/health care precautionary response while blithely managing the results via lock downs at the cost of billions.

This fit testing thing. I just booked to get fit tested in November with a private PPE company – they were available for me to book any day of the month.

How is that possible!? How have they and all their competitors not been effectively nationalised – every fit test machine, every fit test technician, mobilised to deploy to hospitals, GP clinics, aged care homes?

It is a similar story on testing. In Melbourne we are just now trialling the remarkable innovation of “third ring” testing. In Qingdao, the Chinese government’s response to a small outbreak is reportedly to test all nine million residents.

Why – in OCTOBER – has the government not said to every leader of every health function in the state: you have no budget constraints. Buy any mask you need at any price. Test anyone you need at any price. There is no cost you can incur that is not worth it if it means even one day sooner out of lock down.

This response is not just incompetent, it’s insane somehow – deeply dissociative.

And my own retweet from yesterday

Will busy offices return eventually? Of course they will.

Published by Anonymous (not verified) on Mon, 07/09/2020 - 9:22pm in

[message: the “stay at home” firms will see their bored and lonely good young staff jump ship to the hip, drunk, snorting, and cavorting hard-work hard-play offices everyone loves to complain about.]

The Office revives small town Scranton, but it wasn't taped there – SkiftThe estimate from Transport for London is that 72% of workers are still not back at their office this week, which is why the coffee clubs, eateries, bars, and restaurants are still empty and central London feels like a ghost town.

Its the same all over the large cities in the UK: 50% of workers of all industries were still working remotely first week of August and public transport remains shunned like the plague.

Its slightly more “back to normal” in other European countries that ended lock downs sooner than the UK, with around 80% of all workers back to their previous place of work in France, Germany and Italy by August.

Still, nowhere has office life yet returned to the previous levels. More than half of office workers have indicated they would like to work more from home than before and many major employers have indicated they do not expect office life to return to pre-covid levels any time soon. This is fueling widespread speculation around the world of how work is going to change forever and that the nature of inner cities will adapt. You get plenty of visions of workers dialing into virtual offices from their forest homes whilst big cities become terrace-filled pedestrian zones dominated by tourists.

I think its too soon to expect the old office life to be gone forever. Its demise has been prophesised before and quite a few tech companies with the know-how have tried to do away with the office in previous years and failed. Whilst it is true that there has never been such a massive shock to office life as we’ve seen the last 6 months, and so the “new normal” might feel to many like it is sustainable, one can only understand whether one should really expect the “old office life” to be gone forever if we understand what the economic and social reasons were behind the old office life. Why did companies fork out hundreds of billions of dollars per year for decades to afford very expensive places in the middle of cities, forcing their employees into highly wasteful commutes? What made that expense and effort worthwhile in the first place?

There are strong economic forces that made regular office life an equilibrium, with firms that did not comply losing out. Firms that have a regular office life have three things going for them that “from home offices” lack: they offer the staff a venue where they have to socialise; they lead to a rat-race culture that encourages over-working; and they offer a place where clients/suppliers come to visit, come to see up close whom they are dealing with, come to be impressed, and come to be entertained.

So firms with a regular office will pinch the more ambitious, more fun-loving, more socialising workers and clients of firms without a regular office. These disadvantages do not hold if all offices are forced to work from home, which is why they were not so relevant the last 6 months and a different culture seemed sustainable, but they will re-emerge as soon as some offices go back to regular functioning. The “stay at home” firms will see their bored and lonely good young staff jump ship to the hip, drunk, snorting, and cavorting hard-work hard-play offices everyone loves to complain about. Ditto for their richest and most dynamic clients. To put it as simply as possible: you don’t make real friends on zoom. 

One of the central mistakes many commentators thus have about work and office life is that humans are solitary animals only offering their labour time for money which they use to buy stuff online. Its the kind of thing only a half-trained economist can believe. It is just not true. Humans are group animals, first and foremost. They work because they are in competition and in partnership with their colleagues and people around them. They get aroused and motivated via close contact with other humans. Stop offering them an office in which they are a group and you will eventually find they lose all interest.

A related economic force that will lead us back to the office is that offices impress. In old economic parlance, offices burn money and that serves as proof that a firm has something to lose if it doesn’t keep to its contracts. Offices gives contracts and promises credibility because one can see there is a physical place and  social position firms stand to lose if they dont stick to promises and get taken to court. In more normal speak, a big shiny busy office shows you have skin in the game and an adoring tribe to boot.

A nice recent discussion of the basic psychology and sociology of office life is by Randall Collins. Let me just copy the best bit of his quite long article, which discusses failed previous experiments in abandoning offices and also has as the bottom line that online doesn’t arouse people and thus is doomed to death-by-indifference:

Working Remotely

There is disagreement whether working remotely is effective. Some people prefer working from home. What they like about it are: no commuting; reduced meetings which they feel are a waste of time; and fewer distractions in the workplace. Some dislike working at home; what they dislike are more distractions in the household; less team cohesion; and technical and communication difficulties. (Wall Street Journal, May 28, 2020: based on a survey of hiring managers) Similar points were made by the head of a state judicial unit, who emphasized that much additional time by management personnel was now spent on meetings, and attempts to keep up morale by remote contact; meetings were often frustrating because considerable time was wasted trying to get the communications technology working for all participants. (repeated interviews during March-June 2020)  She sometimes went  to her office in order to use secure communications, and found it refreshing whenever encountering a colleague in person. Efforts to re-open court business, with social distancing and masking precautions, were welcomed by part of the staff and opposed by others. The characteristics of one group or the other are unknown; a hypothesis is that the those more committed to their career and professional identity want to return to their customary work setting; those for whom work is more of a routine prefer to stay home.

Hollywood film professionals said they liked spending less time on planes flying around the country; and less high-level meetings which they considered more habitual than necessary. [Los Angeles Times May 3, 2020] One producer said: “I don’t think video conferencing is a substitute for being in a room with someone, but it is better than just talking on the phone. There are so many ways you communicate with your expression… when it’s delayed and small, you just lose all that. My feeling is it’s 50% as good as an in-person meeting.” [p.E6] In the actual work of making movies, most emphasized that it is a collective process, and some insisted that spontaneous adjustments on-set were the key site for creativity. They also reiterated the point that live audiences are the only way to reliably tell whether a film is coming across, and larger audiences amplify both comedy and drama (i.e. via emotional contagion).

Some businesses have tried to compensate by having “virtual water-cooler” sessions several times a week, where any employee can log in and chat. It is unclear what proportion took part,  how enthusiastically, or with what pattern over time. Some managers reported that company-wide “town-hall meetings” to reassure employees lost interest over time [Wall Street Journal, June 6, 2020]. DiMaggio et al. (2019) however, found that on-line “brainstorming events” for employees in a huge international company were consonant with some patterns of interaction rituals; this research was carried out in 2003-4, long before the epidemic. The degree of involvement and solidarity in town-hall meetings is a matter of scale; the court administrator reported that feedback about morale was positive after on-line sessions involving group of around 10; but in larger groups it was hard to get a Q&A discussion going. This is similar to what any speaker can observe in ordinary lecture presentations and panel discussions; even with physical presence, most people are reluctant to “break the ice” after the speakers have been the sole center of attention; but once someone (usually a high-status person in the audience) sizes up the situation and says something, it turns out that many others find they also have comments to make. This is a process of micro-interactional attention, which is especially difficult to handle on remote media.

Many managers said that innovativeness was lost without serendipitous, unscheduled encounters among individuals. [Wall Street Journal, June 6, 2020] In a PricewaterhouseCoopers survey, half of employers reported a dip in productivity with on-line work. Longer trends, going back before the coronavirus epidemic, indicate that the promise of on-line work was not highly successful. During 2005-15, the era of the high-speed Internet, the percentage of persons in the US regularly working from home increased slowly;  those working from home at least half-time reached a pre-epidemic peak of only 4%. [www.npr.org/sections/money/2020/04/28/846671375/why-remote-work-sucks]  During this period several big corporations, initially enthusiastic, tried to shift to primarily on-line work but abandoned it after concluding it was less effective. In the market-dominating I-T companies, the trend instead was to provide more break rooms, food, play and gym services to keep their workers happy on site. This was abruptly reversed in the coronavirus period.

Zoom fatigue

Popular video-conferencing tools such as Zoom attempt to reproduce F2F interaction by showing an array of participants’ faces on the screen, along with one’s own face for feedback in positioning the camera. Reports on how well it works in generating IR-type rhythm and solidarity are mixed. CEOs of high-tech companies tend to claim that it works well. Among rank-and-file participants, however, complaints are widespread and it even acquired a slang term, ‘Zoom fatigue.’ [Wall Street Journal, May 28 and June 17, 2020]  Achieving synchrony with others is hard to do with a screen full of faces, delayed real-time feedback, and lack of full body language. Since there is a limit to how many individual faces can be shown, in larger meetings some persons are seen only occasionally, and leaders looking for responses often find they get none. Some of the ingredients of IR (not necessarily under that name) are now being recognized by communications specialists; these include fine-grained synchrony and eye movements. In ordinary F2F conversation, persons do not stare continuously at others’ eyes, but look and look away (Tom Scheff made this point to me in a personal communication during the 1980s; for detailed transcripts of multi-modal interaction see Scheff and Retzinger 1991). Thus seeing a row of faces staring directly at you is artificial or even disconcerting. Some readers responded with advice: cut off the video to reduce zoom fatigue, go audio-only. Some found hidden benefits in zoom conferencing: once the round of social greetings is over, turn off the video and your mic and do your own work while the boss goes through their agenda.

Continuously seeing one’s own face on the screen is another source of strain. Of course, as Goffman pointed out, everyone is concerned with the presentation of their self, in terms of status as well as appropriateness for the situation. But one does not have one’s image constantly in a mirror; and when interaction starts to flow, one loses self-consciousness and throws oneself into the activity, focusing more on others’ reactions than on oneself. Those who cannot do this find social interaction embarassing and painful.  But enforced viewing of one’s own image feels unnatural.

Prolonged video conferencing as a whole seems to have about the same effects as telephone conference calls. In my experience on the national board of a professional association, our mid-year meeting was canceled by a snowstorm, and a 2-day conference call was substituted. The next time I saw the board in person, I polled everyone as to whether they liked the conference call: 18 of 20 did not. Lack of shared emotion was apparent during the event; for example, when it was announced that we had received a large grant, there was no response. No wonder: applause and cheers are coordinated by looking at others, and it is embarrassing to be the only person applauding. [Clayman 1993] Work gets done remotely, after a fashion; it just lacks moments of shared enthusiasm.

Orwell that ends well: Can evaluation save us from ourselves?

Published by Anonymous (not verified) on Tue, 01/09/2020 - 3:35pm in


I really love this design by Casey Finley, who was kind enough to allow me to publish it here. He has a very distinctive style which is really coming into its own as he works on it. For instance, see here and here.

When I first saw the Productivity Commission’s Draft Indigenous Evaluation Strategy, my heart sank. I’d had had several quite extensive meetings with Romlie Mokak, the Indigenous Commissioner at the PC who struck me as a person of great intelligence, straightforwardness and practical commonsense. He and his team at the PC had also seemed interested in my own thoughts about evaluation and the contradictions involved in the way it is being embraced as a panacea, as something that can save the system from itself. But though that interest turned up in the report, it did so as reportage – including a box on my Evaluator General model – rather than real engagement on the ideas.

Often at times like this, it’s best to just keep one’s mouth shut and not make enemies but, without much hope of being understood within officialdom, I thought I really should bear witness to the travesty that it seemed to me was unfolding. And I’m glad I did. As I wrote, lots of things I’d been pondering for a good while started falling into place. For instance, you’ll see my reference to Lord Acton’s faultline introduced in Section 3 as integral to the analysis rather than a throw-away line. But there are plenty of other examples.

These things enabled me to clarify what I think the challenge is. At bottom, it’s a challenge of bona fides. Are we prepared to face up to uncomfortable truths? I’ve always thought in my own life it’s the only route to self-betterment. And it’s all I have to offer officialdom. It seems to me what Orwell was about.

It turned into a long piece so I’m grateful to Peter Browne who runs the excellent Inside Story for publishing the piece. Like the themeatisation of so many things, publishers seem increasingly preoccupied with the form of a piece, specifying length, subject areas and so on. Of course, targeting is the essence of publishing, but so too is quality, so I’ve always appreciated publishers who vary their guidelines somewhat to accommodate what matters most to authors. Anyway, Peter did that for me here, for which I’m grateful.

I’ve had a very good reaction from those who’ve read the piece – including Romlie Mokak who was interested in its arguments and its suggested changes to the draft (though of course made no promises) – so I look forward to getting feedback from the Tropposphere!

The essay begins below the fold.

Humility is not a peculiar habit of self-effacement, rather like having an inaudible voice. It is selfless respect for reality and one of the most difficult and central of all virtues… Humility is a rare virtue and an unfashionable one… Only rarely does one meet somebody in whom it positively shines, in whom one apprehends with amazement the absence of the anxious avaricious tentacles of the self.
— Iris Murdoch

The divergence between the facts established by the intelligence services — sometimes by the decision makers themselves (as notably in the case of McNamara) and often available to the informed public — and the premises, theories, and hypotheses according to which decisions were finally made is total. And the extent of our failures and disasters throughout these years can be grasped only if one has the totality of this divergence firmly in mind.

— Hannah Arendt

I have a theory that the truth is never told during the nine-to-five hours.
— Hunter S. Thompson

1. Introduction

From 1788 till the 1960s, Europeans established themselves on Indigenous land in a brutal regime, first of dispossession and then of disregard. Yet some among them had strikingly good intentions. A year before Wilberforce took on the cause, nearly eighty years before the Emancipation Proclamation, Arthur Phillip accepted his commission insisting that slavery had no part in the new colony. Phillip sought to treat the Indigenous people fairly, at least according to his own lights. But mutual incomprehension reigned and those with murkier intentions soon prevailed.

Today, good intentions abound, though racism often lives on in unacknowledged assumptions. Governments outlay vast sums, whether adequate or not, on specific Indigenous programs and in general expenditure on Indigenous health, education and social security. Widely supported grand gestures are announced every few years. You might think “Closing the Gap” was Kevin Rudd’s idea but it rebooted (or is that rebranded?) a Hawke government initiative of twelve years earlier. But the results are meagre.

Now comes a new cycle of activity, this one focused on whether formal evaluation processes might allow us to identify and scale up those Indigenous programs that actually “work.” Most recently, the Productivity Commission has been hard at work on a national Indigenous Evaluation Strategy, which was the immediate trigger for this essay and to which I’ll return. Will this cycle of activity produce better results than earlier efforts? I’ll explain below why I have my doubts.

To first clarify where I’m coming from, it is not from deep knowledge of Indigenous policy. My focus here is rather on a prior question: how our formal institutions of government — and most particularly our bureaucracies — might need to change to succeed where previously they have so consistently failed. To make that question concrete I draw on my experience in other intractable areas of social policy that bear family resemblances to Indigenous policy.

Programs to protect children from abuse and neglect, particularly in disadvantaged families and communities, follow the same endlessly repeated cycle of failure followed by grand plans for reform that then run into the sand before the cycle begins again. This essay focuses on how little the system really appreciates the distance it would need to travel to really be effective, in terms of either its own values and objectives, or those of the disadvantaged communities — including Indigenous communities — it claims to be serving.

2. The “what” and the “how,” the saying and the doing

So here’s my very simple description of the problem: despite endless pronouncements of what we must do, there’s minimal comprehension of how to do it.

This is an endemic problem. Dependable know-how itself — whether it’s improving outcomes in an Indigenous community or representing government in the High Court — is not directly legible to government systems. Anyone can claim to have that know-how but a bureaucracy needs something more dependable than that. As a consequence, it will interact with know-how as a certified, decontextualised “what.” That “what” could be a credential, the meeting of a key performance indicator, or a particular bureaucrat’s informal reputation for being a “good operator” or a “safe pair of hands.” In improving Indigenous lives, however, know-how won’t align with any such things, not least because so much of it resides among Indigenous people and communities themselves. We need to access their knowledge and their agency to improve their own lives in ways that matter to them.

The cliché used to convey this idea is “putting people at the centre” or “putting people first.” However well-intentioned such slogans are, more often than not they operate as a kind of doublethink — as if adopting the slogan were to put its intent into practice.

The philosopher Martha Nussbaum offers a story that illustrates this difference between saying and doing. She describes how a development program encounters a woman in a traditional rural community who is uninterested in education for herself or her children. Nussbaum is showing how our (reductive) framing of the other’s perspective can cut us off from the wisdom of the other’s lifeworld. “Clearly,” she writes, “a one-shot logical argument” wouldn’t be enough to engage the woman:

[S]uch a procedure would only reinforce her conviction that education has nothing to do with her. Nor would the exchange get very far if the development workers sat down with her… asking… calm and intellectual questions about what she thinks and says. But suppose, instead, they spent a long time with her, sharing her way of life and entering into it. Suppose, during this time, they vividly set before her stories of ways in which the lives of women in other parts of the world have been transformed by education of various types — all the while eliciting, from careful listening over a long period of time, in an atmosphere of trust that they would need to work hard to develop, a rich sense of what she has experienced, whom she takes herself to be, what at a deeper level she believes about her own capacities and their actualization. If they did all this, and did it with the requisite sensitivity, imagination, responsiveness, and open-mindedness, they might over time discover that she does indeed experience some frustration and anger in connection with her limited role; and she might be able to recognise and to articulate wishes and aspirations for herself that she could not have articulated to Aristotle in the classroom. In short, through narrative, memory, and friendly conversation, a more complicated view of the good might begin to emerge.

Nussbaum’s scenario is based on actual fieldwork in Bangladesh, and couldn’t be cost-effective if it involved professionals engaging rural women en masse. But, as I discovered when I was chairing the Australian Centre for Social Innovation, the spirit of this translational endeavour is already captured in existing and cost-effective programs in Australia.

The centre’s Family by Family program takes families who feel they’re close to crisis. A trained coach then takes each family through a structured program of mentoring by another local family that has come through similar stresses. The family seeking help chooses the family that mentors them and sets the objectives they want to work on.

The program was co-designed with families over many months, but the simplicity and obviousness of the end result gave those involved in it and many lookers-on numerous “aha” moments. Family by Family embodies the rare art of professionals vacating centrestage in a therapeutic intervention to create space for those who must do the real work. Professional knowledge, which grows with the program, is always there — but as midwife, not obstetrician.

Talking to some of these families, I was struck by their visceral engagement with the program and their mentors. To take just one example — of which there were many — one mother in the program had received twenty-seven statutory “notifications” documenting outsiders’ suspicions that she was neglecting her kids. The relevant department was heading to court to take her four kids into guardianship. When her mentor family took her family camping, she learnt many things from them — not least to hug her kids. The department stopped proceedings against her.

The thirty-week program cost around $13,000. If that sounds expensive, it’s a fraction of what social workers would have cost, and much more effective. Moving all four kids into care would have cost around $224,000 per year. So, if Family by Family steered just this family from the shoals of state intervention it probably paid for its development and first couple of years of operation.

Before I saw Family by Family in action, I’d have described my outlook as that of a tragic liberal — committed to fairly generous spending on social disadvantage, but with very modest expectations of how much it could turn things around. After seeing Family by Family, the penny dropped. Ingrained patterns and social reinforcement are immensely powerful, almost immovable forces. But people’s desire to work towards better lives for themselves, their families and their communities is similarly elemental if they can somehow unlock their own agency and that of those around them.

3. Lord Acton’s fault line

After acknowledging the vast gulf between identifying the “what” and mastering the “how,” between the saying and the doing, we should then do something I’m doing for the first time in this essay. For decades I’ve referred to it in asides, but it needs to be brought centrestage so we can look it in the eye. It’s significant that it’s a joke, just as it’s significant that so many of the best insights into bureaucracy are provided by comedies like Yes MinisterThe Office and Utopia.

More than a century ago Lord Acton quipped that rowing was the perfect preparation for public life. Why? Because you face in one direction while moving in the other. One crucial reason that we’ve made so little progress is that in a thousand ways, large and small, the actors in the system face in one direction — with their mission statements, corporate values, strategic plans, evaluation strategies and all the rest of it — while moving in the other.

Of course, they’d prefer to do a good job — most people would. But when push comes to shove, their animating imperative isn’t to keep progress going in the field. It’s to keep up appearances. Seen this way, all those grand announcements we keep making are part of the problem. They’re really directed at our own anxieties. They alleviate and distract us from facing our disappointment — our discomfort — that the world remains so resiliently impervious to our good intentions.

Lord Acton’s fault line appears between the two feet on which we stand — between what we say and what we do. That’s why the words we use matter so much, and why we should take George Orwell’s advice to choose the simplest and clearest words we can. As he put it:

If you simplify your English, you are freed from the worst follies of orthodoxy. You cannot speak any of the necessary dialects, and when you make a stupid remark its stupidity will be obvious, even to yourself. Political language — and with variations, this is true of all political parties [and here we can include officialese]… is designed to make lies sound truthful… and to give an appearance of solidity to pure wind.

Since those in the system are the ones with the power, all we have to appeal to is their own self-respect — their own desire to feel better about themselves. When they say they want to change, the real question is how much. The system has said that it wants greater Indigenous agency in its programs for ages. But as I’ll illustrate, our programs are so dominated by that same system’s routines and perspectives that Indigenous agency barely gets a look-in. Instead it gets reduced to things that are legible to the system — such as Indigenous ethics codes and certified cultural sensitivity. These things may have some benefits. They may also have costs, which I’ll discuss. But they are mostly the system saying rather than doing.

This takes us to the nub of the problem. It is only humility, or some institutionalisation of it, that can create that space within which Indigenous agency might be nurtured and grow. But “humility” itself is now turning up as a cliché in all those “how to” guides (it appears just before “nuance” and after “authenticity” — yes, authenticity really was a corporate value of PwC for a while there). So I’ve tried to revivify it with Iris Murdoch’s magnificent words above. For the non-Indigenous among us who fancy we care, we must find ways to untangle ourselves and our institutions from the “the anxious avaricious tentacles of the self.”

4. Enter evaluation

Like a patient resisting therapy, the system constantly initiates new beginnings. But Lord Acton is never far away. At the political level leaders talk of evidence-based policy, but then shunt it aside when convenient. In fact, substantial performance evaluation was built into the structure of the Aboriginal and Torres Strait Islander Commission but sidelined after ATSIC was dismantled by John Howard’s government. The failure of the Northern Territory Intervention to take an evidence-based approach is legendary, worked up as it was over a few days in Canberra in the run-up to an election and yet largely maintained by the incoming government.

More recently, while stressing his own commitment to following the evidence, newly elected prime minister Malcolm Turnbull expanded income-management schemes without mentioning that the independent evaluations were highly equivocal. However well the idea played in non-Indigenous Australia, the evaluations suggested that compulsory income management has clear, positive impacts in very few cases and gives rise to “considerable feelings of disempowerment and unfairness.” As one might expect, voluntary income management is more successful.

Now, it is one thing for senior officials not to speak publicly of their political masters’ hypocrisy. But their complicity goes deeper. In 2009, a finance department review of Indigenous expenditure stressed the need for “a more rigorous approach to program evaluation at a whole of government level.” In 2016, the nation’s most senior public servant, the secretary of the Department of the Prime Minister and Cabinet, Martin Parkinson, echoed those sentiments. In response to such concerns, $40 million over four years was allocated for evaluation. Parkinson’s department was responsible for Indigenous affairs, but the Audit Office reported three years later that its performance was desultory.

As ANU researcher Michael Dillon has suggested, even the Audit Office’s report was “extraordinarily hedged and timid, and failed to make a substantive assessment of the actual independence of the evaluations undertaken” by the department:

Of thirty-five evaluations on the department’s 2018–19 workplan, fifteen had not commenced. Of the remaining twenty, eight had been published and twelve withheld from publication… In at least four cases (involving very significant and sensitive program evaluations) the department was waiting to brief the minister or awaiting his noting of a brief. In plain language, the minister was preventing timely publication of the evaluations.

Further, Dillon observed, Parkinson’s response to the audit “fails to acknowledge or address in any way the negative content of the audit.” Is it likely that the system will engineer something better if it can’t acknowledge its own failure to do as it says?

Which brings us back to the Productivity Commission’s Indigenous Evaluation Strategy, a draft of which was released in June. The PC has always attempted to pitch its proposals to government within the “Overton window” — that range of options that will be taken seriously by powerful people. Given that constraint, as I’ll explain, I respect its compromises on policy. But the point of the PC’s independence is that, however much it compromises on the policy, it spares no one, least of all itself, the truth. What the great scientist Richard Feynman wrote about science is also true of social science. For me, it’s a holy grail of social policy and aligns nicely with Orwell’s advice: “The first principle is that you must not fool yourself, and you are the easiest person to fool.”

5. Putting Indigenous people at the centre: the words

There’s a kind of ambiguity at the very heart of the PC’s draft strategy that’s increasingly common. It’s Orwellian in the bad sense. I guess the genre was introduced into polite society by the “vision statement.” Here one states an aspiration as a fact. You know the kind of thing: “PHP Residual Solutions is the world’s foremost residual solutions provider.” At least in its awkward baldness, it’s not misleading. We all know that global domination is an aspiration, not a fact.

But this fusion of fact and fancy appears as the fundamental building block of the PC’s draft strategy: “The Strategy puts Aboriginal and Torres Strait Islander people at its centre, and recognises that governments need to draw on the perspectives, priorities and knowledges of Aboriginal and Torres Strait Islander people if outcomes are to improve.”

One of the ways to ensure we remain fixed to the spot with Lord Acton’s fault line yawning beneath us is to encourage the idea that saying something is doing it. Does the PC know how to put Indigenous people at the centre of its strategy? Can it point us to better and worse examples of doing so? Can it highlight cautionary tales where grand claims have been made that are belied by the facts on the ground? These are some of the questions — pointed, uncomfortable questions — that we need to answer if we’re ever to step over Lord Acton’s fault line and enter the promised land of “how.”

At the level of programs, rather than evaluation, there are at least two perilous steps in the expedition to get from saying to doing — from signing the cheques to putting the resources of government properly at the disposal of Indigenous people and their communities:

  1. We need to learn how to put Indigenous people and communities at the centre of these programs — or, to put it differently, how to realise their agency within them.
  2. Then we need emerging successes to spread. That requires validated new knowledge of what’s working in the field — always fragile in large organisations to say nothing of systems of organisations — to trump the institutional imperatives that so often frustrate the spread of successful practice.

To me, these are the great priorities for the Indigenous-specific programs I have focused on in this essay, though analogous priorities would apply when considering the impact of general welfare programs on Indigenous people and communities. And any evaluative strategy would emerge from an appreciation of how evaluation might contribute to their wellbeing. As progress was made it would shed light on how further priorities might be set.

But the draft strategy makes clear that this is not the kind of priority-setting the PC has in mind. Its initial priorities reproduce those of COAG’s Closing the Gap report, and their foremost characteristic is their legibility to the system. They’re even arranged around the system’s existing organisational structure, which includes families, children and youth, health, education, economic development, housing, justice, land and waters. Makes you wonder what isn’t a priority! And all of them identify a “what” rather than a “how.”

6. Putting Indigenous people at the centre: the actions

How will we get Indigenous people and perspectives into the centre of evaluation? In their submission to the PC, researchers from Inala Wangarra and the University of Queensland argue that:

“Accountability” has become a lopsided concept, whereby the focus is overwhelmingly on service providers being accountable to government, and where there is no concomitant focus on the accountability of government to the most important stakeholders: Aboriginal and Torres Strait Islander peoples.

So might placing Indigenous people at the centre of an evaluation strategy involve making service providers and government policies accountable to Indigenous people? This possibility doesn’t seem to have made it into the PC’s strategy, even as a “what.” And even if it had, I’d argue that what the PC has endorsed is likely to be implemented in a way that actively obstructs getting to the “how.” The PC talks about the importance of “whole-of-government” approaches to evaluation. That sounds innocuous enough — commonsensical even. But why does it have me thinking of “whole-of-church” approaches to the solar system at the time of Galileo?

The only way I can imagine a whole-of-government agenda not doing more harm than good is if it were to imagine itself as being at the service of solving the concrete and urgent problems in the field — by identifying good practice in the field, for example, and coordinating the system to expand its influence.

Despite senior officials’ and politicians’ protestations that they aspire to encourage innovation in the field and spread and scale “what works,” progress has been conspicuously lacking. Peter Shergold saw this as a major problem as he rose through the ranks of the public service, but after over a decade at its commanding heights conceded there’d been little change. As he put it in 2005:

If there were a single cultural predilection in the Australian Public Service that I could change, it would be the unspoken belief of many that contributing to the development of government policy is a higher-order function — more prestigious, more influential, more exciting — than delivering results. Perhaps it is because I have spent so much of my career in line agencies, learning to deliver Indigenous, employment, small business, and education programs that I react so strongly against this tendency.

Eight years later he confessed that little more progress had been made:

Too much innovation remains at the margin of public administration. Opportunities are only half‐seized; new modes of service delivery begin and end their working lives as “demonstration projects” or “pilots,” and creative solutions become progressively undermined by risk aversion and a plethora of bureaucratic guidelines.

In its preoccupation with grander narratives than identifying what works and spreading it, the PC sets its evaluation process up to be driven by the system rather than its intended beneficiaries, however much it protests that they’re “at the centre.” In a familiar move, the PC suggests that its strategy is driven by four principles, each identified by a pleasing adjective with them all arranged in a pleasing diagram. According to this diagram, evaluation should be “Credible, Ethical, Transparent and Useful.” But these words are so general, so capaciously flaccid, that they constrain no one, like a scientific hypothesis that couldn’t possibly be falsified. And so, rather than constraining (and so guiding) practice, those words will come to mean whatever people want them to mean, often in retrospect to justify whatever practice is chosen.

Note two further aspects of the high-level pronouncements echoed by the Productivity Commission. First, the PC speaks of evaluation as if its function is to bolster the accountability of those in the field to their senior managers, with evaluation’s function being to objectively certify the extent to which the program meets the system’s stated objectives. Second, it shows little awareness of how broad and permissive this relatively new discipline of evaluation is. In reaching for some actionable means of validating that it is embracing a thing called “evidence-based policy,” evaluation is taken to be something far more settled and definitive than it is — as if getting something evaluated were like getting an auditor to check financial accounts or an engineer to check the structural integrity of a bridge.

As Michael Dillon has observed, the assumption that there are or should be simple linear relationships between objectives and performance is “problematic in cross-cultural contexts and certainly not necessarily the case in the… Indigenous domain.” In that regard, the system — and the PC — seems oblivious even to the existence of “goal-free evaluation.” There, the evaluator investigates the impacts of the program without referring to — or ideally even knowing — a program’s stated goals.

In an increasingly managerial world oriented to the needs of organisations and their senior managers, this unconstrained focus deploys the evaluator’s skills in an open-minded way that can more fully reflect the interests and aspirations of other actors in the system — most particularly, intended beneficiaries of the program and the families and communities of which they are a part. Goal-free evaluation puts the evaluator in the best possible position to notice and document all consequences, both good and bad. It can also improve program hygiene just as double blindness adds to the hygiene of a randomised controlled trial.

7. The anatomy of Lord Acton’s work

Then there’s the question of exactly how evaluation will identify what is and is not working, and how these findings will improve policy and practice. This raises several challenges at the heart of the PC’s draft strategy. First, evaluation should be independent so that it is candid. Second, it should be published, in order to help develop a “knowledge commons” around “what works” (and what doesn’t) and to strengthen incentives for policy, programs and practice to follow the evidence. Yet past behaviour shows that the system responds to such constraints by saying one thing and doing another. So why would it be any different here?

Indeed, the woods are full of regimes in which higher-order objectives are foisted on policymakers to do the Lord’s work (Lord Acton’s work that is). These systems allow those at the top to say one thing as they face towards an objective in general, while they do another thing that quietly prevents it happening in particular. And thus ensues a prosaic variant of something Oscar Wilde told us about life:

Yet each man kills the thing he loves…
The coward does it with a kiss,
The brave man with a sword!

Freedom of information regimes sit atop Lord Acton’s fault line. And the discomfort this induces is all too often relieved with strategic cowardice. Having been lowered from on high, freedom of information faces boldly towards transparency. At least in general and at least when it comes to the saying. When it comes to the particular, to what is actually done, officials travel in the other direction. Transgressions go off the record — into corridors, personal phones and email accounts — or are reclassified “cabinet in confidence” or some such. And that’s just the tip of the iceberg as far as actions that are routinely taken to delay and obfuscate transparency under FOI.

If FOI solves its problems the coward’s way, regulation reviews use the sword. Today, new regulation can’t be introduced without a “regulatory impact analysis” duly demonstrating that its benefits exceed its costs. Australia introduced it in 1986, and it seemed like such a good idea that it was replicated around the world — but invariably with the same (desultory) result. Here’s the British Chambers of Commerce back in 2007:

Both Conservative and Labour administrations approach deregulation with apparent enthusiasm, learn little or nothing from previous efforts and have little if anything to show from each initiative.

Sound familiar? Regulation review is another take on the Lord Acton quickstep. Those at the top introduce a compliance regime, but those administering it are trying to get things done for their ministers. So they obey the letter but not the spirit of the regime, and it degrades into empty box-ticking.

8. Getting past Lord Acton’s fault line

To recap: as attractive as they sound, independence and transparency cannot be imposed without setting off powerful and perverse incentives. Any attempt to deal with these dilemmas must look them in the eye. I foregrounded them in 2016 with my own proposal for an evaluation architecture. I called it the evaluator-general to stress the importance of independence and transparency, and also to structurally separate the delivery of services from the means by which we validate their fitness for purpose.

The organisation of the public sector already honours this principle of structural separation — between doing and validating the effects of what we’re doing. Thus, the Audit Office and the Bureau of Statistics are independent information and integrity agencies whose work helps inform us of the success or otherwise of other “doing” agencies directed by ministers — such as the health department and Treasury. At the same time, we expect all these agencies to collaborate — sometimes quite closely.

My proposal for an evaluator-general provides the institutional scaffolding within which the same close collaboration amid structural separation between doing and knowing can be brought right down to operations in the field. That way independence and buy-in can grow quietly from the bottom up within organisations rather than being heroically imposed from the top in a grand gesture that experience suggests will fail and fail again.

My aim was to nurture the self-accountability of those out in the field — Feynman’s imperative that one mustn’t fool oneself — and to build system accountability on that foundation. That’s how Toyota revolutionised manufacturing productivity in a way that’s now imitated the world over. It found a way to build from “how.” It did so by placing the workers on the line, the suppliers and the customers at the centre.

Are my ideas viable or just naive? We’ll only know when we give something like them a good try. We’d need no more than a dozen or so teams to try them. In the PC’s near 400-page background paper there’s some reporting on these problems of independence and transparency, but not in the context of any critical vision or clear explanation of how they can be overcome.

9. Independence-for-hire and the he-who-pays-the-piper problem

The PC’s incuriosity extends to its ignoring the incentive issues arising from how evaluation is commissioned and conducted. As I’ve argued, allowing firms in our private sector to appoint their own auditor profoundly compromises auditors’ independence. By contrast, the auditing of government finances is overseen by an independent auditor-general. Still, while it’s far from optimal, we’ve made the independence-for-hire of private sector auditors work tolerably by specifying highly prescriptive auditing standards. With evaluation, things are very different, there being any number of ways to conduct evaluations to serve numerous tastes and purposes. So evaluators’ independence-for-hire provides wide scope for doing Lord Acton’s work.

As I’ve argued elsewhere, independence-for-hire sits at the heart of a “now-you-see-it-now-you-don’t” catch 22 that prevents promising developments in the field even becoming visible to the system, let alone having their expansion supported by it.

It goes like this. Responding to all the stirring visions of government “scaling what works,” non-government organisations seek government funding to expand their most promising programs. At this point, departments of finance oppose such funding, as well they might, until the programs are independently evaluated. They don’t take responsibility by commissioning the evaluation themselves or even specifying what kind of evaluation they require. Thus, when the NGO returns, a few hundred thousand dollars poorer, with a Deloitte, PwC or Lateral Economics report in hand (we’re cheaper!), it’s ignored again because independence-for-hire isn’t independence. And so the process of “scaling what works” is stopped dead in its tracks.

Though it understands the value of independence in evaluation, the PC completely flubs the “independence-for-hire” problem, simply associating contracted-out evaluation with independence. And it won’t bite the bullet and recommend true independence because it knows it lies beyond the confines of the Overton Window. It would be rejected out of hand. But to keep the idea of independence in play, it proposes Lord Acton’s independence — an independent Office of Indigenous Policy Evaluation that will “oversee” evaluation, though the actual evaluation will continue to be conducted within the very agencies whose performance is being evaluated.

No doubt the PC hopes that this might introduce some independence into the process. But progress, if any, will be agonisingly slow. Allowing agencies to do their own regulatory impact analysis has kept the tiger of regulation review pristinely toothless for thirty-five years now in every country where it’s been introduced. The old Office of Regulation Review operated within the PC itself, but the greater notional independence it had there made not the slightest bit of difference. The requisite boxes were ticked and regulations — both the good and the bad — went on piling up as normal.

10. Stated intentions and animating imperatives

It’s Lord Acton pretty much all the way down. The PC’s draft strategy stresses the need for evaluations to:

• be done ethically
• involve and engage Indigenous people
• be respectful of and in sympathy with Indigenous cultures and knowledges.

Now, each of these is a commendable objective as a “what.” As I keep saying, the hard part is working out the “how.” And tackling each of these matters productively requires great insight. Further (and astonishingly), the importance of each of these requirements is relatively new to the system even as a “what.” Should we really put that same system in charge of learning the “how”? What will happen is already a foregone conclusion — the PC more or less recommends it. Rather than proceed humbly, foregrounding its ignorance, the system will go through its well-worn routines. Codes of practice will be developed. I assume there’ll be lots of consultation.

But these codes won’t deliver what is written on the packet any more than the mission statement “putting families at the centre” would have delivered Family by Family. However well-intentioned, these codes’ animating intent — what will matter when push comes to shove and someone might end up on the telly or in a headline — will be the institutional safety of those developing and administering the codes.

This is what happens when the system’s commanding heights are put in charge of delivering something that is difficult and context-sensitive but not highly valued in our political culture. Those defending Indigenous interests would be well advised to look on the burgeoning performance regimes in numerous sectors — particularly education and university research — where more and more practitioner time is taken up complying with relentlessly expanding requirements from bureaucracies that have neither the slightest knowledge of nor regard for what’s going on out in the field. As the accountability theatre ramps up, administrative numbers and salaries swell at the centre and performance declines. As Britain’s Institute for Government documented in a different context, inquiries and restructurings abound and new ten-year plans are announced once every three or four years.

I recall when, in response to another paedophilia scandal, South Australia strengthened its child safety requirements. The very department whose lapses had produced the outrage refused to stagger the starting date of the new system for different community organisations. With the department’s processing capacity thus overwhelmed, it took over a month to clear the new paperwork. Family by Family was paralysed. If exceptions were allowed to the deadline, they were for more important folks than us. Overnight, practices that had worked brilliantly and safely for several years — that placed families at the centre of the program — became an offence. I don’t know about then, but today the department describes itself as “a customer-focused organisation that puts people first.”

In fact, an evaluation was done on Family by Family. The process was a train wreck. From memory numerous preliminary ethics processes took around nine months, though this was simply to ask families questions about their progress — as they’d been asked regularly within the program. The evaluation ignored the program’s effect on children. Why? Because getting that aspect through the ethics procedures would have been too expensive, uncertain and time-consuming. How ethical can you get?

When the evaluation finally began, the department funding the program wouldn’t give evaluators the data to identify our cohort of families. So the evaluation was forced to compare impacts on all families in the host suburb against two other areas (one of which was bizarrely incomparable). As I recall, the result was mildly positive but inconsequential — unsurprisingly, given the small number of families involved. To use J.K. Galbraith’s term, it was all “innocent fraud” — that is, all that effort and money produced an outcome that amounted to nothing. But its worthlessness was a system failure despite the best of intentions of everyone in it.

I expect that the National Health and Medical Research Council, which issued the ethics guidelines, the family services department and the university centre for family studies thought of themselves as putting people first. But far from nurturing the innovation breaking out on the edges of the system — driven by bright, idealistic, young professionals and increasingly enthusiastic families — the incumbent organisations imposed their own routines and imperatives, each one making the labyrinth denser, more bewildering, more dysfunctional, each one making it harder to put the families first.

Whether or not the evaluation report was released (I don’t believe it was), we all cooperated in covering up its worthlessness, which required nothing more than not to advertise it. This is just one close-up of a phenomenon the disillusioned development economist William Easterly has called “the cartel of good intentions.” It is built on Lord Acton’s fault line. But you won’t see any serious engagement with any of this in the PC’s material on Indigenous evaluation.

11. The perils and the promise of candour

You may think what I’ve written so far is scathing. Yet, as I indicated above, I think the PC makes the right basic calls in its draft strategy. Bereft as the report is of suggestions about how to bring it about, it nevertheless endorses more Indigenous involvement in evaluation. And it backs independence and transparency. In a system that’s nowhere near ready to seriously engage with such things, it also makes defensible compromises in shepherding those values into policy. The real shame is that the pathologies of the existing system are deeply entrenched and yet they hardly get a look-in in the commission’s analysis. So any strategy for shifting them requires something much more hard-headed — more problem-focused — than four pleasing adjectives and a well-intentioned tagline about putting Indigenous perspectives at its centre.

Here we get to Orwell’s point. The greatest service the PC could do Indigenous people — the way it could really put their interests at the centre of its concerns — would be to express itself simply and candidly. Its draft strategy asserts that program participants and the broader community should “have confidence that policies and programs are being assessed objectively and independently.” Poppycock. It should stop pretending and fess up on behalf of the system. Having recommended a highly compromised form of independence for now, it should explain that the system isn’t ready for much candour right now and explain why.

Now you can see the power of Orwell’s advice about speaking simply. Speaking simply makes it hard, excruciating even, for you to cover your tracks — to mask your motives — with the usual sophistry. Once the officialese is jettisoned (or should that be official-ease?) the discomfort that the system is defending itself against becomes its own discomfort in explaining the sorry situation it is dealing with. And the only way to relieve that discomfort would be to go further and sketch out a longer-term plan to reach the outcome described in the honeyed words.

12. Towards the final strategy

For the final strategy to deliver a minimum viable product, I think it needs these changes to the draft.

First, it should base its policy compromise on a much harder-headed understanding of the obstacles that stand between us and the land of “how.” After explaining why the whole system can’t possibly embrace real independence and transparency at the moment, it should go on to sketch its own vision of how that might be grown from the bottom up. I’ve shown one possible model with my proposal for an evaluator-general, which involves structural separation between the system’s doing on the one hand and its knowing and evaluating on the other. It needn’t be grandiose and system-wide: it can be built on a small scale and grown from there. Some submissions to the PC seem to think it has merit. The PC itself gives the idea considerable elaboration, but only as reportage. If it has a better model it should set it out.

Second, if the strategy is its contribution to thought, its direct contribution to action should be to call for and begin the process of designing a new burst of energy and innovation that might grow at the margins of current activity and begin to spread through the system.

Here, the current weakness of the system lies not so much in the lack of promising experiments in the field as in the relationship between them and the system itself. The system must be able to identify, validate and acknowledge the best of those experiments. Currently, it can’t do that. Evaluation can play some role in fixing that, though we should guard against something that’s already clearly in evidence — the system grabbing hold of evaluation as a deus ex machinait’s next fad diet that will save it from itself.

And there are two far graver obstacles to progress. First, as those in the field can attest, our politicians frequently play to their own political advantage irrespective of the evidence. Second, bureaucracies have terrible trouble responding to knowledge of what’s working from the field, for such bottom-up learning is countercultural in a hierarchy where power is at the top. Further, if learning were to rise from the bottom at any scale, it would involve the discomfort and uncertainty of change for large numbers of people.

The PC can do little about the first of these more serious problems. But it can hope to be influential regarding the second. I think it’s possible to be very concrete and specific about what is necessary here. The system can only sustainably expand what works by bolstering the status of the individuals and communities who have made it work and giving them much more authority and resources within that system.

Those at the centre of the system are just as important as the successes in the field, but there’s nothing unique about them — or there shouldn’t be if the system is working properly. Those in the system need to be made accountable not just for talking about expanding what works but for making sure it happens, despite the discomfort it will undoubtedly cause. To that end, a regular report could be recommended, by the auditor-general or some other independent guardian of integrity in the system, to document, say every two years, what progress was being made towards this goal of spreading “what works” and particularly the increasing empowerment of those who make it work.

For those of us who call ourselves Australians to properly begin the task that governor Arthur Phillip began with such high ideals and so little to show for it, we can only do it to the extent that non-Indigenous people and their institutions unloose themselves from those “anxious avaricious tentacles of the self.” To the extent we falter, the soft voice of conscience will keep whispering that destiny to us. •

This essay benefited from helpful comments on earlier drafts from Romlie Mokak, Keryn Hassall, Janina Gawler, Michael Griffith, Jon Altman, Mike Dillon, Christos Tsiolkas and Clive Kanes. As always, I am wholly responsible for the essay’s remaining inadequacies. The title “Orwell that ends well” is shamelessly stolen from my friend Konstantin Kisin.

Evaluation is not a thing

Published by Anonymous (not verified) on Tue, 14/07/2020 - 3:49pm in

An earlier version of this piece was published last week on the Mandarin.

Because the idea I have called “the Evaluator General” is several ideas knitted together to try to resolve a number of dilemmas, it comes with numerous implications that are often missed or misunderstood. So I’ve addressed them separately in specific articles. This article does the same, explaining that a central goal for me is for evaluation to become less of a ‘thing’ – separate from the activity it’s evaluating.

For better or worse, policymakers tend to come at evaluation from one of just two perspectives. First, for program managers it can give them an independent set of eyes to help assess how they’re going and how to improve. This can be particularly important in the public sector where objectives are multiple and won’t generally map onto any financial metric the way profit is the ultimate indicator of success in the private sector. Second, for those governing and funding programs, separate evaluation also meets accountability needs.

This and various other exigencies, such as people’s desire to build and participate in ‘professions’ has led to the growing institutionalisation and professionalisation of evaluation. There’s plenty to like about this. And as a new discipline and profession, evaluation is much fresher than mature disciplines whose intellectual foundations ossified years ago even when palpably unsatisfactory. This is true of my discipline economics,1
but of others too, their commanding heights confined to academia, an increasingly bureaucratised, fast foodified institution

The discipline of evaluation contains riches. But it is also a vast, loose network of approaches. Alas, in the push for more evaluation, it is being taken to be something far more settled and definitive than it is – as if getting something evaluated were like getting an auditor to check financial accounts or an engineer to check the structural integrity of a bridge. 

Indeed so called ‘goal free evaluation’ is an interesting and productive area of the discipline. There, the evaluator assesses the impact of the program without calibrating it against – or ideally even knowing – the program’s stated goals. This can improve program hygiene just as double blindness adds to the hygiene of a randomised controlled trial. It can also facilitate wider, and so potentially more powerful evaluative insights. These include unintended and/or negative consequences of a program, as well as its efficiency and effectiveness including system/network effects normally outside the program’s defined scope. (Nothing could demonstrate its value better than the central agencies obliviousness to its existence. It rarely dawns on the Great and the Good to forbear from exhaustively specifying the goals of the endeavours they fund).

Further, ‘evaluation’ didn’t play much of a role in the great technical achievements of humanity – the Apollo program or the development of the internet. And nor did ‘evaluation’ – conceived as formal and separate from delivering the goods – play much of a role in the delivery of AlphaZero’s technical wizardry in chess or the miracle of the Toyota Production System.

All those achievements required endless evaluative thinking.  But it took place as part of the process of doing the work, not as a ‘thing’ delivered from outside. But this isn’t how professions work. Professions sell services and so ‘evaluation’ is being brought into the production of government services as plumbing or landscaping would be. That’s just one reason why it’s not working well and won’t if we continue to misunderstand it. 

The PC’s recent work on indigenous evaluation, argues that:

Evaluation is most effective when it is integrated into each stage of policy and program development, from setting policy objectives and collecting baseline data, through to using evaluation findings to inform future policy and program design.

But it’s hard to operationalise these requirements except by bringing evaluation into and alongside operations in an ongoing capacity. Evaluative thinking is of the essence in most of the improvement organisations manage. And it’s in short supply – thus for instance the New Zealand Government’s Wellbeing strategy is focused on measuring wellbeing without directly considering how they can improve it. I hope an evaluator draws their attention to that sometime. But they’d be much better reflecting on it now. 

Good program design should contain a great deal of evaluation. If a particular mechanism is important – that children with particular learning needs are best handled in some particular way – it can be tested before we commit to it. And then again and again after we have. This is one of the things that, sad to say, it took ‘nudge units’ to introduce into many government programs – but as consultants from the outside of programs. But evaluation and testing goes on all the time in a well run organisation. It’s going on in Facebook and Google and Amazon and Toyota in numerous sites and programs as we speak.

Sometimes there’ll be a case for stepping back and so putting some space between operations and their evaluation. But that’s really quite rare in well run organisations. In many if not all of numerous examples presented in boxes in the PC’s work on indigenous evaluation, evaluation answers questions that come up, and could easily be handled as the program went on.  

Be that as it may, this was one of the things I wanted to encourage with my proposal for an Evaluator General. Under the arrangements as I envisage them, those delivering services work away for their line agency alongside those with expertise in evaluation who report to the line agency but are formally under the direction of the Evaluator General. Together those whose job is to do, and those whose job is to know collaborate to understand and improve the program day in day out. 

In his best-seller The lean start-up Eric Rees writes about how start-ups should use their presence in the market to learn. Instead of making complex plans based on lots of assumptions, he recommends making:2

constant adjustments with a steering wheel called the Build-Measure-Learn feedback loop. Through this process of steering, we can learn when and if it’s time to make a sharp turn called a pivot or whether we should persevere along our current path.

Now re-read the earlier passage from the PC. I defy you to explain how what’s called for can be delivered if evaluation is separated from what its evaluating. That’s why in my model, the Evaluator General is responsible for monitoring and evaluation. It also creates a scaffolding in which the distinctions between different types of evaluation in the literature, for instance between a summative focus (focused on accountability for impact) and a ‘formative’ one (focused on program improvement) can often mutually reinforce one another rather than be formally separated. 

The Evaluator General’s officers are tasked with knowing and recording, and prompting the evaluative thinking which, while it should assist with meeting pre-set program goals, should also range more broadly around all the things the program is achieving and might be brought to achieve.

Thanks to Keryn Hassall and Alexandra Ellinson for helpful comments on earlier drafts.

  1. As the philosopher Martha Nussbaum put it, “we have to grapple with the sad fact that contemporary economics has not yet put itself onto the map of conceptually respectable theories of human action. (Indeed, it has repudiated the rich foundations that the philosophical anthropology of Adam Smith offered it)”.
  2. Eric Ries. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, Currency, p. 41.

Against decentralising: why crowded is good

Published by Anonymous (not verified) on Wed, 20/05/2020 - 3:35am in

Tags 

innovation

Note: This post was original published on 6 July 2015; I’ve updated it several times because both parties keep revisiting a decentralisation agenda.

Once again we’re hearing the argument that Australia would be a much better place if only we could actively “decentralise” population. The argument is we should encourage people out of our big cities – notably Sydney and Melbourne – and into smaller cities, like Wollongong and Ballarat. One recent claim comes from the Liberal Party’s Tim Smith, the member for Kew and Victorian Shadow Parliamentary Secretary to the Leader (Population Policy and Housing Affordability). In an article in The Australian, he argues:

Opposition Leader Matthew Guy’s vision is to decentralise Victoria and develop its regional cities, to take the pressure off Melbourne and grow country Victoria.

The state desperately needs a government that is committed to decreasing the percentage of newcomers who make their home in Melbourne. Our state needs a government that will ­engage in a mature debate about how to incentivise newcomers to move to country Victoria, or give them the confidence that if they move to a regional centre they can commute to Melbourne with reliability and ease …

… An effective decentralisation agenda is key to improving capital city liveability and the economic wellbeing of the regions.

In pursuit of this, various governments over the years have tried to move departments out to regional cities. Smith implies that Labor doesn’t want decentralisation, but the evidence suggests Labor is just as keen on the idea as Smith is. The Victorian government under John Brumby even ran an advertising campaign in Melbourne encouraging people to move out and resettle in regional Victoria.

This sort of argument has often been based on the idea that these regional areas have lots of existing infrastructure that we can exploit at little cost. It has been encouraged by talk of the “Death of Distance” and “The Flat World” – the idea that globalisation and modern telecommunications are making location obsolete, so you might as well live in the countryside. It’s particularly popular wherever there are plenty of marginal regional electorates.

And this argument seem to be spreading. So here’s the case against spending government resources to actively encourage decentralisation.

The first argument against encouraging decentralisation is that it is essentially a policy to encourage people to live where they don’t want to live. People are capable of figuring out where they want to live, and of moving when things don’t suit them. At least for most people, the reasons for living in the city and the country aren’t hidden; they’re well-known. Let people vote with their feet, and don’t rig the poll.

A second argument, advanced in earlier eras is that we need to fill the countryside with people in order to defend it from invasion. This view is now largely obsolete, although I suspect it is still in the backs of the minds of some older Australians.

A third argument is that regional infrastructure is not necessarily up to the job of coping with strong growth after all. A couple of years ago I read the CEO of the Committee for Gippsland arguing in the Herald-Sun that a coming influx of residents to rural Victoria meant we need to spend more on regional infrastructure. This is in part a bait-and-switch: first we need decentralisation to exploit unused infrastructure, then we need more infrastructure to cope with the new population.

And the fourth and strongest argument against active decentralisation policies is that big, dense cities are good for innovation and economic growth and personal growth too, and that those benefits should be embraced rather than resisted. Crowding people into cities has literally built our civilisation. We shouldn’t start trying to fight the process without good reason. We should be finding ways to support it.

Australia’s city-dwellers have a long tradition of not criticising country life. It seems almost rude to point out that living in big cities is, in very many ways and for the vast majority of people, better than living in towns or the bush. Big cities benefit from enormous economies of scale and scope. Small towns struggle to get doctors; big cities have specialists who will be able to recognise your specific type of epileptic attack. A gay Chinese 17-year-old in a small town is likely horribly alone; put them in a city and they’ll find their subculture. An industrial designer in Roma has to rely on social media for conversation about widget design; in Prahan they can go down the pub with half-a-dozen colleagues and talk more and more wildly about widgets as the evening grows late.

The growth of cities is no accident. They represent a lot of people voting with their feet.

Recent years have brought a flood of new US books in praise of cities. Ed Glaeser is probably the world’s leading urban economist, and his 2011 book Triumph of the City made the argument for cities. “There is a near-perfect correlation between urbanization and prosperity across nations,” he wrote. “On average, as the share of a country’s population that is urban rises by 10 percent, the country’s per capita output increases by 30 percent … Across countries, reported life satisfaction rises with the share of the population that lives in cities, even when controlling for the countries’ income and education.”

Also in 2011, The Economist’s Ryan Avent published The Gated City. And in 2012 Berkeley economics professor Enrico Moretti, a human capital specialist, published The New Geography of Jobs (available at Amazon), buttressing the case for cities. Moretti’s work includes an NBER working paper called Human Capital Externalities in Cities – download it from the NBER or read a summary at CNN.

Enrico Moretti’s argument is that by jamming smart people together, cities produce ideas and boost productivity. Think Florence in the fifteenth century, London almost anytime in the past 600 years, but also Silicon Valley and San Francisco in California. The California example is the strangest: a huge slice of the people who are supposed to making distance obsolete live and work within 100 kilometres of each other. And that’s despite the fact that this clustering has made homes wildly expensive.

The argument is less common in Australian. The Grattan Institute has done some work on cities as engines of prosperity. And in 2006, the Australian economist Glenn Withers published a CEDA paper, Can Distance Be Defeated?, in which he made essentially the same argument as Moretti. Withers referred in particular to the work of Frank Levy and Richard Murnane, who were among the pioneers of the idea that the high-level knowledge that fuelled innovation was tacit and serendipitous and hence was “conveyed in the direct personal interaction that is a function of co-location”.

Or to put it another way, some of us get our best ideas down the pub with our mates.

Since I wrote the first version of this post in 2015, an empirical study has also popped up – The Optimal Distribution of Population across Cities, an NBER paper by David Albouy, Kristian Behrens, Frédéric Robert-Nicoud and Nathan Seegert. The authors are blessedly modest about their ability to calculate the right size for cities. But they do say that their modelling suggests cities “may well be too numerous and underpopulated for a wide range of plausible parameter values”. (The paper is also useful for the history and analysis it provides regarding economists’ claims over previous decades about city sizes.) None of Australia’s largest cities ranks in the world’s top 50 by population. On the NBER authors’ numbers, they may be smaller than is optimal for us.

No-one quite knows what a pro-cities policy would look like. Moretti favours policies that increase labour mobility and education levels. The Grattan Institute argues for better urban transport. Avent gives a more important role to regulatory policy. Withers, one of the original authors of Australia’s successful skilled migration program, emphasises the need to keep managing skilled migration successfully. But there’s more work to do teasing out any positive policy implications of this thinking. And I’m not arguing for any of these policies here.

Nevertheless, if these analysts are right, one policy implication is clear. If you want people to expand their minds and grow the economy, don’t actively discourage the growth in our cities or spend tax dollars to encourage “decentralisation”. Yes, avoid growth that destroys what has made our big cities attractive in the first place. But don’t pretend that getting people out of cities is a sensible policy goal in its own right. It may help economic growth in Wollongong and Ballarat, but the evidence suggests it will do so at the expense of national productivity, innovation and creativity.

Update: More recent writing on the economic benefits of cities:

  • Why Do Cities Matter? Local Growth and Aggregate Growth, by Chang-Tai Hsieh and Enrico Moretti  – Argues that increased constraints to housing supply in high-productivity cities like New York, San Francisco and San Jose lowered US GDP.
  • Urban Growth and its Aggregate Implications, by Gilles Duranton and Diego Puga – Another urban growth model where human capital spillovers foster entrepreneurship and learning in heterogenous cities.
  •  Order without Design: How Markets Shape Cities, by Alain Bertaud – A much-admired book which spends time on applying economics to cities. “Everybody should read it” – Paul Romer.

To drive innovation, scientists should open their doors to more equitable relations with the arts

Published by Anonymous (not verified) on Thu, 30/04/2020 - 8:00pm in

Interdisciplinary collaborations between scientific researchers and artists can often be one dimensional, with artists simply illustrating scientific findings. In this post Paige Jarreau argues that by engaging more openly and equitably with artists scientists and other researchers stand not only to better understand their own research and its reception, but also to develop new insights … Continued

Refugees Have Entrepreneurial Potential – Report

Published by Anonymous (not verified) on Mon, 26/10/2015 - 11:30am in

Tags 

innovation

Creating Digital Connections

Published by Anonymous (not verified) on Wed, 21/10/2015 - 10:49am in

Venture Philanthropy Fund Launches in Queensland

Published by Anonymous (not verified) on Wed, 21/10/2015 - 9:53am in

SA Surgeon Wins Social Entrepreneur of the Year

Published by Anonymous (not verified) on Wed, 21/10/2015 - 9:41am in

Pages