Expert Panel - Review of Elective Surgery and Emergency Access Targets under the National Partnership Agreement on Improving Public Hospital Services: Supplementary Annexure

2.3 Performance targets in health care

Kelman and Friedman (2009) noted that targets can drive improved performance in health care in a number of ways, for example:

  • psychological literature demonstrates that giving people goals motivates better performance, particularly if attached to incentives, although this is dependent on people accepting the need for the goals even in the absence of the incentives;
  • setting a performance target for one endeavour rather than another indicates to employees and other stakeholders what the organisation wishes to focus on; and
  • performance information improves learning within a unit by providing a source of feedback about the success of previous endeavours.36

There are, however, varied views on the appropriateness and value of performance targets, in the field of health care delivery. For example, Boxall (2009) highlights that health care activity can be impacted by a wide number of variables, such as the complexity of a case or the demographics of a patient, which may not be adequately taken into account within measurement processes.37 The complexity of performance measurement in health care is compounded by deciding who should set the performance criteria, how the criteria should be set and whether the criteria should be empirical or normative (that is, based on existing health care provision or in accordance with a specific standard).38

Monitoring and measurement of performance

Performance measurement seeks to monitor, evaluate and communicate the extent to which various aspects of the health system meet key objectives.39 Indicators can assist understanding of system performance and allow comparison over time and between systems.40

The Victorian Auditor-General’s Report (2009) states that such indicators are fundamental to performance measurement and quality assurance, and are important tools in the evaluation and audit process. Good indicators should support performance mechanisms by being:
  • relevant – linked to an objective in a logical and consistent way;
  • appropriate – give sufficient information to demonstrate the extent to which the objective is met;
  • fairly representative, meaning:
    • capable of measurement;
    • represent consistently what they purport to indicate; and
    • accurate.41

To support quality of care, there is evidence that performance strategies should be further supported by indicators focused on measuring patient-based outcomes, rather than solely relying on targets that are activity or time-based.42, 43 O’Connor et al (2006) noted that such an approach promotes good quality care by encouraging clinicians to adopt better medical treatments and minimise adverse events.44

Success of performance targets in reducing waiting times for hospital care

The use of targets as a performance tool in health and hospital care is increasingly reported in the literature. Targets are often seen as a way of defining and setting priorities, creating commitment to particular outputs, and providing a basis for follow-up and evaluation.45 In particular, as reflected by experiences in Australia and internationally, performance targets are being used as a measure to drive reduced waiting times for services.

As mentioned above, the NHS provides the most contemporary experience of performance targets at the whole-of-health system level. National wait-time targets, backed up by clear incentives for meeting them, were a major component of NHS policies between 1997 and 2010,46 and indicators of waiting times have been maintained.47 Progressively tougher targets for maximum waiting times for health services, such as elective surgery and emergency care, were set and enforced through close monitoring and financial incentives, as well as sanctions for low performance.48, 49

Statistical evidence from the NHS demonstrates improvements following introduction of the performance targets. Performance data reported under the NHS Four Hour Target found that the proportion of patients whose treatment was completed within four hours increased from 75 per cent in 2002 to nearly 98 per cent in 2006.50 With regards to elective surgery, data also demonstrates reductions in the number of patients on waiting lists and the length of waiting times for surgical services following the introduction of targets in the NHS,51 including high volume elective surgical procedures such as knee replacements, hip replacements and cataract surgery.52

Within the Australian context, there is also statistical evidence that time-based performance targets can be a driver of improved performance. For example, in its most recent quarterly performance report (to June 2011) WA Health noted that under its Four Hour Rule Program, the percentage of emergency department attendances seen and admitted, transferred or discharged in four hours or less in the four Stage One hospitals increased by 12.3% over the same quarter in 2010.53 Furthermore, in Section 4 of this Annexure we note information provided by WA Health that the level of access block in four tertiary hospitals in Western Australia fell from 30.3 per cent in April 2009, to 15.6 per cent in April 2010 and 6.9 per cent in April 2011.54

Also, results from Period 1 and Period 2 of the nationally implemented Elective Surgery Waiting List Reduction Plan demonstrate that states and territories have largely met the targets to reduce the number of patients waiting longer than clinically recommended times and to increase the number of elective surgery procedures being undertaken.55

In the NHS experience, Cooper et al (2009) noted the difficulty of ascribing the drop in waiting times for select elective surgery procedures to one policy reform rather than another, given the number of reforms that were introduced aimed at reducing waiting times.56

Epstein (2009) identified increasing interest in tying performance measurement to financial incentives but found that, despite this interest, data on the effectiveness of pay for performance was inconclusive and multiple factors were likely to impact upon its success.57 Boxall (2009) also concluded that there has been limited research on the effectiveness of pay for performance, and the available evidence showed mixed results. Boxall further reported on a 2008 literature review of pay for performance schemes in hospitals that found most studies were unable to attribute improvements to incentive payments alone as they were usually implemented along with other quality improvement initiatives.58

Risks of performance targets

The literature on performance targets also regularly refers to the possible risks and unintended consequences that can result from implementation of targets. The most widely reported risk is that of ‘gaming’,59, 60, 61, 62, 63, 64 which refers to behaviour that conceals non-performance, often through ambiguity in the way the data is reported or through outright fabrication.

‘Gaming’ can manifest in various forms, including ‘cherry-picking’ of patients (by giving preferential treatment or access to less complex patients to improve chances of reaching performance targets) and manipulating data in order to meet targets.65

The literature puts forward a number of strategies aimed at overcoming or minimising the potential for ‘gaming’ in association with health targets. For example, Bevan and Hood (2006) suggested more randomness and unpredictability in the assessment of performance, such as making it harder for providers to ascertain in advance who will assess them or making the relative weights of the targets unpredictable.66 Nocera (2010) suggested nationally consistent reporting and regular independent audits, as well as criminalising public sector data fraud.67

Another potential risk with performance targets which has been identified in the literature is ‘effort substitution’.68 This refers to a fixation on the achievement of targets, where providers become so preoccupied by meeting targets that patient outcomes and experiences become secondary – thereby ‘hitting the target but missing the point’.69 This is a particular risk if volume of output is easy to measure, but the quality is not, potentially leading providers focusing on volume (such as the discharge of patients from the emergency department within four hours) at the expense of quality care. Concerns regarding reduced quality of care have been linked with the NHS Four Hour Target, with criticisms raised that patient needs were sometimes treated as secondary to the importance of achieving the target.70, 71

A strategy identified in the literature as having the potential to overcome ‘effort substitution’ is if multiple performance domains, both qualitative and quantitative, are independently audited in conjunction with the target.72 Such an approach may involve measuring patient outcomes through the use of appropriate tools to capture enhancement to a patient’s quality of life and functioning as a result of the medical treatment received.73

In addition, targets can result in the alienation of key stakeholders. Hayes (2002), cited in Mortimore and Cooper, (2007) noted clinical concerns and criticisms with the NHS Four Hour Target over the lack of consultation, planning and communication involved in the formulation of the target.74, 75

Finally, there is some evidence that performance targets may generate increased demand. For example, NHS data demonstrates a 37 per cent rise in emergency department presentations between 2002 and 2006, in conjunction with the introduction and enactment of the Four Hour Target for emergency departments. However, there is some suggestion that other NHS reforms during this period, such as changes in the provision of after-hours primary care, contributed to the sharp increase in emergency department presentations.76

36. Kelman, S. and Friedman, J., ‘Performance improvement and performance dysfunction’. Journal of Public Administration Research 2009, Vol. 19, pp. 917-946.
37. Boxall, A., ‘Research Paper: Should we expand the use of pay-for-performance in health care?’ Parliamentary Library, Canberra, 2009.
38. O’Connor, R. and Neumann, V., ‘Payment by results or payment by outcome? The history of measuring medicine’. Journal of the Royal Society of Medicine 2006, Vol.99, pp. 226-231.
39. Smith, P., Mossialos, E., Papanicolas I. and Leatherman, S., 'Chapter 1.1: Introduction’ in ‘Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects', European Observatory on Health Systems and Policies, 2009.
40. Victorian Auditor-General's Report. Access to Public Hospitals. April 2009.
41. ibid.
42. O’Connor, R. and Neumann, V., 'Payment by results or payment by outcome? The history of measuring medicine'. Journal of the Royal Society of Medicine 2006, Vol. 99, pp. 226-231.
43. Nocera, A., 'Performance-based hospital funding: a reform tool or an incentive for fraud?' Medical Journal of Australia 2010, Vol. 194, No. 4, pp. 222-224.
44. O'Connor, R. and Neumann, V., 'Payment by results or payment by outcome? The history of measuring medicine'. Journal of the Royal Society of Medicine 2006, Vol. 99, pp. 226-231.
45. Smith, P., Mossialos, E., Papanicolas I. and Leatherman, S., ‘Chapter 1.1: Introduction’ in ‘Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects’, European Observatory on Health Systems and Policies, 2009.
46. Thorlby, R. and Maybin, J., (eds), ‘A High Performing NHS? A review of progress 1997-2010’, The King’s Fund, 2010.
47. UK Department of Health, A&E Clinical Quality Indicators: Best Practice Guidance for Local Publication, 2011.
48. Thorlby, R. and Maybin, J., (eds), ‘A High Performing NHS? A review of progress 1997-2010’, The King’s Fund, 2010.
49. Hughes, G., ‘Four hour target for EDs: The UK experience’. Emergency Medicine Australia 2010, Vol. 22, pp. 368-73.
50. Wanless, D., Appleby, J., Harrison, A. and Patel, D., ‘Our Future Health Secured: A Review of NHS Funding and Performance’. 2007, Kings Fund, p. 30.
51. Thorlby, R. and Maybin, J., (eds), ‘A High Performing NHS? A review of progress 1997-2010’, The King’s Fund, 2010.
52. Cooper, Z., McGuire A., Jones S., Le Grand J. and Titmuss R., ‘Equity, waiting times and NHS reforms’. British Medical Journal 2009, Vol. 339, No. 3264, pp. 1-7.
53. Government of Western Australia, ‘WA Health Performance Report: April to June 2011 Quarter’, Department of Health, 2011.
54. Government of Western Australia, Submission to the Expert Panel, Department of Health, 2011.
55. COAG Reform Council, National Partnership Agreement on the Elective Surgery Waiting List Reduction Plan: Period 1 Assessment Report, 2010 and National Partnership Agreement on the Elective Surgery Waiting List Reduction Plan: Period 2 Assessment Report, 2010.
56. Cooper, Z., McGuire A., Jones S., Le Grand J. and Titmuss R., ‘Equity, waiting times and NHS reforms’. British Medical Journal 2009, Vol. 339, No. 3264, pp. 1-7.
57. Epstein, A., ‘Chapter 5.5: Performance measurement and professional improvement’ in ‘Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects’, European Observatory on Health Systems and Policies, 2009.
58. Boxall, A., ‘Research Paper: Should we expand the use of pay-for-performance in health care?’ Parliamentary Library, Canberra, 2009.
59. ibid.
60. Bevan, G. and Hood, B., ‘Targets, inspections and transparency: Too much predictability in the name of transparency weakens control’. British Medical Journal 2004, Vol. 328, No. 7440, p. 598.
61. Mountain, D., ‘Introduction of a 4-hour rule in Western Australian EDs’. Emergency Medicine Australasia 2010, Vol. 22, pp. 374-378.
62. Bevan, G. and Hood, B., ‘Have targets improved performance in the English NHS?’ British Medical Journal 2006, Vol. 332, No. 7538, pp. 419-422.
63. Bevan, G. and Hood, B., ‘What’s Measured is What Matters: Targets and Gaming in the English Public Health Care System’. The Public Services Programme: Quality, Performance and Delivery, 2005.
64. Nocera, A., ‘Performance-based hospital funding: a reform tool or an incentive for fraud?’ Medical Journal of Australia 2010, Vol. 194, No. 4, pp. 222-224.
65. Boxall, A., ‘Research Paper: Should we expand the use of pay-for-performance in health care?’ Parliamentary Library, Canberra, 2009.
66. Bevan, G. and Hood, B., ‘Targets, inspections and transparency: Too much predictability in the name of transparency weakens control’. British Medical Journal 2004, Vol. 328, No. 7440, p. 598.
67. Nocera, A., ‘Performance-based hospital funding: a reform tool or an incentive for fraud?’ Medical Journal of Australia 2010, Vol. 194, No. 4, pp. 222-224.
68. Kelman, S. and Friedman, J., ‘Performance improvement and performance dysfunction’. Journal of Public Administration Research 2009, Vol. 19, pp. 917-946.
69. Boxall, A., ‘Research Paper: Should we expand the use of pay-for-performance in health care?’ Parliamentary Library, Canberra, 2009.
70. Smith, P., ‘England: Intended and Unintended Effects’ in Wismar, M, McKee, M, Ernst, K, Srivastava, D, Busse, R, (2008) Health targets in Europe, Learning from experience, Observatory Studies Series № 13, European Observatory on Health Systems and Policies, Copenhagen, p. 72.
71. Healthcare Commission (2009), Investigation into Mid Staffordshire NHS Trust, Commission for Healthcare Audit and Inspection, London, pp. 49-50.
72. Bevan, G. and Hood, B., ‘Have targets improved performance in the English NHS?’ British Medical Journal 2006, Vol. 332, No. 7538, pp. 419-422.
73. O’Connor, R. and Neumann, V., ‘Payment by results or payment by outcome? The history of measuring medicine’. Journal of the Royal Society of Medicine 2006, Vol. 99, pp. 226-231.
74. Hayes, J., (2002), cited in Mortimore, A. and Cooper, S., ‘The “four-hour target”: emergency nurses views’, Emergency Medicine Journal, 2007, Vol. 24, No. 6, pp. 402-404.
75. Hughes, G., ‘Four hour target for EDs: The UK experience’. Emergency Medicine Australia 2010, Vol. 22, pp. 368-73.
76. UK Healthcare Commission (2008) and Robinson, P. (2007), cited in Jones, P. and Schimanski, K., ‘The four hour target to reduce emergency department ‘waiting time’: A systematic review of clinical outcomes’. Emergency Medicine Australasia 2010, Vol. 22, pp. 391-398.

Top of page

prev pageContents |next page

Table of contents

prev pageContents |next page

Page last updated 26 August, 2011