Surveyors perceptions of the impact of accreditation

Events

Surveyors’ perceptions of the impact of accreditation on patient safety in general practice

Article published in the Medical Journal of Australia, August 2014

Authors: Amr Abou Elnour, Andrea L Hernan, Dale Ford, Stephen Clark, Jeffrey Fuller, Julie K Johnson and James A Dunbar

“The APCC Program’s success in improving quality is evident, and was well endorsed by participants in our study to be a program that could promote and improve the patient safety culture in Australian primary care. With this in mind, we recommend that the APCC Program add patient safety to its agenda for improvement”.


Abstract

Objectives: To explore Australian General Practice Accreditation Limited (AGPAL) surveyors’ perceptions of the impact of accreditation on patient safety and to elicit suggestions for improving patient safety in Australian general practices.

Design, setting and participants: We conducted semi-structured telephone interviews with a purposive national sample of 10 AGPAL surveyors from 2 July to 14 December 2012. All interviews were audio recorded, transcribed and summarised.

Results: All participants agreed that accreditation has improved general practices’ performance in quality and safety. Participants noted specific areas that need further attention, including sufficient evidence for clinical risk management, which half the participants estimated occurs in about 5%–10% of Australian general practices. Tangible evidence of patient safety activities included having a significant incidents register, providing documentation of near misses, slips, lapses or mistakes, and engaging in regular clinical meetings to discuss incidents and how to avoid them in the future. Participants agreed that the accreditation process could be improved through the inclusion of tighter clinical safety indicators and the requirement of verifiable evidence of a working clinical risk management system.

Conclusions: Accreditation has had a positive role in improving quality and safety in general practice. The inclusion of tighter indicators that require verifiable evidence will be a step forward. The Australian Primary Care Collaboratives (APCC) Program has an opportunity to build on its previous success in general practice quality improvement to further enhance patient safety in general practice.

Accreditation of health services and its potential role towards improving health care has been described previously.1-3 Improving patient safety through the process of accreditation has been noted in the acute care setting.4-6 Further research is needed to evaluate patient safety in accreditation of primary care, its impact, and how to improve it.

The Royal Australian College of General Practitioners (RACGP) has developed a set of standards to protect and improve the safety and quality of health care provided in general practices. These standards serve as a “template for safe and high quality care”, for the general practices that have joined an accreditation program (about 80% of practices in Australia).7 These standards are used by accreditation agencies to carry out the accreditation process through experienced surveyors who are involved in primary care. GP surveyors and co-surveyors, such as practice managers or practice nurses, carry out the accreditation process, which is approved by the agency accreditation review committee.

One of the RACGP standards is to provide a systematic approach for clinical risk management, to recognise and avoid near misses, slips, lapses or mistakes.7,8 Risk management is defined by the RACGP as “the culture, processes and structures that are directed toward the effective management of potential opportunities for adverse events”.7

Patient safety in primary care is underestimated due to poor use of available data and difficulties in measuring safety incidents,9,10 although several studies have described the errors and harms that occur in primary care in Australia11,12 and internationally.13-17

The Australian Safety and Quality Framework for Health Care,18 endorsed by heath ministers in 2010 as an important driver of quality, has highlighted three key domains for creating safe, high-quality care. One of these domains requires health services to be “organised for safety”, which further emphasises the importance of accreditation in reducing harm in general practice.

This study aimed to explore Australian General Practice Accreditation Limited (AGPAL) surveyors’ perceptions of the impact of accreditation on patient safety, and to elicit suggestions for improving patient safety in Australian general practices.

Methods
We asked AGPAL to recruit a national purposive sample of their surveyors on our behalf to represent most Australian states and territories, their experience in the use of different editions of the RACGP standards for accreditation, and number of practices they had accredited. AGPAL sent an email invitation with the study information and consent form to surveyors. Interview questions centred on the process of accreditation, general practice performance and patient safety (Box 1).

We conducted semi-structured telephone interviews (40–60 min) with participant surveyors from 2 July to 14 December 2012. All interviews were audio recorded, transcribed and summarised using the interview schedule as a guiding framework to identify participant perspectives of the impact of accreditation on safety in general practice.

This project was granted ethics approval from Flinders University Social and Behavioural Research Ethics Committee (project no. 5609).

Results
Eleven surveyors consented to participate; one declined at a later time due to time constraints. The 10 AGPAL surveyors who participated in the study were involved in the accreditation of 2022 general practices over 15 years across the Australian states of New South Wales, Victoria, Queensland, Western Australia, South Australia and Tasmania, and the Australian Capital Territory. Seven were GP surveyors and three were co-surveyors. The number of practices accredited by each surveyor varied from 100 to more than 400 practices, except for two surveyors who had accredited only five practices (Box 2). Several editions of RACGP standards had been used by eight surveyors.

Accreditation and overall general practice performance

Overall, participants were of the view that accreditation improved general practice performance. This was particularly noted as an improvement over time and as a result of the accreditation process. Examples of improvement centred on the physical environment of general practice, such as equipment safety, appointment systems, patient records and electronic records.

No doubt. Over the years I’ve seen a significant improvement in general practice systems and the way they approach patient care, and a lot of that is directed to the standards. (Participant 10)

Some participants mentioned that the systematic approach of accreditation provided a method for improving general practice systems, such as through establishing business thinking and activity in general practice.

It’s made general practices realise that they’re a business, like any other business that has standards that they must conform to. I think what it’s also done is raise the profile and the importance of good staff and good nursing staff. (Participant 2)

However, doubts were also expressed about accreditation and its impact on improved practice performance, such as the “one-off” nature of accreditation and sustainability of improvement efforts.

… there is often a nagging doubt, are they [general practices] like this all the time? (Participant 4)

General practice staff awareness of patient safety

There were mixed responses regarding general practice staff awareness of patient safety, evident in the two quotes below, from concerns about an overall lack of awareness to views that awareness was related to staff roles and responsibilities; especially for practice managers and practice nurses, who were regarded as having high awareness because they were mainly responsible for the accreditation process and activity.

I don’t think there’s a huge awareness, I don’t think it’s great, I think it’s certainly an area that can be improved a lot. (Participant 6)

It’s clinical staff, not medical staff, that adhere to it and embrace it more enthusiastically. (Participant 3)

The following participant noted that safety awareness was discipline-specific, with GPs being aware of the clinical components of safety and other staff being aware of the physical safety elements of general practice.

I would say that doctors would be highly aware of clinical component. Staff I would say would be awfully conscious of the physical environment. (Participant 5)

Patient safety as a component of the accreditation process

Most participants stated that patient safety indicators are included in the accreditation process. The indicators primarily mentioned were from section five of the RACGP standards,7,8 which covers the physical environment of the practice such as infection control and sterilisation, cold chain, vaccination and physical access. Here, it was noted that accreditation lacks the ability to effectively assess safety in clinical practice as distinct from this focus on safety processes in the physical environment, which are easily recognised as being achieved or not.

They say there are standards for patient safety but there’s nothing that you can really tie down to patient safety. (Participant 10)

Participants suggested that the accreditation process could be improved through the inclusion of tighter clinical safety indicators and the requirement of verifiable evidence of a working clinical risk management system.

Evidence of clinical risk management systems in general practice

Participants confirmed that most general practices did not have sufficient evidence to demonstrate the existence of a clinical risk management system.

I think this was a really important standard introduced into general practice without anywhere near enough education for practices to understand. So most practices I go into, when I talk about slips, lapses and mistakes they look at me blankly; that would be 70%–80% of practices that I go to. (Participant 6)

The general practices considered to be high performing on patient safety indicators were described as having evidence that verified an active and effective clinical risk management system. These included having a working clinical risk management document, incident register or policy, and staff participating in meetings and discussion about slips, lapses and mistakes, and producing documented outcomes and actions.

I guess a living, breathing document, not one that’s just for the surveyor team but one that’s got quite a few entries in it and better still they can go into that area [and] show an improvement that has been effective as a result of that. (Participant 3)

Participants were asked to estimate the number or proportion of practices performing well in patient safety indicators and in clinical risk management. There were mixed estimates; five of the participants estimated that 5%–10% of general practices they had accredited were high performers in patient safety, three participants estimated 30%, and one participant estimated less than 1%. Only one participant gave a high estimate of around 90%. It was noteworthy that the more experienced surveyors (accredited more practices) provided lower proportions of general practice performing well on patient safety indicators.

Improving patient safety culture in general practice

Participants were asked to recommend approaches to improve the safety culture in general practice. Their responses included further education and training, and novel ways of thinking, emphasising the roles of practice managers and practice nurses, instilling an interest and inclination towards improvement and safety, establishing open, honest, and sharing communication practices along with reflection and documentation. Also mentioned were introducing a business model to practices, creating systematic and “multi-pronged” interventions to introduce change, and funding and incentives for change. While the above recommendations were viewed as enabling change, some also acknowledged the difficulties and challenges required when attempting to change practice behaviour and activities.

To optimise change in general practice, you have to think about all the ways in which you might influence GPs to bring about change. Change is not one simple thing — it takes a range of things. (Participant 6)

Following on with discussions around possible ways to improve patient safety culture in general practice, participants were asked about the Australian Primary Care Collaboratives (APCC) Program. One surveyor who had recently joined AGPAL was not aware of the program. Nine surveyors believed that the APCC Program is commendable, with potential to promote and improve patient safety culture in Australian primary care because of the systems that it introduces to practices.

Absolutely, I’ve seen it time and time again that when the practice is involved in APCC, patient safety and their interest in recall systems [and] registers becomes high and they encourage it. (Participant 10)

Discussion
This is the first study in Australia to examine AGPAL surveyors’ perceptions of the impact of accreditation on patient safety in general practice. The findings suggest that accreditation has improved the safety and quality of Australian general practice, but there is still room for improvement, particularly concerning clinical safety and providing verifiable indicators that require practices to demonstrate evidence around clinical risk management.

Participants affirmed that the physical factors of the environment affecting safety, such as infection control and sterilisation, cold chain, vaccination and physical access, are well addressed during the accreditation process. Conversely, clinical risk management indicators lacked sufficient verifiable evidence needed to demonstrate acceptable levels of safety. Furthermore, this type of evidence was thought to be provided by only 5%–10% of Australian general practices.

One solution could be the addition of extra procedures in the accreditation process that require evidence of clinical risk management. Those suggested by participants included: a) having a significant incidents register; b) providing documentation of near misses, slips, lapses or mistakes; and c) engaging in regular clinical meetings to discuss incidents and how to avoid them in the future.

Accreditation could advance the use of clinical risk management in general practice, as outlined in the RACGP standards with the proposed recommendations listed above. Additionally, these recommendations are a step forward to meeting the proposed actions to prevent or minimise harm from health care errors reported in the Australian Safety and Quality Framework for Health Care.18

Incident registers with documentation of significant safety incidents could be the foundation to developing a national register for anonymous reporting of errors and near misses.

The APCC Program’s success in improving quality is evident,19,20 and was well endorsed by participants in our study to be a program that could promote and improve the patient safety culture in Australian primary care. With this in mind, we recommend that the APCC Program add patient safety to its agenda for improvement.

Limitations to this study include the possibility of selection bias and the use of self-report. The number of participants was small and surveyors not sampled for this study may have different responses and hence different recommendations and conclusions to study participants. Finally, the proportion of practices who were perceived as high performers for clinical risk management activities are based on self-reported estimates from participants, and therefore cannot be considered to be an actual estimate.

Our study suggests the current “softness” around patient safety indicators in the accreditation process for general practice needs to be improved. The inclusion of tighter indicators that require verifiable evidence is a step forward to retaining the positive role of accreditation in improving general practice performance in quality and safety. Furthermore, given the APCC’s previous success in general practice quality improvement, it would be an appropriate organisation to further improve patient safety in general practice.

Box 1 – Interview questions posed to surveyors

•Do you think accreditation improves general practice performance? If yes, how?
•In general, are general practice staff aware of patient safety?
•Is the accreditation process examining patient safety? If yes, how?
•During accreditation, have you ever seen evidence of patient safety indicators (ie, a clinical risk management system)?
•Do general practice staff have regular meetings to review slips, lapses and mistakes?
•Can you estimate the percentage of best practices in patient safety?
•What is the optimal way to improve and disseminate patient safety culture in general practice?
•Do you think the Australian Primary Care Collaboratives Program could help improve patient safety culture?

Box 2 – Characteristics of participating surveyors (n = 10)

——————————————————————————–
Provenance: Commissioned; externally peer reviewed.
Received  12 Feb 2014,  accepted  21 May 2014

PDF version of article

August 18, 2014

IMPROVEMENT FOUNDATION
Level 8, 19 Grenfell St Adelaide SA 5000 | PO Box 3645
T: (08) 8422 7400 | TOLL FREE (within Aust): 1800 771 522 | F: 08 8231 6690 | E: enquiries@improve.org.au

The Improvement Foundation acknowledges the Traditional Custodians past and present on whose land we walk and we live.

TQCSI Quality Certified
ISQua