Journals Library

• Other NIHR Research
  • School for Primary Care Research
  • School for Public Health Research
  • School for Social Care Research
  • Canadian Institutes of Health Research (CIHR) and NIHR Collaboration

Developing testing and implementing the NIHR School for Primary Care Research Patient Safety Toolkit in general practices in England

Developing, testing and implementing the NIHR School for Primary Care Research Patient Safety Toolkit in general practices in England

113

31 August 2015

01 November 2011

31 July 2015 (No cost extension of 7 months)

3 years 6 months 

Patient safety, Primary care, Toolkit, Quantitative analysis, Qualitative analysis

Professor Anthony Avery, Dean of the School of Medicine; Professor of Primary Health Care, Faculty of Medicine & Health Sciences, University of Nottingham

• Professor Stephen Campbell, University of Manchester
• Professor Aneez Esmail, University of Manchester
• Dr David Reeves, University of Manchester
• Professor Umesh Kadam, University of Keele
• Dr Manny Samra, University of Keele
• Dr Mark Porcheret, University of Keele
• Dr Paramjit Gill, University of Birmingham
• Professor Sheila Greenfield, University of Birmingham
• Professor Jose Valderas, University of Exeter
• Professor Michael Moore, University of Southampton
• Professor Paul Little, University of Southampton
• Dr Jeffrey Aronson, University of Oxford
• Dr Ignacio Riccicabello, University of Oxford
• Dr Sarah Purdy, University of Bristol
• Dr Rachel Spencer, University of Nottingham
• Kate Marsden, University of Nottingham
• Dr Sarah Rogers, University of Nottingham
• Dr Brian Bell, University of Nottingham
• Mr Antony Chuter, PPI representative
• Professor Aziz Sheikh, University of Edinburgh
• Dr Sarah Slight, University of Durham

• Dr Ian Litchfield, University of Birmingham
• Dr Lucy Doos, University of Birmingham
• Dr Katherine Perryman, University of Manchester
• Jane Barnett, University of Southampton
• Katie Dixon, University of Keele

Project objectives 

Purpose

The overall purpose of this study is to develop, test and implement the NIHR School for Primary Care Research (SPCR) Patient Safety Toolkit in general practices in England.

Aims:

Primary Objectives

1. To identify tools for assessing the safety of general practices and obtain consensus on their use in the Patient Safety Toolkit;
2. To test these patient safety tools in a sample of general practices to help decide which should be included in the final toolkit;
3. To investigate the implementation of the complete Toolkit.

Secondary Objectives

Stage 1 (Developing the Patient Safety Toolkit)

1. To undertake two focused systematic literature reviews to identify: i) tools, and associated outcome measures, used to assess aspects of patient safety in general practices; and ii) qualitative studies on the experience of patients and/or health professionals of patient safety in general practice;
2. To conduct a survey of international general practice organisations to identify any additional tools for assessing patient safety;
3. To create a conceptual framework of the key attributes of safe general practice;
4. To obtain consensus on the attributes of safe general practice that need to be covered by the Patient Safety Toolkit;
5. To develop new methods for extracting data from GP computer systems for prescribing safety indicators;

Stage 2 (Testing the elements of the Toolkit)

1. To test the particular attributes (acceptability, technical feasibility, reliability, and validity) of the different elements of the Patient Safety Toolkit;
2. To select a final set of tools based on their performance and coverage of key safety issues.

Stage 3 (Implementing the complete Patient Safety Toolkit)

1. To obtain feedback from practices on implementation issues;
2. To identify and (where possible) estimate the potential costs and benefits of implementing the Patient Safety Toolkit;
3. To identify potential predictors of patient safety;
4. To obtain data for future sample size calculations.

Changes to Protocol

We had intended to develop a website/dashboard to promote the Patient Safety Toolkit. By stage 3 the team had developed a storyboard and were in discussions with a web developer. However, with the RCGP taking up the Patient Safety Toolkit as part of the Spotlight Project they have used our storyboard information as the basis for the information to host the toolkit on their website.

Brief summary

Background

This was a multi-centre study, combining quantitative and qualitative methods. The overall purpose of this study was to develop, test and implement a Patient Safety Toolkit in general practices in England. 

Aims and objectives

Stage 1

This stage had previously been completed and involved identification of tools considered suitable for identifying and addressing patient safety issues. This included:

a) Undertaking focused systematic literature reviews to identify:

b) Conducting a survey of international general practice organisations to identify any additional tools for assessing patient safety;

c) Conducting interviews with UK and international experts to explore the attributes of safe general practice;

d) Selecting a set of tools for testing in a sample of general practices

 The set of tools identified in this study included the following:

• The Trigger Tool, developed by the NHS Education for Scotland (NES) The Trigger tool is intended to be an efficient way of performing a case note review to highlight any potential area of poor patient safety
• The Medicines Reconciliation Bundle, developed by NES. This tool assesses the safety of the interface between primary and secondary care in terms of the general practice role in medication reconciliation following a patient’s discharge from hospital.
• PC SafeQuest. Developed by NES this is an on-line anonymised questionnaire that can be used by all members of a general practice team. It allows staff to rate the perceived safety climate within their practice.
• Manchester Patient Safety Framework. This tool allows for team based self-reflection on safety culture. It provides a qualitative assessment of safety culture that can help practices to identify areas for improvement.
• Prescribing Safety Indicators. Our team designed a set of computerised indicators to assess prescribing safety and we have been working with PRIMIS to develop this tool. As a result of a further literature review and consensus building exercise we now have a set of 56 indicators. It will involve the use of CHART (Care and Health Analysis in Real Time) software to extract data on patients at risk of medication-related injury.
• Significant Event Analysis and reporting of adverse events. Significant event analysis is a process that can be used by general practices to investigate adverse events (and near misses) relevant to patient safety.
• Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC). As part of Stage 1 of the study we undertook systematic reviews to identify studies of the experience of patients with respect to patient safety in general practice, and on patient reported safety outcomes. As a result, we found that there were no suitable tools available for assessing patient reported experiences and outcomes of safety in primary care. We therefore developed a new tool (the Patient Reported Experiences and Outcomes of Safety in Primary Care tool), taking account of best practice in instrument design. PREOS-PC is a questionnaire that will be posted to patients. Two focus groups, were conducted with patient and public involvement volunteers in order to obtain further clarification on, and increased understanding of, any issues arising from use of PREOS-PC.
• Patient Safety checklists. As a result of the literature reviews and consensus building exercise completed in stage 1, the team identified a number of important aspects of patient safety not currently covered by the tools highlighted above. As part of Stage 2 of this study we planned to address these gaps by developing a number of patient safety checklists.

Stage 2

This stage of the study involved the piloting of the above set of patient safety tools with practices recruited from the three centres; Manchester, Keele and Nottingham. Each centre was asked to recruit up to 10 practices plus we recruited an additional 15 practices in Nottingham to pilot the Prescribing Indicators tool. Each practice was only given up to 4 tools so that detailed feedback (qualitative and quantitative) could be obtained.

In each participating practice, staff were asked to complete workload diaries, which allowed the effort associated with using each of the different tools in the Patient Safety Toolkit to be assessed. The diaries included sections on the amounts of time necessary to undertake planned (and any other related but unplanned) activities. This information has been important for assessing the costs of implementing the toolkit.

We assessed the acceptability, technical feasibility, reliability, and validity of each element of the Patient Safety Toolkit. Semi-structured interviews were conducted with a diverse range of practice staff, including GPs, practice managers, nurses, and other relevant staff. These interviews explored interviewees’ expectations, experiences, perceived needs, and unintended consequences associated with implementing the tools in their practices.

Various quantitative methods were agreed and utilised to measure the results arising from using the tools. Some of these tools, such as the NHS Education for Scotland Trigger Tool, required clinicians from the practices to examine patient notes, but no patient identifiable information was fed back to the research team.

For the PREOS-PC we asked the participating practices to send out the questionnaire to a random selection of 75 patients (criteria being over the age of 18 and not classified as vulnerable). The completed forms were returned to our colleagues in Oxford who analysed the results.

Stage 3

As a result of the quantitative and qualitative analysis, the project team considered which tools to take forward for the implementation stage of the project. The tools decided on:

• The Trigger Tool was adapted as a result of comments arising from interviews and the quantitative analysis.
• The Medicines Reconciliation Tool was adapted as a result of the quantitative and qualitative analysis from stage 2
• PC SafeQuest was used again without changes
• Manchester Patient Safety Framework was not used in stage 3
• Prescribing Safety Indicators – The original 8 prescribing safety indicators and another 15 new prescribing safety indicators were used in Stage 2. In Stage 3 only the 8 original indicators were used.
• Patient Safety Checklists were adapted and refined arising from the analysis in stage 2, later it was re-named the Concise Safe Systems Checklist
• Significant Event Analysis and reporting of adverse events was not used in stage 3
• Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) was further refined as a result of the responses and the analysis from stage 2

Stage 3 expanded the number of practices by retaining the ones used in the previous stage and by recruiting up to 10 practices each from Southampton and Birmingham. Each practice was again asked to use up to 4 tools from the toolkit. Each practice was also asked to fill in a workload diary to assess the time and effort put into each tool. Interviews were conducted at each practice following completion of the tools to explore the acceptance and validity of each tool. For the PREOS-PC all practices recruited were asked send out the questionnaire to 150 patients in their practices, the criteria remained the same as in stage 2. In one area, Keele, we sent reminders to the same patients, to identify whether the rate of response improved with the extra posting.

During stage 2 and 3, other tools were assessed that had been designed and published. As a result the team assessed another checklist instrument which was subsequently included in the completed toolkit.

Key Findings 

The results from stage 3 indicated that:

1. Staff members generally thought that safety climate goals (measured by the PC-Safequest) were achieved in their practice, although relatively low scores on the Workload scale suggested that excessive workload may compromise patient safety.
2. Most (more than 70%) staff members reported that medicines reconciliation occurred quickly following hospital discharge (within 2 working days after receipt of the necessary documents). However, discussions with the patient or the carer often did not occur (53% of the time) although a discussion was clinically necessary most of the time (57% of the time).
3.  More than a quarter (27%) of the triggers found in patient records had the potential to cause moderate or severe harm with a third (32%) of these potentially preventable and to have originated in primary care.
4.  For the prescribing indicators, the percentage of patients at risk ranged from a low of 0.12% (for female patients with a history of thromboembolism and arterial thromobosis prescribed combined oral contraceptives) to a high of 52% (for patients prescribed amiodarone for 6 months or longer without a thyroid function test).
5.  The safety checklist revealed that although most (75% or more) of the respondents thought their practices performed well, two areas showed a need for improvement (a) follow-up of vulnerable patients who are discharged from hospital and (b) non-collection of prescriptions.
6.  The PREOS-PC showed that patients generally reported that their practice was safe (mean safety score of 8.6 out of 10) although 40% of the patients experienced at least one safety problem in the past 12 months and 20% reported experiencing harm in the last 12 months

Plain English summary

Patient safety is a complex topic as it encompasses many aspects of care. Therefore, no tool would provide a comprehensive way of measuring patient safety. Patient safety includes prescribing and diagnostics safety, communication (within and beyond the practice), organisational culture, and patient reported problems. This project has identified tools that address all of these topics:

1) The Trigger Tool - an audit for reviewing patient notes to identify harm in primary care

2) Medicines reconciliation – to identify the safety of the interface between primary and secondary care in terms of medicine reconciliation

3) PC SafeQuest - an on-line anonymised questionnaire that can be used by all members of a general practice team to rate the perceived safety climate within the practice. 

4) Manchester Patient Safety Framework - to help organisations understand how safety is perceived by staff, which allows practice groups to reflect on their safety culture.

5) Concise Safe Systems Checklist - allows practices to think about those background systems which are important for patient safety, but are often overlooked. It is deliberately designed not to include items already covered by legislation or mandatory requirements.

6) The General Practice Safety Checklist - identifies hazards across the wider work systems that may threaten patient safety, as well as those hazards that have an impact on the health, safety and well-being of all involved.

7) Prescribing Indicators - these are a set of computerised indicators to assess prescribing safety.

8) Patient Safety Questionnaire – to assess patient reported experiences and outcomes of patient safety in primary care

Some of these tools have been adapted from other sources, some have been accepted as they are and others have been designed and developed as a result of this project.

Dissemination

Published papers

Ricci-Cabello I, Gonçalves DC, Rojas-García A, Valderas JM. Measuring experiences and outcomes of patient safety in primary care: a systematic review of available instruments Fam Pract. 2015 Feb;32(1):106-19.

Accepted for publication

Bell B, Reeves D, Marsden K, Avery A. Safety Climate in English General Practices: Workload Pressures May Compromise Safety, Journal of Evaluation in Clinical Practice

Submissions under review

Ricci-Cabello I, Pons-Vigués M, Berenguera A, Pujol-Ribera E, Slight S P, Valderas J M. Patients’ Perceptions and Experiences of Patient Safety in Primary Care in England. submitted to:BMJ Qual & Saf

Submissions in preparation

Ricci-Cabello I, Avery AJ, Reeves D, Kadam UT, Valderas JM. Measuring Patient Safety in Primary Care: The development and validation of the “Patient Reported Experiences and Outcomes of Safety in Primary Care” planned for submission to: (PREOS-PC). Annals of Family Medicine

Ricci-Cabello I, Avery A, Valderas JM et al. Patients’ reports and evaluations of patient safety in English general practices: a cross sectional study. 

Bell B, et al The Development and Testing of the NIHR-SPCR Patient Safety Toolkit for General Practices in England

Bell B, Avery A, Marsden K et al The Patient Safety Toolkit Development 

Perryman K, Marsden K, Campbell S, Avery T Barriers and enablers for the adoption of patient safety tools to improve patient safety in UK primary care - 

Litchfield I, Greenfield S, Avery A, Perryman K, Marsden K, A qualitative exploration of the influences on the adoption of patient safety innovations in primary care 

Marsden K, Litchfield I, Doos L, Greenfield S An evaluation of two safety climate tools in primary health care: staff perspectives

Web reports published by the School for Primary Care

14/08/2014 Report Link: Tools for measuring patient safety in primary care settings using the RAND/UCLA appropriateness method

10/10/2014 Report Link: Measuring experiences and outcomes of patient safety in primary care: a systematic review of available instruments.

02/03/2015 Report Link: Trust, temporality and systems: how do patients understand patient safety in primary care? A qualitative study

01/09/2015 Report Link: Multimorbidity and Patient Safety Incidents in Primary Care: A Systematic Review and Meta-Analysis

25/05/2016 Report Link: Measuring Patient Safety in Primary Care: The Development and Validation of the “Patient Reported Experiences and Outcomes of Safety in Primary Care” (PREOS-PC)

17/06/2016 Report Link: Patients’ perceptions and experiences of patient safety in primary care in England

03/03/2017 Report Link: Identifying patient-centred recommendations for improving patient safety in General Practices in England: a qualitative content analysis of free-text responses using the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS-PC) questionnaire

15/06/2017 Report Link: Patients’ evaluations of patient safety in English general practices: a cross-sectional study

09/08/2017 Report Link: Identifying patient and practice characteristics associated with patient-reported experiences of safety problems and harm: a cross-sectional study using a multilevel modelling approach

23/02/2018 Report Link: A Patient Safety Toolkit for Family Practices

The NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre has been closely involved in this project and has contributed to funding a national roll-out (via a recently successful RCGP ‘Spotlight’ project). http://www.rcgp.org.uk/clinical-and-research/toolkits/patient-safety.aspx

Public involvement

Mr Antony Chuter, a patient with a long-term condition and experience of medication-related harm, led the public and patient involvement for this research project.

During the development of the PREOS-PC, several focus groups were undertaken with patients and this knowledge and guidance was used to enhance the questionnaire.

In addition, we have had GPs on our steering group to advise and support this project.

Impact

The toolkit has a number of tools that are individually useful in supporting general practices in their delivery of care to patients. Taken together this toolkit offers a powerful asset to improve patient safety in primary care.

At present there has been little impact of the Toolkit but the hosting of the toolkit on the RCGP website offers a huge potential for supporting practices in improving patient safety. As part of the RCGP Clinical Innovation and Research Centre’s (CIRC) efforts to engage with GPs across the UK on the Patient Safety Toolkit website, the Clinical Priorities Team is working with RCGP Faculties to hold four interactive workshops across the UK. They will be held in Bristol, Sheffield, Nottingham, and Glasgow in November.

The RCGP will be collating statistics on the use of the Patient Safety Toolkit website, which will be analysed on a regular basis.

The Patient Safety Collaboration in the East Midlands has supported the development of a database specifically for the patient questionnaire. At present it is in pdf form so that practices need to print off copies and either hand out or post them to their patients. The database will allow practices to email the questionnaire to patients. Completed returns will be emailed back only to the practice, which will then analyse the responses. However, overall anonymised results will be available on a local and national basis. This has the capacity to provide a unique assessment of patients’ attitudes to safety in general practice.

This project was funded by the National Institute for Health Research School for Primary Care Research (project number 113 )

Department of Health Disclaimer

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR School for Primary Care Research, NIHR, NHS or the Department of Health.

© NIHR 2024

• Contact us
• FAQs
• Rights and permissions
• Information for libraries
• Legal