Patient experience is a key feature on the Government healthcare agenda. Patient Reported Experience Measures (PREMs) can provide important evidence regarding clinical care from the patient perspective. In conjunction with organisational and clinical audit data, they provide a rounded view of a patient’s care experience and support the development of quality improvements.
As part of the National COPD Audit Programme, one year of development work for the future incorporation of PREMs into the audit programme was commissioned; to inform decisions regarding future roll-out of this element and associated funding. Delivery of a pilot study was managed by the British Lung Foundation, working with the Picker Institute Europe. It considered a range of approaches to measuring people’s experiences of COPD in three key settings: primary care, secondary care, and pulmonary rehabilitation (PR).
The project focussed on testing methods and assessing the feasibility of collecting PREMs. In the absence of any existing validated COPD specific PREM tools, the project aimed to use a generic tool, or tools, for piloting. A literature review identified a number of potential generic questionnaires and these were tested with patients via focus groups to ensure their broad suitability and to assess the extent to which tailoring across and within settings would be necessary.
The PREM pilot employed three methodologies:
- Method one – paper survey, sent via post
- Method two – paper survey, handed out at the point of care
- Method three – online survey, with login details distributed via postcard at the point of care.
Each methodology was tested in the three settings – primary care, secondary care and pulmonary rehabilitation (a three-by-three nested design). The characteristics of each method were assessed against three criteria: their practical feasibility, the level of data provided, and the likely costs and cost efficiencies for a national rollout. Collectively these factors provide a rounded overview of the feasibility of adding PREMs into the National COPD Audit Programme.
Developing and rolling out the pilot highlighted a range of challenges and barriers associated with using a PREM tool in support of a COPD audit – some were setting or method specific but the majority were applicable across all three settings. Proposed data collection tool(s) and methodology will need to address these if the benefits of collecting COPD PREMs are to be realised.
Of the three methods tested, postal surveys were the best option for standard use. These had the highest response rates in all three care settings and gave good value for money.
Based on the pilot results, a number of initial recommendations were made:
- In the absence of a validated COPD specific PREM questionnaire, use of an existing generic tool, with the addition of disease-specific questions on issues of particular importance to COPD patients, provided a basis for measuring people’s experiences of COPD care in a range of settings
- A national survey in primary care will only be feasible if supported by necessary Information Governance (section 251) approval to enable central administration of questionnaires and reduce burden at practice level. A fully centralised approach would require section 251 approvals in all settings to allow for patient identifiable data to be shared for the purposes of administration.
- An online survey approach is not recommended at this time for a survey of people’s experiences of COPD, given a) evidence of extremely low response rates and b) the lack of support for such an approach from patients and providers alike.
- A postal survey methodology is recommended for use in the national audit. It produced reasonable response rates and is easier to standardise in the event of a wider rollout across the audit programme.
Conclusion: A centralised postal survey, with section 251 support, for each setting would be the approach most likely to be successful for national use.
Having identified a recommended data collection methodology through the feasibility pilot study, further practicalities require detailed consideration if the recommended PREM methodology is to be successfully integrated into each workstream of the audit programme in the future.
These include: governance, staffing, delivery - external Provider vs ‘In-house’, PREM tool/questionnaire, technology, data flows and linkage, ethical / information governance requirements, timing – alignment of audits and PREM questionnaires, communications, including recruitment strategy, data analysis and reporting, risk management.
The National COPD Audit Programme PREMS feasibility report was submitted to Healthcare Quality Improvement Partnership (HQIP) in August 2014 and was formally evaluated by HQIP and members of the National Advisory Group for Clinical Audits and Enquiries (NAGCAE) Renewals Sub-Group on 6 May 2015.
The purpose of the meeting was to feedback on the quality of the report and in particular to determine whether national roll-out of PREMs should be invited as part of the proposed extension of the National COPD Audit Programme for a fourth and fifth year.
The audit team received positive feedback with regards to the quality of the report and work undertaken.
Following discussions and clarifications with the national COPD audit team at the evaluation meeting, it was, however, agreed that there were a number of reasons that commissioning of a national roll-out of PREMs as part of the National COPD Audit Programme could not be supported at this time.
- New commissioning of national PROMs and PREMs across the National Clinical Audit and Patient Outcome Programme (NCAPOP) is on hold pending a national strategic review.
- The potential for response bias between responders and non-responders and between responders via different collection methods needs further exploration.
- The issues identified in the report itself including primary care engagement, patient identification, and the cost of a centralised, postal-based delivery model would need resolving.
- The establishment of COPD audit data flow from primary care and linkage of audit records across primary, secondary and pulmonary rehab has been slow to establish due to national Information Governance issues and the lengthy and complex processes now required to transfer and link data. There is value in establishing this successfully before PROMs/PREMs are rolled out nationally, if their value is to be fully realised.
Therefore, national roll-out of PREMs as part of the National COPD Audit Programme is not recommended for inclusion in the proposed scope of work for the audit programme 2 year contract extension. However, the panel recognised and supported the very valuable work ongoing around improving the ways in which patient-reported data could be used to improve the quality of care.