Journals Library

An error occurred retrieving content to display, please try again.

Page not found (404)

Sorry - the page you requested could not be found.

Please choose a page from the navigation or try a website search above to find the information you need.

{{metadata.Title}}

Study proposed changes to the National Institute for Health and Clinical Excellence website to enable easier access to relevant information, suggested guidance for more appropriate manufacturer submissions and provided recommendations for guiding Evidence Review Group teams and improving reporting methods

{{author}}{{author}}{{($index < metadata.AuthorsAndEtalArray.length-1) ? ',' : '.'}}

E Kaltenthaler 1,*, A Boland 2, C Carroll 1, R Dickson 2, P Fitzgerald 1, D Papaioannou 1

1 School of Health and Related Research (ScHARR) Technology Assessment Group, University of Sheffield, Sheffield, UK
2 Liverpool Reviews and Implementation Group (LRiG), University of Liverpool, Liverpool, UK
* Corresponding author Email: e.kaltenthaler@sheffield.ac.uk

Funding: {{metadata.Funding}}

{{metadata.Journal}} Volume: {{metadata.Volume}}, Issue: {{metadata.Issue}}, Published in {{metadata.PublicationDate | date:'MMMM yyyy'}}

https://doi.org/{{metadata.DOI}}

Citation: {{author}}{{ (($index < metadata.AuthorsArray.length-1) && ($index <=6)) ? ', ' : '' }}{{(metadata.AuthorsArray.length <= 6) ? '.' : '' }} {{(metadata.AuthorsArray.length > 6) ? 'et al.' : ''}} {{metadata.Title}}. {{metadata.JournalShortName}} {{metadata.PublicationDate | date:'yyyy'}};{{metadata.Volume}}({{metadata.Issue}})

Crossmark status check

Report Content

The full text of this issue is available as a PDF document from the Toolkit section on this page.

The full text of this issue is available as a PDF document from the Toolkit section on this page.

Abstract

BACKGROUND

The National Institute for Health and Clinical Excellence (NICE) single technology appraisal (STA) process was set up as a rapid way to appraise new technologies for use within the NHS in England and Wales and has been in place since 2005.

OBJECTIVES

This study had five primary objectives: (1) to provide a map of the STA process to date; (2) to identify current approaches to the critical appraisal of manufacturers' submissions (MSs) by Evidence Review Groups (ERGs); (3) to identify recurring themes in clarification letters sent to manufacturers; (4) to provide recommendations for possible alternative approaches to be used in the critical appraisal process; and (5) to revise the current ERG report template.

DATA SOURCES

Data for the mapping of the STA process were obtained from the NICE website (www.nice.org.uk). Data for the analyses of the ERG reports and clarification letters were taken from the reports and letters themselves.

REVIEW METHODS

For the mapping, a spreadsheet was developed to collect data on 22 predefined variables related to timings and outcomes. Simple descriptive statistics were used to analyse the data. For the thematic analysis, a documentary analysis of 30 ERG reports was undertaken. Data on key elements of the MSs, the processes undertaken by ERGs and the strengths and weaknesses of MSs were extracted. A framework of a priori themes was developed. Data were extracted, coded and analysed according to a framework approach. Twenty-one clarification letters were examined and data were extracted using a set of codes to cover report quality, systematic review methods and clinical/economic issues. The current ERG report template was modified and sent to the current ERG teams for comment. All comments were considered and formed the basis for further revisions to the template.

RESULTS

Ninety-five STAs were included in the mapping exercise. Many STAs were subject to delay or cancellation for a variety of reasons. The ERG reports highlighted the strengths and weaknesses of MSs to the STA process. Thematic analysis of these data offered a means of clarifying and describing these aspects of the submissions. This analysis generated five themes: process, reporting, satisfaction of objectives, reliability and validity of findings, and content. Points from clarification letters were analysed and presented in four main categories: report quality, systematic review methods, clinical data analysis and economic data analysis.

LIMITATIONS

Nearly all data were obtained from the NICE website; therefore, any errors in the data on the website will be reflected in the mapping analysis presented in this report. Missing data for the mapping exercise limit the generalisability of the findings. Analyses were limited to what was reported in the ERG reports and the clarification letters.

CONCLUSIONS

Guidance suggested for manufacturers will help to ensure that more appropriate submissions are received in the future while recommendations provided for ERG teams will help to guide teams to ensure that reporting methods are transparent.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Abstract

BACKGROUND

The National Institute for Health and Clinical Excellence (NICE) single technology appraisal (STA) process was set up as a rapid way to appraise new technologies for use within the NHS in England and Wales and has been in place since 2005.

OBJECTIVES

This study had five primary objectives: (1) to provide a map of the STA process to date; (2) to identify current approaches to the critical appraisal of manufacturers' submissions (MSs) by Evidence Review Groups (ERGs); (3) to identify recurring themes in clarification letters sent to manufacturers; (4) to provide recommendations for possible alternative approaches to be used in the critical appraisal process; and (5) to revise the current ERG report template.

DATA SOURCES

Data for the mapping of the STA process were obtained from the NICE website (www.nice.org.uk). Data for the analyses of the ERG reports and clarification letters were taken from the reports and letters themselves.

REVIEW METHODS

For the mapping, a spreadsheet was developed to collect data on 22 predefined variables related to timings and outcomes. Simple descriptive statistics were used to analyse the data. For the thematic analysis, a documentary analysis of 30 ERG reports was undertaken. Data on key elements of the MSs, the processes undertaken by ERGs and the strengths and weaknesses of MSs were extracted. A framework of a priori themes was developed. Data were extracted, coded and analysed according to a framework approach. Twenty-one clarification letters were examined and data were extracted using a set of codes to cover report quality, systematic review methods and clinical/economic issues. The current ERG report template was modified and sent to the current ERG teams for comment. All comments were considered and formed the basis for further revisions to the template.

RESULTS

Ninety-five STAs were included in the mapping exercise. Many STAs were subject to delay or cancellation for a variety of reasons. The ERG reports highlighted the strengths and weaknesses of MSs to the STA process. Thematic analysis of these data offered a means of clarifying and describing these aspects of the submissions. This analysis generated five themes: process, reporting, satisfaction of objectives, reliability and validity of findings, and content. Points from clarification letters were analysed and presented in four main categories: report quality, systematic review methods, clinical data analysis and economic data analysis.

LIMITATIONS

Nearly all data were obtained from the NICE website; therefore, any errors in the data on the website will be reflected in the mapping analysis presented in this report. Missing data for the mapping exercise limit the generalisability of the findings. Analyses were limited to what was reported in the ERG reports and the clarification letters.

CONCLUSIONS

Guidance suggested for manufacturers will help to ensure that more appropriate submissions are received in the future while recommendations provided for ERG teams will help to guide teams to ensure that reporting methods are transparent.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

If you would like to receive a notification when this project publishes in the NIHR Journals Library, please submit your email address below.

 

Responses to this report

No responses have been published.

 

If you would like to submit a response to this publication, please do so using the form below:

Comments submitted to the NIHR Journals Library are electronic letters to the editor. They enable our readers to debate issues raised in research reports published in the Journals Library. We aim to post within 2 working days all responses that contribute substantially to the topic investigated, as determined by the Editors.

Your name and affiliations will be published with your comment.

Once published, you will not have the right to remove or edit your response. The Editors may add, remove, or edit comments at their absolute discretion.

By submitting your response, you are stating that you agree to the terms & conditions