Report

Do the findings of case series studies vary significantly according to methodological characteristics?

Authors: Dalziel K, Round A, Stein K, Garside R, Castelnuovo E, Payne L

Journal: Health Technology Assessment Volume: 9 Issue: 2

Publication date: January 2005

DOI: 10.3310/hta9020

Citation:

Dalziel K, Round A, Stein K, Garside R, Castelnuovo E, Payne L.Do the findings of case series studies vary significantly according to methodological characteristics?. Health Technol Assess 2005;9(2)


Journal issues* can be purchased by completing the form.


The cost of reports varies according to number of pages and postage address. The minimum cost for a copy sent to a UK address is £30.00. We will contact you on receipt of your completed form to advise you of actual cost. If you have any queries, please contact nihredit@southampton.ac.uk.


*We regret that unfortunately we are unable to supply bound print copies of Health Technology Assessment published before issue 12:31. However, PDFs are available to print from the "Downloads" tab of the issue page.

Responses

No responses have been published. If you would like to submit a response to this publication, please do so using the form below.

Comments submitted to the NIHR Journals Library are electronic letters to the editor. They enable our readers to debate issues raised in research reports published in the Journals Library. We aim to post within 2 working days all responses that contribute substantially to the topic investigated, as determined by the Editors.

Your name and affiliations will be published with your comment.

Once published, you will not have the right to remove or edit your response. The Editors may add, remove, or edit comments at their absolute discretion.

Post your response

Surname

Forename

Middle Initial

Occupation / Job title

Affiliation / Employer

Email

Address

Other authors

For example, if you are responding as a team or group. Please ensure you include full names and separate these using commas

Statement of competing interests

We believe that readers should be aware of any competing interests (conflicts of interest).

The International Committee of Medical Journal Editors (ICMJE) define competing interests as including: financial relationships with industry (for example through employment, consultancies, stock, ownership, honoraria, and expert testimony), either directly or through immediate family; personal relationships; academic competition; and intellectual passion.

If yes, provide details below:

Enter response title

Enter response message

Enter CAPTCHA

Security key

Regenerate security key

By submitting your response, you are stating that you agree to the terms & conditions

  • Abstract

Abstract

Objectives

To review the use of case series in National Institute for Clinical Excellence (NICE) Health Technology Assessment (HTA) reports, to review systematically the methodological literature for papers relating to the validity of aspects of case series design, and to investigate characteristics and findings of case series using examples from the UK's Health Technology Assessment programme.

Data sources

Electronic databases. NICE website. Reports produced as part of the UK's HTA programme.

Review methods

NICE HTAs that used information from case series studies were obtained from the NICE website and a range of quality criteria applied. Searches of electronic databases, handsearched journals and the bibliographies of papers were made in order to find studies that assessed aspects of case series design, analysis or quality in relation to study validity. Hypotheses relating to the design of case series studies were developed and empirically investigated using four case examples from existing reports produced as part of the UK's HTA programme (functional endoscopic sinus surgery for nasal polyps, spinal cord stimulation for chronic back pain, percutaneous transluminal coronary angioplasty and coronary artery bypass grafting for chronic angina). Analysis was undertaken comparing studies within each review.

Results

There was no consensus on which case series to include in HTAs, how to use them or how to assess their quality, despite them being used in 30% of NICE HTAs. No previous studies empirically investigating methodological characteristics of case series were found. However, it is possible that the search strategy failed to find relevant studies. Poor reporting of case series characteristics severely constrained analysis and there were insufficient data to investigate all the hypotheses. Findings were not consistent across the different topics and were subject to considerable uncertainty. All the examples in our analysis were surgical interventions, which are prone to additional confounding factors due to difficulties of standardisation compared with drug treatment. Our findings may not be generalisable outside the interventions studied. The case series reports included generally exhibited poor reporting of methodological characteristics. This constrained our analysis. The use of several methods of analysis has led to apparently discrepant results. Given the number of analysis performed, the usual level of significance (p = 0.05) should be viewed with caution. The most important limitation of this study is the small number of cases on which the findings are based. The results are therefore tentative and should be viewed with caution.

Conclusions

Case series are incorporated in a significant proportion of health technology assessments. Quality criteria have been used to appraise the quality of case series and decide on their inclusion in reviews of studies using this design. In this small series of case studies drawn from HTAs carried out for the NHS HTA programme, little evidence was found to support the use of many of the factors included in quality assessment tools. Importantly, no relationship was found between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, such as blinding of outcome measurement, but these were not shown consistently across the small number of examples studied. This study is based on a very small sample of studies and should therefore be considered as exploratory. Further investigation of the relationship between methodological features and outcome is justified given the frequency of use of case series in health technology assessments. Further research into the methodological features of case series and their outcome is justified in a wider sample of technologies and larger sets of case series. Value of information analyses including case series could be explored. Further exploration of the differences between case series and randomised controlled trial results, preferably using registry or comprehensive case series data, would be valuable.

Publication updates

If you would like to receive information on publications and the latest news, click below to sign up.