Stefanie Maxion-Bergemann

Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity

BackgroundIt is often stated that external validity is not sufficiently considered in the assessment of clinical studies. Although tools for its evaluation have been established, there is a lack of awareness of their significance and application. In this article, a comprehensive checklist is presented addressing these relevant criteria.MethodsThe checklist was developed by listing the most commonly used assessment criteria for clinical studies. Additionally, specific lists for individual applications were included. The categories of biases of internal validity (selection, performance, attrition and detection bias) correspond to structural, treatment-related and observational differences between the test and control groups. Analogously, we have extended these categories to address external validity and model validity, regarding similarity between the study population/conditions and the general population/conditions related to structure, treatment and observation.ResultsA checklist is presented, in which the evaluation criteria concerning external validity and model validity are systemised and transformed into a questionnaire format.ConclusionThe checklist presented in this article can be applied to both planning and evaluating of clinical studies. We encourage the prospective user to modify the checklists according to the respective application and research question. The higher expenditure needed for the evaluation of clinical studies in systematic reviews is justified, particularly in the light of the influential nature of their conclusions on therapeutic decisions and the creation of clinical guidelines.

Use of complementary medicine in Switzerland

Within the framework of the Swiss governmental Program of Evaluation of Complementary Medicine (PEK) we assessed the prevalence, use, perceived effectiveness and appreciation of complementary medicine (CAM) in Switzerland, according to published surveys. Materials and Methods: Search was performed through electronic databases, by hand-searching and by contacting experts at universities, hospitals, health insurances, patient organizations and pharmaceutical companies. Results: Surveys were carried out among the general population (40%), physicians (20%), hospitalized patients (30%) and obstetric institutions (5%). The number of publications increased strongly between 1981 and 2004. The mean ± SD prevalence (use) of CAM is 49 ± 22% and varies depending on the survey’s topic and the population group interviewed. The acceptance, appreciation or demand for CAM among individuals specifically interviewed on CAM is 91 ± 6%. When asked about favored general improvements in healthcare, 6.5% of the individuals spontaneously mentioned CAM. CAM therapies are considered to be effective by the majority of CAM users and by about 40% of cancer patients using CAM. Approximately 50% of the population stated a preference for hospitals that also provide CAM. 85% of the population wishes the costs for CAM to be covered by the basic health insurance. Conclusion: Approximately half of the Swiss population has used CAM. CAM treatment is considered to be effective by the majority of CAM users. About 50% of the population would prefer hospitals that also provide CAM therapies and the majority of the population wishes the cost for CAM therapies to be covered by basic health insurance.

Complementary and alternative medicine costs - a systematic literature review

Objective: The aim of this literature review, performed within the framework of the Swiss governmental Program of Evaluation of Complementary Medicine (PEK), was to investigate costs of complementary and alternative medicine (CAM). Materials and Methods: A systematic literature search was conducted in 11 electronic databases. All retrieved titles and reference lists were also hand-searched. Results: 38 publications were found: 23 on CAM of various definitions (medical and non-medical practitioners, over-the-counter products), 13 on homeopathy, 2 on phytotherapy. Studies investigated different kinds of costs (direct or indirect) and used different methods (prospective or retrospective questionnaires, data analyses, cost-effectiveness models). Most studies report ‘out of pocket’ costs, because CAM is usually not covered by health insurance. Costs per CAM-treatment / patient / month were AUD 7-66, CAD 250 and GBP 13.62 ± 1.61. Costs per treatment were EUR 205 (range: 15-1,278), USD 414 ± 269 and USD 1,127. In two analyses phytotherapy proved to be cost-effective. One study revealed a reduction of 1.5 days of absenteeism from work in the CAM group compared to conventionally treated patients. Another study, performed by a health insurance company reported a slight increase in direct costs for CAM. Costs for CAM covered by insurance companies amounted to approximately 0.2-0.5% of the total healthcare budget (Switzerland, 2003). Publications had several limitations, e.g. efficacy of therapies was rarely reported. As compared to conventional patients, CAM patients tend to cause lower costs. Conclusion: Results suggest lower costs for CAM than for conventional patients, but the limited methodological quality lowers the significance of the available data. Further well-designed studies and models are required.

Traditional Chinese Medicine (Phytotherapy): Health Technology Assessment Report – Selected Aspects

Objective: A summary of main aspects from a Health Technology Assessment report on Traditional Chinese Medicine (TCM) in Switzerland concerning effectiveness and safety is given. Materials and Methods: Literature search was performed through 13 databases, by scanning reference lists of articles and by contacting experts. Assessed were quality of documentation, internal and external validity. Results: Effectiveness: 43 articles concerning ‘gastrointestinal tract and liver’ were assessed. The studies covering 7,436 patients were undertaken in China (35), Japan (3), USA (2) and Australia (3); 33/43 being controlled studies. 34/40 show significantly better results in the TCM-treated group. A comparison of studies on results of treatment based on a diagnosis according to TCM criteria and studies on results of treatment according to Western diagnosis shows that treatment based on TCM diagnosis improves the result. The comparison of treatment by individual medication and standard medication showed a trend in favor of individual medication. Safety: TCM training and practice for physicians in Switzerland are officially regulated. Side effects occur, but no severe effects have been registered up to now in Switzerland. TCM medicinals are imported; admission regulations are being installed. Problems due to production abroad, Internet trade, self-medication or admixtures are possible. Conclusion: The evaluation of the literature search provides evidence for a basic clinical effectiveness of TCM therapy. Severe side effects were not observed in Switzerland. Regulations for trading and use of medicinals prevent treatment risks. Further clinical studies in a Western context are required.

Familiarity, objectivity – and misconduct: Counterstatement to Shaw DM. The Swiss Report on homoeopathy: a case study of research misconduct

David M Shaw recently published a critique of our Health Technology Assessment (HTA) on homoeopathy in this journal [1], which calls for a response. We expected a scientific debate to ensue on our chosen research methods and were surprised to be confronted with a biased mixture of false or incomplete citations, wrong matches and, as a consequence, incorrect conclusions that reveal a lack of familiarity and experience with both homoeopathy and the Swiss setting. We will refute the allegations and rectify misunderstandings, in the order of appearance in Shaw’s text. Because of numerous repetitions in Shaw’s text we have subsumed some of the allegations thematically:

International Utilisation of Complementary Medical Approaches

What kinds of individuals or groups are using CAM (complementary alternative medicine) for which reasons, and how often? These questions are of much interest to health authorities, service providers, service guarantors, the economy and the public in general. This chapter describes studies published on these topics in Europe and beyond. Studies from Switzerland are examined in more detail in Chapter 8.

HTA Homeopathy: Methods and Material

The compilation of this HTA followed predefined steps to ensure the quality of process and results. The procedure was based on the commission documents provided and the generally accepted international guidelines for the compilation of HTA reports, as well as on the special requirements set by the ‘Complementary Medicine Evaluation Programme’ (PEK) in Switzerland (ECHTA 2001, BSV 2001, DIMDI 2004, INHTA 2001, Heusser 2001).

Cost-effectiveness of Homeopathy

Surveys conducted in the Western world all show that health-care costs tend to rise faster than general living costs. In the USA they rose from US

Background and objectives of HTA

141 per person per year in 1960 through US

Counterstatement to a FACT Perspective article: ‘A critique of the Swiss report Homeopathy in Healthcare’

341 in 1970, US