Liesl Osman

Lessons from a failure: generating tailored smoking cessation letters

STOP is a Natural Language Generation (NLG) system that generates short tailored smoking cessation letters, based on responses to a four-page smoking questionnaire. A clinical trial with 2553 smokers showed that STOP was not effective; that is, recipients of a non-tailored letter were as likely to stop smoking as recipients of a tailored letter. In this paper we describe the STOP system and clinical trial. Although it is rare for AI papers to present negative results, we believe that useful lessons can be learned from STOP. We also believe that the AI community as a whole could benefit from considering the issue of how, when, and why negative results should be reported; certainly a major difference between AI and more established fields such as medicine is that very few AI papers report negative results.

Cost effectiveness of computer tailored and non-tailored smoking cessation letters in general practice: randomised controlled trial

Abstract Objectives: To develop and evaluate, in a primary care setting, a computerised system for generating tailored letters about smoking cessation. Design: Randomised controlled trial. Setting: Six general practices in Aberdeen, Scotland. Participants: 2553 smokers aged 17 to 65. Interventions: All participants received a questionnaire asking about their smoking. Participants subsequently received either a computer tailored or a non-tailored, standard letter on smoking cessation, or no letter. Main outcome measures: Prevalence of validated abstinence at six months; change in intention to stop smoking in the next six months. Results: The validated cessation rate at six months was 3.5% (30/857) (95% confidence interval 2.3% to 4.7%) for the tailored letter group, 4.4% (37/846) (3.0% to 5.8%) for the non-tailored letter group, and 2.6% (22/850) (1.5% to 3.7%) for the control (no letter) group. After adjustment for significant covariates, the cessation rate was 66% greater (−4% to 186%; P=0.07) in the non-tailored letter group than that in the no letter group. Among participants who smoked <20 cigarettes per day, the cessation rate in the non-tailored letter group was 87% greater (0% to 246%; P=0.05) than that in the no letter group. Among heavy smokers who did not quit, a 76% higher rate of positive shift in “stage of change” (intention to quit within a particular period of time) was seen compared with those who received no letter (11% to 180%; P=0.02). The increase in cost for each additional quitter in the non-tailored letter group compared with the no letter group was £89. Conclusions: In a large general practice, a brief non-tailored letter effectively increased cessation rates among smokers. A tailored letter was not effective in increasing cessation rates but promoted shift in movement towards cessation (“stage of change”) in heavy smokers. As a pragmatic tool to encourage cessation of smoking, a mass mailing of non-tailored letters from general practices is more cost effective than computer tailored letters or no letters. What is already known on this topic Brief opportunistic advice on stopping smoking that is given face to face by health professionals increases rates of cessation by 2-3% Intensive, expert-led interventions increase cessation rates by up to 20% or more but are expensive and reach only a small proportion of smokers Written advice tailored to an individuals “stage of change” (intention to stop in a particular period of time) has been claimed to be as effective as intensive interventions, but previous studies of tailored written advice did not biochemically validate cessation What this paper adds A simple standard letter sent to patients of general practices that gave brief advice on stopping smoking increased the biochemically validated rate of cessation by 2% A letter tailored to the individuals “stage of change” was not more effective than the non-tailored standard letter Although the increase in cessation resulting from the non-tailored standard letter was small, this intervention was highly cost effective

Symptom-based outcome measures for asthma: the use of discrete choice methods to assess patient preferences.

This paper reports on an application of discrete choice modelling to the measurement of patient preferences over asthma symptoms. A sample of patients with moderate to severe asthma was asked to choose between a series of pairs of scenarios characterised by different combinations of asthma symptoms. Their responses were analysed using a random effects ordered probit model. The results implied that patients weighted some symptoms more highly than others. Discrete choice modelling proved to be a useful approach for developing preference based outcome measures, although the results show how, in contexts where preferences over health care outcomes based on symptoms or some measure of health status are involved, a conventional linear additive model may not always be suitable.

The patient perspective. What should a new anti-asthma agent provide?

SummaryPatients want medication that they feel confident using and that will control symptoms and be well tolerated. Patients may dislike and use ineffectively some asthma delivery devices such as inhalers, and they may have anxieties about adverse effects of inhaled corticosteroid medication. Oral medication may offer the advantage of improved patient compliance and, in 2 studies, patients have stated a preference for oral medication. However, the introduction of any new medication is likely to arouse anxieties and expectations in patients, and it is important that they be given clear information on medication use and that time be spent discussing any concerns about change. In the general practice consultation, patients are not always able to raise all the issues that are important to them in their asthma management. Asthma clinics staffed by nurses have more time and the opportunity to deal with patient concerns and teach good medication practice. Pharmacists may also have a role to play in patient counselling and giving advice on changes in medication.

Management of asthma in patients supervised by primary care physicians or by specialists.

French asthma patients may be supervised by general practitioners (GPs) and/or specialists. Therefore, this study examined asthma management in patients exclusively supervised by specialists (SPE), GPs, (GP) and both (GP+SPE group), and compared the findings. Asthma patients were consecutively recruited in 348 pharmacies. Each patient completed a questionnaire providing data on personal characteristics, asthma management, perception of disease and asthma supervision. Asthma control was measured using the Asthma Control Test. Questionnaires were linked to computerised records of medications which had been dispensed before inclusion in the study. From the 1,256 patients (mean age = 36.1 yrs, 54.3% females), 11.4, 36.6, and 52.0% were placed in the SPE, GP, and GP+SPE groups, respectively. During the previous 4 weeks, most patients in the SPE group were properly controlled (52.2 versus 26.4 and 21.5% in GP and GP+SPE groups, respectively). The SPE group made more use of fixed combinations of long-acting beta agonist and inhaled corticosteroid, while receiving less short-acting beta agonists, antitussives and antibiotics. Striking differences in symptoms and asthma management were observed according to the type of asthma supervision. The current results strongly support the need to improve the management of asthma in primary care, and the coordination of care between general practitioners and specialists.

Knowledge Acquisition for Natural Language Generation

We describe the knowledge acquisition (KA) techniques used to build the STOP system, especially sorting and think-aloud protocols. That is, we describe the ways in which we interacted with domain experts to determine appropriate user categories, schemas, detailed content rules, and so forth for STOP. Informal evaluations of these techniques suggest that they had some benefit, but perhaps were most successful as a source of insight and hypotheses, and should ideally have been supplemented by other techniques when deciding on the specific rules and knowledge incorporated into STOP.

Types of Knowledge Required to Personalise Smoking Cessation Letters

The STOP system generates personalised smoking-cessation letters, using as input responses to a smoking questionnaire. Generating personalised patient-information material is an area of growing interest to the medical community, since for many people changing health-related behaviour is the most effective possible medical intervention. While previous AI systems that generated personalised patient-information material were primarily based on medical knowledge, stop is largely based on knowledge of psychology, empathy, and readability. We believe such knowledge is essential in systems whose goal is to change peoples behaviour or mental state; but there are many open questions about how this knowledge should be acquired, represented, and reasoned with.

A randomised trial of home energy efficiency improvement in the homes of elderly COPD patients

A randomised trial of 178 patients in Aberdeen, UK with a previous hospital admission for chronic obstructive pulmonary disease (COPD) was carried out in order to determine whether improving home energy efficiency improves health-related quality of life in COPD patients. 118 patients were randomised and 60 agreed to monitoring only. Energy efficiency upgrading was carried out in 42% of homes randomised to intervention. Independent energy efficiency action was taken by 15% of control participants and 18% in the monitoring group. The main outcome measures were respiratory and general health status, home energy efficiency and hospital admissions. Intention-to-treat analysis found no difference in outcomes between the two groups. In 45 patients, who had energy efficiency action independent of original randomisation, there were significant improvements in respiratory symptom scores (adjusted mean 9.0, 95% CI 2.5–15.5), decreases in estimated annual fuel costs (-£65.3, 95% CI -£31.9– -£98.7) and improved home energy efficiency rating (1.1, 95% CI 0–1.4). COPD patients are unlikely to take up home energy efficiency upgrading, if offered. Secondary “pragmatic” analysis suggests that those who do take action may achieve clinically significant improvement in respiratory health, which is not associated with an increase in indoor warmth.

Annual review of patients with sleep apnea/hypopnea syndrome--a pragmatic randomised trial of nurse home visit versus consultant clinic review.

BACKGROUND This pragmatic randomised, controlled trial investigated annual review of patients with sleep apnea/hypopnea syndrome (SAHS). Clinical outcomes and costs were compared for consultant clinic review versus specialist nurse home visit. METHOD One hundred and seventy-four patients were randomised to annual review by consultant clinic appointment or by specialist nurse home visit. SAHS symptoms, Epworth score, hospital anxiety and depression scale (HADS), Short Form-36 (SF-36) and hours of use of constant positive airway pressure (CPAP) were measured before and 3 months after review. The costs and patient preference for review were determined. RESULTS After review, both groups significantly increased CPAP use (mean (SD) increase: nurse, 0.66 (1.71) h; consultant, 0.45 (1.69) h) and reduced symptom scores (nurse, -2 (7); consultant, -3 (9)), compared to baseline. There were no differences between groups in these improvements, or in HADS or SF-36 scores. Average duration of a nurse home visit, excluding travel time, was 26 (6) min. Total NHS cost per visit was 52.26 UK pounds (49.85) (

Using a Randomised Controlled Clinical Trial to Evaluate an NLG System

83.62 (79.76)), of which 6.57 UK pounds (1.43) (