Gerarda Derksen-Lubsen

External validation is necessary in prediction research: A clinical example

BACKGROUND AND OBJECTIVES Prediction models tend to perform better on data on which the model was constructed than on new data. This difference in performance is an indication of the optimism in the apparent performance in the derivation set. For internal model validation, bootstrapping methods are recommended to provide bias-corrected estimates of model performance. Results are often accepted without sufficient regard to the importance of external validation. This report illustrates the limitations of internal validation to determine generalizability of a diagnostic prediction model to future settings. METHODS A prediction model for the presence of serious bacterial infections in children with fever without source was derived and validated internally using bootstrap resampling techniques. Subsequently, the model was validated externally. RESULTS In the derivation set (n=376), nine predictors were identified. The apparent area under the receiver operating characteristic curve (95% confidence interval) of the model was 0.83 (0.78-0.87) and 0.76 (0.67-0.85) after bootstrap correction. In the validation set (n=179) the performance was 0.57 (0.47-0.67). CONCLUSION For relatively small data sets, internal validation of prediction models by bootstrap techniques may not be sufficient and indicative for the models performance in future patients. External validation is essential before implementing prediction models in clinical practice.

Risk factors for seizure recurrence in children with febrile seizures: a pooled analysis of individual patient data from five studies.

To reassess the relations between postulated risk factors and seizure recurrence after a first febrile seizure (FS), the individual data from five follow-up studies that used similar definitions of FSs and risk factors were pooled and reanalyzed. The risk of frequent recurrent seizures and of the occurrence of complex seizures in previously healthy, untreated children was studied. Seizure recurrence hazard was described as a function of the childs attained age. The influence of various risk factors on the recurrence hazard was assessed, with control for other factors. Of a total of 2496 children with 1410 episodes of recurrent seizures, 32% had one, 15% had two, and 7% had three or more recurrent seizures after a first FS; 7% had a complex seizure. The hazard of recurrent seizures was highest between the ages of 12 and 24 months. After a first and a second recurrence, the risk of further FSs was two and two and one-half times higher, respectively. A history of febrile or unprovoked seizures in a first-degree family member and a relatively low temperature at the time of the first seizure were also associated with an increased risk of subsequent recurrences. Young age at onset (< 12 months), a family history of unprovoked seizures, and a partial initial FS were all associated with an increased risk of complex seizures. A higher recurrence rate in clinic-based studies compared with population-based studies could not be explained by a difference in the presence of the risk factors studied. Thus other factors must influence seizure recurrence after an initial FS.

Informed consent, parental awareness, and reasons for participating in a randomised controlled study

BACKGROUND The informed consent procedure plays a central role in randomised controlled trials but has only been explored in a few studies on children. AIM To assess the quality of the informed consent process in a paediatric setting. METHODS A questionnaire was sent to parents who volunteered their child (230 children) for a randomised, double blind, placebo controlled trial of ibuprofen syrup to prevent recurrent febrile seizures. RESULTS 181 (79%) parents responded. On average, 73% of parents were aware of the major study characteristics. A few had difficulty understanding the information provided. Major factors in parents granting approval were the contribution to clinical science (51%) and benefit to the child (32%). Sociodemographic status did not influence initial participation but west European origin of the father was associated with willingness to participate in future trials. 89% of participants felt positive about the informed consent procedure; however, 25% stated that they felt obliged to participate. Although their reasons for granting approval and their evaluation of the informed consent procedure did not differ, relatively more were hesitant about participating in future. Parents appreciated the investigator being on call 24 hours a day (38%) and the extra medical care and information provided (37%) as advantages of participation. Disadvantages were mainly the time consuming aspects and the work involved (23%). CONCLUSIONS Parents’ understanding of trial characteristics might be improved by designing less difficult informed consent forms and by the investigator giving extra attention and information to non-west European parents. Adequate measures should be taken to avoid parents feeling obliged to participate, rather than giving true informed consent.

Randomized, controlled trial of ibuprofen syrup administered during febrile illnesses to prevent febrile seizure recurrences

Objectives. Febrile seizures recur frequently. Factors increasing the risk of febrile seizure recurrence include young age at onset, family history of febrile seizures, previous recurrent febrile seizures, time lapse since previous seizure <6 months, relative low temperature at the initial seizure, multiple type initial seizure, and frequent febrile illnesses. Prevention of seizure recurrences serves two useful purposes: meeting parental fear of recurrent febrile seizures in general and reducing the (small) risk of a long-lasting and eventually injurious recurrent seizure. In daily practice, children with febrile seizures often are treated with antipyretics during fever to prevent febrile seizure recurrences. Thus far, no randomized placebo-controlled trial has been performed to assess the efficacy of intermittent antipyretic treatment in the prevention of seizure recurrence. Methods. We performed a randomized, double-blind, placebo-controlled trial. Children 1 to 4 years of age who had had at least one risk factor for febrile seizure recurrence were enrolled. They were randomly assigned to either ibuprofen syrup, 20 mg/mL, 0.25 mL (= 5 mg) per kilogram of body weight per dose, or matching placebo, to be administered every 6 hours during fever (temperature, ≥38.5°C). Parents were instructed to take the childs rectal temperature immediately when the child seemed ill or feverish and to promptly administer the study medication when the temperature was ≥38.5°C. Doses were to be administered every 6 hours until the child was afebrile for 24 hours. The parents were instructed not to administer any other antipyretic drug to the child. For measuring rectal temperature, a Philips HP5316 digital thermometer (Philips, Eindhoven, The Netherlands) was distributed. During subsequent treatment of the fever episode, parents had to call the investigator at least once each day to notify the investigator in case of febrile seizure recurrence. The investigator could be contacted by parents 24 hours per day. The primary outcome was the first recurrence of a febrile seizure. Kaplan–Meier curves and Cox regression were used for the statistical analysis. The treatment effect on the course of the temperature was assessed using analysis of covariance, with temperature at fever onset as covariate. Two analyses were performed. In an intention-to-treat analysis, all first recurrences were considered regardless of study medication compliance. A per-protocol analysis was limited to those recurrences that occurred in the context of study medication compliance. Results. Between October 1, 1994, and April 1, 1996, 230 children were randomly assigned to ibuprofen syrup (111 children) or placebo (119 children). Median follow-up time was 1.04 years (25th–75th percentiles; 0.7–1.8 years) in the ibuprofen group and 0.98 years (0.7–1.6 years) in the placebo group. Of all children, 67 had a first febrile seizure recurrence, with 31 in the ibuprofen group and 36 in the placebo group. The 2-year recurrence probabilities were 32% and 39%, respectively. The recurrence risk in the ibuprofen group was 0.9 (95% confidence interval: 0.6–1.5) times the recurrence risk in the placebo group (intention to treat). Adjustment for baseline characteristics did not affect the risk-reduction estimate. Of the 67 recurrences, 30 occurred in the context of study medication compliance (13 ibuprofen, 17 placebo). The per-protocol analysis, which was limited to these events, showed similar results. A significant reduction in temperature (0.7°C) after fever onset in the ibuprofen group compared with the placebo group was demonstrated if all 555 fever episodes were considered. In the fever episodes with a seizure recurrence, a similar temperature increase was shown in both groups, with no significant difference between the intention-to-treat and the per-protocol analysis. Discussion. The present study failed to demonstrate a preventive effect of intermittent antipyretic treatment during fever on the number of febrile seizure recurrences in children at increased risk. The possibility that antipyretics can reduce recurrence has been addressed before. None of these studies were placebo-controlled trials with a standardized antipyretic treatment schedule; hence, the results were inconclusive. Although it had been described previously that ibuprofen reduces fever safely and adequately in children with febrile seizures, which is confirmed by the present study, we found that fever was not reduced in those fever episodes in which a recurrence occurred. Factors that may have influenced these results are discussed. Preventive treatment alternatives include primarily intermittent treatment with diazepam. Only children with a high recurrence risk may benefit from it. In a meta-analysis, its efficacy could not be demonstrated. The most important issue in treating children with febrile seizures is reducing parental anxiety by providing information about the excellent prognosis. Conclusion. We found no evidence that ibuprofen administration during fever prevents febrile seizure recurrence.

Predicting serious bacterial infection in young children with fever without apparent source

The aim of this study was to design a clinical rule to predict the presence of a serious bacterial infection in children with fever without apparent source. Information was collected from the records of children aged 1‐36 mo who attended the paediatric emergency department because of fever without source (temperature ≥38°C and no apparent source found after evaluation by a general practitioner or history by a paediatrician). Serious bacterial infection included bacterial meningitis, sepsis, bacteraemia, pneumonia, urinary tract infection, bacterial gastroenteritis, osteomyelitis and ethmoiditis. Using multivariate logistic regression and the area under the receiver operating characteristic curve (ROC area), the diagnostic value of predictors for serious bacterial infection was judged, resulting in a risk stratification. Twenty‐five percent of the 231 patients enrolled in the study (mean age 1.1 y) had a serious bacterial infection. Independent predictors from history and examination included duration of fever, poor micturition, vomiting, age, temperature <36.7°C or ≥40°C at examination, chest‐wall retractions and poor peripheral circulation (ROC area: 0.75). Independent predictors from laboratory tests were white blood cell count, serum C‐reactive protein and the presence of ≥70 white blood cells in urinalysis (ROC area: 0.83). The risk stratification for serious bacterial infection ranged from 6% to 92%.

Signs of meningeal irritation at the emergency department : How often bacterial meningitis?

Objective Although signs of meningeal irritation are highly indicative of meningitis, they are not pathognomonic. In this study, we described the final diagnoses in children with signs of meningeal irritation, and we assessed the frequency of bacterial meningitis related to specific signs of meningeal irritation. Methods Information was collected from records of 326 patients (aged 1 month to 15 years) who visited the emergency department of the Sophia Children’s Hospital between 1988 and 1998 with signs of meningeal irritation, assessed by either the general practitioner or the pediatrician. Results Bacterial meningitis was diagnosed in 99 patients (30%), viral or aseptic meningitis in 43 (13%). Other diagnoses were pneumonia (8%), other serious bacterial infections (2%), and upper respiratory tract infections or other self-limiting diseases (46%). Presence of one of the signs of meningeal irritation assessed by the pediatrician was related to bacterial meningitis in 39%. Specific tests eliciting meningeal irritation, such as Brudzinski’s and Kernig’s signs, were not related to a higher frequency of bacterial meningitis than neck stiffness and the tripod phenomenon. In children ≤1 year, bacterial meningitis is more frequently related to presence of irritability and a bulging fontanel. Conclusion Bacterial meningitis is present in 30% of children with signs of meningeal irritation. Presence of meningeal irritation as assessed by the pediatrician is related to bacterial meningitis in 39%. A better prediction of bacterial meningitis was not achieved by using more specific tests for signs of meningeal irritation.

Validating and updating a prediction rule for serious bacterial infection in patients with fever without source

Aim: To externally validate and update a previously developed rule for predicting the presence of serious bacterial infections in children with fever without apparent source.

Seizure recurrence after a first febrile seizure: a multivariate approach

The results are presented of a follow‐up study of 155 Dutch children after the first febrile seizure. Of these initially untreated children 37 per cent had had at least one, 30 per cent at least two and 17 per cent at least three subsequent seizures. The vulnerable period for recurrent seizures after the first febrile seizure was between 12 and 24 months, whereafter the risk was four to five times lower; after any seizure the risk was highest within the first six months, declining steadily after six months without seizures. A first‐degree family history of any type of seizure predicted multiple recurrences; an age of at least 30 months and a temperature of 40d̀C at initial seizure were associated with reduced risk. Factors in combination influenced the risk of recurrent seizures, sometimes in opposite ways.

A predictive model to estimate the risk of serious bacterial infections in febrile infants

Low risk criteria have been defined to identify febrile infants unlikely to have serious bacterial infection (SBI). Using these criteria approximately 40% of all febrile infants can be defined as being at low risk. Of the remaining infants (60%) only 10%–20% have an SBI. No adequate criteria exist to identify these infants. All infants aged 2 weeks-1 year, presenting during a 1-year-period with rectal temperature ≥38.0°C to the Sophia Childrens Hospital were included in a prospective study. Infants with a history of prematurity, perinatal complications, known underlying disease, antibiotic treatment or vaccination during the preceding 48 h were excluded. Clinical and laboratory variables at presentation were evaluated by a multivariate logistic regression model using SBI as the dependent variable. By using likelihood ratios a predictive model was derived, providing a post test probability of SBI for every individual patient. Of the 138 infants included in the study, 33 (24%) had SBI. Logistic regression analysis defined C-reactive protein (CRP), duration of fever, a standardized clinical impression score, a history of diarrhoea and focal signs of infection as independent predictors of SBI.ConclusionCRP, duration of fever, the “standardized clinical impression score”, a history of diarrhoea and focal signs of infection were the independent, most powerful predictors of SBI in febrile infants, identified by logistic regression analysis. Although the predictive model is not validated for direct clinical use, it illustrates the clinical potential of the used technique. This technique offers the advantage to assess the probability of SBI in every individual infant. This probability will form the best basis for well-founded decisions in the management of the individual febrile infant.

Early prediction of neurological sequelae or death after bacterial meningitis

This study determined independent predictors of the occurrence of permanent neurological sequelae or death after childhood bacterial meningitis. Data were used from a large study on children (aged 1 mo to 15 y) initially presenting with meningeal irritation. A nested case‐control study was performed on children with (n= 23) and without (n= 70) permanent neurological sequelae (hearing impairment, locomotor dysfunction, mental retardation or epilepsy) or death after bacterial meningitis. Predictors obtained from clinical evaluation and laboratory tests at presentation and during the clinical course were identified by multivariate logistic regression and receiver operating characteristic (ROC) curve analyses. The study population comprised 23 cases and 70 controls (52% boys, median age 2.8 y). Independent predictors for an adverse outcome after bacterial meningitis were male gender, atypical convulsions in history, low body temperature at admission and the pathogen Streptococcus pneumoniae. The area under the ROC curve of this prediction rule was 0.87 (95% confidence interval: 0.78–0.96), which was not improved by adding other characteristics. A score including these independent predictors could classify patients into categories with increasing risk for an adverse outcome.