Carole L. Kimberlin

Validity and reliability of measurement instruments used in research

PURPOSE Issues related to the validity and reliability of measurement instruments used in research are reviewed. SUMMARY Key indicators of the quality of a measuring instrument are the reliability and validity of the measures. The process of developing and validating an instrument is in large part focused on reducing error in the measurement process. Reliability estimates evaluate the stability of measures, internal consistency of measurement instruments, and interrater reliability of instrument scores. Validity is the extent to which the interpretations of the results of a test are warranted, which depends on the particular use the test is intended to serve. The responsiveness of the measure to change is of interest in many of the applications in health care where improvement in outcomes as a result of treatment is a primary goal of research. Several issues may affect the accuracy of data collected, such as those related to self-report and secondary data sources. Self-report of patients or subjects is required for many of the measurements conducted in health care, but self-reports of behavior are particularly subject to problems with social desirability biases. Data that were originally gathered for a different purpose are often used to answer a research question, which can affect the applicability to the study at hand. CONCLUSION In health care and social science research, many of the variables of interest and outcomes that are important are abstract concepts known as theoretical constructs. Using tests or instruments that are valid and reliable to measure such constructs is a crucial component of research quality.

Relationship between quality of life, social support and disease-related factors in HIV-infected persons in Venezuela

This study examines the relationships among health-related quality of life (HRQL), social support, sociodemographic factors and disease-related factors in persons infected with the human immunodeficiency virus (HIV) living in Venezuela. A sample of 118 HIV-infected persons living in Caracas, Venezuela, was surveyed using a written questionnaire that included a Spanish translation of the Interpersonal Support Evaluation List (ISEL) developed for this study, the Medical Outcomes Study Short Form-36 (SF-36) and a symptom inventory. All three instruments showed good internal consistency reliability. Multiple regression analyses were used to model SF-36 sub-scale scores as a function of symptoms, social support, HIV-status and use of antiretroviral drugs. The models explained between 16 and 39% of the variance in the different HRQL domains. Controlling for other variables in the model, level of symptomatology was significantly associated with all HRQL domains except social functioning and role-emotional scores. Social support was significantly associated with all HRQL domains except physical functioning and bodily pain. The use of antiretroviral drugs was significantly associated with social functioning. The study indicates the importance of social support to the quality of life of HIV-infected individuals in this culture.

Effects of an Education Program for Community Pharmacists on Detecting Drug-related Problems in Elderly Patients

Community pharmacists are in a position to assume increased responsibility for preventing and resolving drug-related problems in ambulatory patients. Such an expanded role is mandated under provisions of the Omnibus Budget Reconciliation Act of 1990. The need for pharmacist oversight of drug therapy may be most acute in elderly patients. This study reports on a program to teach community pharmacists a process of assessing drug therapy of elderly patients and intervening to correct problems. Community pharmacists (N = 102) were assigned to treatment and control conditions. Both groups targeted patients meeting criteria and enrolled them into the study. Treatment group pharmacists, who participated in a training program, also assessed the medication use of enrolled patients to identify and resolve medication-related problems. Patients (N = 762) were telephoned by researchers 1 month after enrollment for an interview. Comparisons between treatment and control group patients were made on reports of pharmacist activities, knowledge of regimens, compliance, and potential drug therapy problems, such as interactions and side effects. Treatment patients were more likely to report that pharmacists provided information and assessed for problems than were control patients. These differences were maintained on 3-month follow-up questionnaires. No differences were found on the odds that patients indicated misunderstanding of regimens, noncompliance, or potential therapeutic problems.

Perceived access to care and medication use among ambulatory elderly in Rio de Janeiro, Brazil

This study examined physician and non-physician prescribed medication use of a growing segment of Brazilian society--the elderly. Personal interviews were conducted with 436 subjects in a stratified random sampling of elderly respondents to the previously completed 1990 Brazilian Old Age Survey (BOAS). The BOAS sample had been stratified according to the socioeconomic status (SES) of three communities within Rio de Janeiro. This study focused on medication use of these subjects as a function of the predisposing, enabling, and need variables which have been found in previous research to predict other types of health services utilization (HSU). The enabling variables of access to care were measured as patient perceived availability, affordability, and acceptability of both medical and pharmacy services. ANOVA results found differences among the different communities in perceived availability and affordability of medical and pharmacy services and acceptability of pharmacy services. Subjects from Santa Cruz, the lowest SES area, consistently reported lesser availability of services, more difficulties with affordability but greater perceived acceptability of pharmacy services than those from the highest SES area. Multivariate regressions modeling both prescribed and non-prescribed medication use for the three areas found that the access-oriented HSU framework was much more effective in explaining the variance in medication use for the lowest SES area (45% and 48% for prescribed and non-prescribed use, respectively) than for the highest SES area where only 16% and 18% of prescribed and non-prescribed medication use was explained.

Patient counseling practices in U.S. pharmacies: Effects of having pharmacists hand the medication to the patient and state regulations on pharmacist counseling

OBJECTIVES To determine the amount and type of oral counseling given to shoppers posing as new patients with new prescriptions and to determine how state regulations, pharmacy and pharmacist characteristics, and environmental factors affect oral counseling practices. DESIGN Cross-sectional, observational, correlational study. SETTING 41 states and the District of Columbia between January 28 and March 31, 2008. PARTICIPANTS 365 community pharmacy staff members had interactions with shopper-patients. INTERVENTION Shoppers presented new prescriptions in community pharmacies and recorded observations related to oral communication with pharmacy staff. MAIN OUTCOME MEASURES Oral provision of medication information and risk information to shoppers by pharmacy staff, as well as questions asked of shoppers by pharmacy staff. RESULTS Some form of oral communication related to a medication was reported in 68% of encounters. At least one informational item for either medication was provided for approximately 42% of encounters. At least one risk information item was provided in 22% of encounters. Logistic regression findings indicated that the strongest predictor of oral counseling, either providing information or asking questions, was the pharmacist being the pharmacy staff member who handed the medication to the patient, controlling for all other variables in the models. In addition, having strict state regulations specifying that pharmacists must counsel all patients receiving new prescriptions predicted whether patients received counseling. A more private area for prescription pick up also was a significant predictor. CONCLUSION The importance of the direct encounter between the pharmacist and patient and strict state regulations mandating that pharmacists counsel patients with new prescriptions were highlighted by these findings.

Physicians’ intent to comply with the American Medical Association’s guidelines on gifts from the pharmaceutical industry

Objective: To identify factors that predict physicians’ intent to comply with the American Medical Association’s (AMA’s) ethical guidelines on gifts from the pharmaceutical industry. Methods: A survey was designed and mailed in June 2004 to a random sample of 850 physicians in Florida, USA, excluding physicians with inactive licences, incomplete addresses, addresses in other states and pretest participants. Factor analysis extracted six factors: attitude towards following the guidelines, subjective norms (eg, peers, patients, etc), facilitating conditions (eg, knowledge of the guidelines, etc), profession-specific precedents (eg, institution’s policies, etc), individual-specific precedents (physicians’ own discretion, policies, etc) and intent. Multivariate regression modelling was conducted. Results: Surveys were received from 213 physicians representing all specialties, with a net response rate of 25.5%. 62% (n = 133) of respondents were aware of the guidelines; 50% (n = 107) had read them. 48% (n = 102) thought that following the guidelines would increase physicians’ credibility and professional image; 68% (n = 145) agreed that it was important to do so. Intent to comply was positively associated with attitude, subjective norms, facilitators and sponsorship of continuing medical education (CME) events, while individual-specific precedents had a negative relationship with intent to comply. Predictors of intent (R2 = 0.52, p <0) were attitude, subjective norms, the interaction term (attitude and subjective norms), sponsorship of CME events and individual-specific precedents. Conclusions: Physicians are more likely to follow the AMA guidelines if they have positive attitudes towards the guidelines, greater subjective norms, fewer expectations of CME sponsorship and fewer individual-specific precedents. Physicians believing that important individuals or organisations expect them to comply with the guidelines are more likely to express intent, despite having fewer beliefs that positive outcomes would result through compliance.

Predicting Success of Pharmacy Students in Basic Science and Clinical Clerkship Courses

A number of studies on the ability of admissions variables to predict success in pharmacy schools have examined only success in the first professional year, which typically consists primarily of basic science courses. This study examined not only grades in basic science courses but also performance on clinical clerkships, for two classes of students. It also examined the ability of various personality variables to predict performance in clinical and basic science coursework. Previous grade point average (GPA) was the best single predictor of performance. In one class, the personality variable of Responsibility best predicted clinical clerkship performance. However, it only accounted for 13 percent of the variance in clerkship grades. Pharmacy College Admission Test (PCAT) Biology and PCAT Verbal Ability scores added to the predictive ability of previous GPA in one class, but none of the PCAT scales entered a prediction equation for the other class. The limitations on our ability to predict, with any consistency, academic performance in pharmacy school is discussed.

Questions elderly patients have about on-going therapy: a pilot study to assist in communication with physicians.

Purpose of the study: This pilot study examined the prevalence and types of questions elderly patients have about their current drug therapy. It also evaluated the effectiveness of a brief intervention to prepare patients to ask questions about drug therapy during medical visits.Design and methods: The research used a posttest‐only experimental design. Forty‐five elderly patients seen at a primary care clinic during a one‐month period consented to participate and completed the study. After consent, subjects were randomly assigned to intervention and control conditions. A brief interview with intervention group subjects conducted by a medical student assigned to the clinic as part of a summer research experience helped subjects formulate questions they had about current therapy before they went into medical visits. Patient‐physician visits were audiotaped and patient questions about medications and health care were tallied.Results: Subjects in the intervention group were significantly more likely to ask questions of providers than were subjects in the control group. Intervention group subjects were found to ask a wider variety of medication‐related questions than were control group subjects, including questions related to proper use, problems perceived with medications, and effectiveness of treatment.Implications: Assisting patients to formulate questions before medical visits results in an increased likelihood that patients will ask questions and will ask a wider variety of questions during the medical visit.

Influence of explanatory and confounding variables on HRQoL after controlling for measurement bias and response shift in measurement

The purpose of this study was to examine the influence of explanatory and confounding variables on health-related quality of life after accounting for response shift, measurement bias and response shift in measurement using structural equation modeling. Hypertensive patients with coronary artery disease randomized to anti-hypertensive treatment, completed the ShortForm-36 questionnaire at both baseline and 1 year (n = 788). Three measurement biases were found and all three were considered as response shift in measurement. Older patients reported worse scores for both physical functioning (PF) and role-physical at baseline and 1 year later compared to younger patients; and males reported better PF than females after conditioning on the latent trait of general physical health. Before controlling for response shift, patients’ PF scores were not statistically different over time; however, PF scores significantly improved (p < 0.01) after controlling for recalibration response shift. Assessment of how patients perceive their change in health-related quality of life over time is warranted.

Investigation of the Potential Effects of Registration Requirements on Florida Pharmacy Technician Credentials

Background: On June 23, 2008, Florida adopted regulations mandating credentialing requirements for pharmacy technicians. Objective: To investigate the Pharmacy Technician Certification Board (PTCB) certified pharmacy technician self-report and pharmacist report of work experience, education, training, and non-PTCB certification credentials of Florida community pharmacy technicians prior to the adoption of registration requirements in that state and to predict the effects of the requirements on Florida pharmacy technicians. Methods: A self-administered questionnaire was mailed to 2000 Florida pharmacists and 2000 Florida PTCB Certified Pharmacy Technicians (CPhTs) in 2004. Results: Of the 1230 returned questionnaires, 571 (46%) were from pharmacists and 659 (54%) were from technicians. Most of the pharmacists in the sample reported that they had supervised a CPhT, while few had supervised a technician with an AA in Pharmacy Technician Sciences. Most of the technicians sampled reported having some higher education. Less than half reported having formal technician training. Few reported having non-PTCB pharmacy technician certificates. Conclusions: Florida pharmacy technicians had some higher education or technician training credentialing, despite its not being mandatory. Pharmacists may not always be aware of the education and credentials of the technicians they supervise. The new registration requirements will most likely have little effect on existing Florida pharmacy technicians who register before January 1, 2011. Those who are CPhTs will have to complete types of continuing education that are different from those that they were required to complete for certification. Those who are not CPhTs will have to complete continuing education programs.