Uri Kartoun

A Human-Robot Collaborative Reinforcement Learning Algorithm

This paper presents a new reinforcement learning algorithm that enables collaborative learning between a robot and a human. The algorithm which is based on the Q(λ) approach expedites the learning process by taking advantage of human intelligence and expertise. The algorithm denoted as CQ(λ) provides the robot with self awareness to adaptively switch its collaboration level from autonomous (self performing, the robot decides which actions to take, according to its learning function) to semi-autonomous (a human advisor guides the robot and the robot combines this knowledge into its learning function). This awareness is represented by a self test of its learning performance. The approach of variable autonomy is demonstrated and evaluated using a fixed-arm robot for finding the optimal shaking policy to empty the contents of a plastic bag. A comparison between the CQ(λ) and the traditional Q(λ)-reinforcement learning algorithm, resulted in faster convergence for the CQ(λ) collaborative reinforcement learning algorithm.

Vision-Based Autonomous Robot Self-Docking and Recharging

This paper presents a method for autonomous recharging of a mobile robot, a necessity for achieving long-term robotic activity without human intervention. A recharging station is designed consisting of a stationary docking station and a docking mechanism mounted to an ER-1 Evolution Robotics robot. The docking station and docking mechanism serve as a dual-power source, providing a mechanical and electrical connection between the recharging system of the robot and a laptop placed on it. Docking strategy algorithms use vision based navigation. The result is a significantly low-cost, high-entrance angle tolerant system. Iterative improvements to the system, to resist environmental perturbations and implement obstacle avoidance, ultimately resulted in a docking success rate of 100 percent over 50 trials.

Using an Electronic Medical Records Database to Identify Non-Traditional Cardiovascular Risk Factors in Nonalcoholic Fatty Liver Disease

Objectives:Among adults with nonalcoholic fatty liver disease (NAFLD), 25% of deaths are attributable to cardiovascular disease (CVD). CVD risk reduction in NAFLD requires not only modification of traditional CVD risk factors but identification of risk factors unique to NAFLD.Methods:In a NAFLD cohort, we sought to identify non-traditional risk factors associated with CVD. NAFLD was determined by a previously described algorithm and a multivariable logistic regression model determined predictors of CVD.Results:Of the 8,409 individuals with NAFLD, 3,243 had CVD and 5,166 did not. On multivariable analysis, CVD among NAFLD patients was associated with traditional CVD risk factors including family history of CVD (OR 4.25, P=0.0007), hypertension (OR 2.54, P=0.0017), renal failure (OR 1.59, P=0.04), and age (OR 1.05, P<0.0001). Several non-traditional CVD risk factors including albumin, sodium, and Model for End-Stage Liver Disease (MELD) score were associated with CVD. On multivariable analysis, an increased MELD score (OR 1.10, P<0.0001) was associated with an increased risk of CVD. Albumin (OR 0.52, P<0.0001) and sodium (OR 0.96, P=0.037) were inversely associated with CVD. In addition, CVD was more common among those with a NAFLD fibrosis score >0.676 than those with a score ≤0.676 (39 vs. 20%, P<0.0001).Conclusions:CVD in NAFLD is associated with traditional CVD risk factors, as well as higher MELD scores and lower albumin and sodium levels. Individuals with evidence of advanced fibrosis were more likely to have CVD. These findings suggest that the drivers of NAFLD may also promote CVD development and progression.

Use of Chronic Oral Anticoagulation and Associated Outcomes Among Patients Undergoing Percutaneous Coronary Intervention

Background Contemporary rates of oral anticoagulant (OAC) therapy and associated outcomes among patients undergoing percutaneous coronary intervention (PCI) have been poorly described. Methods and Results Using data from an integrated health care system from 2009 to 2014, we identified patients on OACs within 30 days of PCI. Outcomes included in‐hospital bleeding and mortality. Of 9566 PCIs, 837 patients (8.8%) were on OACs, and of these, 7.9% used non–vitamin K antagonist agents. OAC use remained stable during the study (8.1% in 2009, 9.0% in 2014; P=0.11), whereas use of non–vitamin K antagonist agents in those on OACs increased (0% in 2009, 16% in 2014; P<0.01). Following PCI, OAC‐treated patients had higher crude rates of major bleeding (11% versus 6.5%; P<0.01), access‐site bleeding (2.3% versus 1.3%; P=0.017), and non–access‐site bleeding (8.2% versus 5.2%; P<0.01) but similar crude rates of in‐hospital stent thrombosis (0.4% versus 0.3%; P=0.85), myocardial infarction (2.5% versus 3.0%; P=0.40), and stroke (0.48% versus 0.52%; P=0.88). In addition, prior to adjustment, OAC‐treated patients had longer hospitalizations (3.9±5.5 versus 2.8±4.6 days; P<0.01), more transfusions (7.2% versus 4.2%; P<0.01), and higher 90‐day readmission rates (22.1% versus 13.1%; P<0.01). In adjusted models, OAC use was associated with increased risks of in‐hospital bleeding (odds ratio 1.50; P<0.01), 90‐day readmission (odds ratio 1.40; P<0.01), and long‐term mortality (hazard ratio 1.36; P<0.01). Conclusions Chronic OAC therapy is frequent among contemporary patients undergoing PCI. After adjustment for potential confounders, OAC‐treated patients experienced greater in‐hospital bleeding, more readmissions, and decreased long‐term survival following PCI. Efforts are needed to reduce the occurrence of adverse events in this population.

Predictive Modeling of Physician-Patient Dynamics That Influence Sleep Medication Prescriptions and Clinical Decision-Making

Insomnia remains under-diagnosed and poorly treated despite its high economic and social costs. Though previous work has examined how patient characteristics affect sleep medication prescriptions, the role of physician characteristics that influence this clinical decision remains unclear. We sought to understand patient and physician factors that influence sleep medication prescribing patterns by analyzing Electronic Medical Records (EMRs) including the narrative clinical notes as well as codified data. Zolpidem and trazodone were the most widely prescribed initial sleep medication in a cohort of 1,105 patients. Some providers showed a historical preference for one medication, which was highly predictive of their future prescribing behavior. Using a predictive model (AUC = 0.77), physician preference largely determined which medication a patient received (OR = 3.13; p = 3 × 10−37). In addition to the dominant effect of empirically determined physician preference, discussion of depression in a patient’s note was found to have a statistically significant association with receiving a prescription for trazodone (OR = 1.38, p = 0.04). EMR data can yield insights into physician prescribing behavior based on real-world physician-patient interactions.

NATURAL LANGUAGE PROCESSING IMPROVES PHENOTYPIC ACCURACY IN AN ELECTRONIC MEDICAL RECORD COHORT OF TYPE 2 DIABETES AND CARDIOVASCULAR DISEASE

Electronic Medical Records (EMR) use clinical data to enable large-scale clinical studies. We created an EMR cohort of type 2 diabetes (T2D) patients from a large academic hospital system, to enable risk stratification of T2D patients at population scale. We hypothesize that natural language

Physical Modeling of a Bag Knot in a Robot Learning System

This paper presents a physical model developed to find the directions of forces and moments required to open a plastic bag - which forces will contribute toward opening the knot and which forces will lock it further. The analysis is part of the implementation of a Q(¿)-learning algorithm on a robot system. The learning task is to let a fixed-arm robot observe the position of a plastic bag located on a platform, grasp it, and learn how to shake out its contents in minimum time. The physical model proves that the learned optimal bag shaking policy is consistent with the physical model and shows that there were no subjective influences. Experimental results show that the learned policy actually converged to the best policy.

Bag Classification Using Support Vector Machines

This paper describes the design of multi-category support vector machines (SVMs) for classification of bags. To train and test the SVMs a collection of 120 images of different types of bags were used (backpacks, small shoulder bags, plastic flexible bags, and small briefcases). Tests were conducted to establish the best polynomial and Gaussian RBF (radial basis function) kernels. As it is well known that SVMs are sensitive to the number of features in pattern classification applications, the performance of the SVMs as a function of the number and type of features was also studied. Our goal here, in feature selection is to obtain a smaller set of features that accurately represent the original set. A K-fold cross validation procedure with three subsets was applied to assure reliability. In a kernel optimization experiment using nine popular shape features (area, bounding box ratio, major axis length, minor axis length, eccentricity, equivalent diameter, extent, roundness and convex perimeter), a classification rate of 95% was achieved using a polynomial kernel with degree six, and a classification rate of 90% was achieved using a RBF kernel with 27 sigma. To improve these results a feature selection procedure was performed. Using the optimal feature set, comprised of bounding box ratio, major axis length, extent and roundness, resulted in a classification rate of 96.25% using a polynomial kernel with degree of nine. The collinearity between the features was confirmed using principle component analysis, where a reduction to four components accounted for 99.3% of the variation for each of the bag types.

The MELD-Plus: A generalizable prediction risk score in cirrhosis

Background and aims Accurate assessment of the risk of mortality following a cirrhosis-related admission can enable health-care providers to identify high-risk patients and modify treatment plans to decrease the risk of mortality. Methods We developed a post-discharge mortality prediction model for patients with a cirrhosis-related admission using a population of 314,292 patients who received care either at Massachusetts General Hospital (MGH) or Brigham and Women’s Hospital (BWH) between 1992 and 2010. We extracted 68 variables from the electronic medical records (EMRs), including demographics, laboratory values, diagnosis codes, and medications. We then used a regularized logistic regression to select the most informative variables and created a risk score that comprises the selected variables. To evaluate the potential for generalizability of our score, we applied it on all cirrhosis-related admissions between 2010 and 2015 at an independent EMR data source of more than 18 million patients, pooled from different health-care systems with EMRs. We calculated the areas under the receiver operating characteristic curves (AUROCs) to assess prediction performance. Results We identified 4,781 cirrhosis-related admissions at MGH/BWH hospitals, of which 778 resulted in death within 90 days of discharge. Nine variables were the most effective predictors for 90-day mortality, and these included all MELD-Na’s components, as well as albumin, total cholesterol, white blood cell count, age, and length of stay. Applying our nine-variable risk score (denoted as “MELD-Plus”) resulted in an improvement over MELD and MELD-Na scores in several prediction models. On the MGH/BWH 90-day model, MELD-Plus improved the performance of MELD-Na by 11.4% (0.78 [95% CI, 0.75–0.81] versus 0.70 [95% CI, 0.66–0.73]). In the MGH/BWH approximate 1-year model, MELD-Plus improved the performance of MELD-Na by 8.3% (0.78 [95% CI, 0.76–0.79] versus 0.72 [95% CI, 0.71–0.73]). Performance improvement was similar when the novel MELD-Plus risk score was applied to an independent database; when considering 24,042 cirrhosis-related admissions, MELD-Plus improved the performance of MELD-Na by 16.9% (0.69 [95% CI, 0.69–0.70] versus 0.59 [95% CI, 0.58–0.60]). Conclusions We developed a new risk score, MELD-Plus that accurately stratifies the short-term mortality of patients with established cirrhosis, following a hospital admission. Our findings demonstrate that using a small set of easily accessible structured variables can help identify novel predictors of outcomes in cirrhosis patients and improve the performance of widely used traditional risk scores.

Development of an Algorithm to Identify Patients with Physician-Documented Insomnia

We developed an insomnia classification algorithm by interrogating an electronic medical records (EMR) database of 314,292 patients. The patients received care at Massachusetts General Hospital (MGH), Brigham and Women’s Hospital (BWH), or both, between 1992 and 2010. Our algorithm combined structured variables (such as International Classification of Diseases 9th Revision [ICD-9] codes, prescriptions, laboratory observations) and unstructured variables (such as text mentions of sleep and psychiatric disorders in clinical narrative notes). The highest classification performance of our algorithm was achieved when it included a combination of structured variables (billing codes for insomnia, common psychiatric conditions, and joint disorders) and unstructured variables (sleep disorders and psychiatric disorders). Our algorithm had superior performance in identifying insomnia patients compared to billing codes alone (area under the receiver operating characteristic curve [AUROC] = 0.83 vs. 0.55 with 95% confidence intervals [CI] of 0.76–0.90 and 0.51–0.58, respectively). When applied to the 314,292-patient population, our algorithm classified 36,810 of the patients with insomnia, of which less than 17% had a billing code for insomnia. In conclusion, an insomnia classification algorithm that incorporates clinical notes is superior to one based solely on billing codes. Compared to traditional methods, our study demonstrates that a classification algorithm that incorporates physician notes can more accurately, comprehensively, and quickly identify large cohorts of insomnia patients.