EvoK: Connecting loved ones through Heart Rate sharing
EEvoK: Connecting loved ones through Heart Rate sharing
ESHA SHANDILYA ∗ , Rochester Institute of Technology, USA
YIWEN WANG ∗ , Rochester Institute of Technology, USA
XUAN ZHAO ∗ , Rochester Institute of Technology, USA
MINGMING FAN,
Rochester Institute of Technology, USA
In this work, we present EvoK, a new way of sharing one’s heart rate with feedback from their close contacts to alleviate socialisolation and loneliness. EvoK consists of a pair of wearable prototype devices (i.e., sender and receiver). The sender is designed as aheadband enabling continuous sensing of heart rate with aesthetic designs to maximize social acceptance. The receiver is designed asa wristwatch enabling unobtrusive receiving of the loved one’s continuous heart rate with multi-modal notification systems.CCS Concepts: •
Human-centered computing → Human computer interaction (HCI) .Additional Key Words and Phrases: Emotion sharing, heart rate, social isolation, mental health, wearable device
According to WHO (World Health Organization), more than 264 million people worldwide have suffered from depression[18]. Without any intervention, depression could lead to suicide and cause mortality in a hazardous condition. Onefactor that seriously affects people’s mental health is social isolation and loneliness because of the lack of family ties,and communication [1]. Moreover, because of the COVID-19, the depression even doubled due to the implementationof new social rules such as social distancing and quarantine policy to protect everyone from the viruses [9]. The lackof interaction further intensifies their loneliness and adversely affects their mental health. In serious conditions, theisolated environment could induce physical illness and mental diseases [5].Researchers have investigated ways to alleviate the social loneliness of individuals, such as increasing one’s interactionwith their families and designing education programs and virtual companions [11, 15, 16]. One way to improve one’ssocial interaction and induce empathy is to sense and share their biosignals, such as breath patterns and heart rates[2, 3, 7, 10]. However, due to the limitation in comfort levels and elusive vibration feedback, it is necessary to investigatenew wearable form factors to comfortably and effectively sense and share people’s biosignals to increase their emotionalconnections with their loved ones. In this work, we design a pair of wearable devices for people to communicatetheir heart rate and for their loved ones to receive the heart rate notification with visual and audio feedback. The pairof devices is designed to strengthen the relationship between users and their loved ones through ambient feedback,comfortable and socially-acceptable design.
Reproducibility:
The source code for EvoK is available at https://github.com/EshaShandilya/evok.
Biosignals, such as pulse sensing (PPG signal), brain electrical activity (EEG signal), and respiratory rate, are usuallymeasured to understand the physiological process and activity of human beings [3, 12, 14]. One such biosignal is heartrate, which is helpful in expressing distinct aspects of a person’s emotional state [13]. Moreover, exchanging heart ratecan affect users’ social interaction and enhance their engagement [2, 13]. ∗ equal contribution 1 a r X i v : . [ c s . H C ] F e b handilya and Wang, et al. This has motivated researchers to design different form factors to sense and share heart rates. For example, Werneret al. [17] designed a ring that can detect the wearer’s heart rate and send it to the partner’s ring via vibrationfeedback. However, participants felt the vibration feedback as a feeling of "electric shock" and was elusive to inferthe corresponding heart rate. Croft and Lotan [8] created a device named imPulse with a curved surface so that userscould put it on their laps and palms on the surface, providing synchronizing light and vibration feedback. However,imPulse is nonwearable, which makes it immobile and unusable when a user is performing other activities. Min andNam designed WearBEAT to share body sound including the sounds of heartbeat [10]. In their design, one user worethe sound input part on the chest mount to sense the heart rate, and the other user received vibration feedback on theirwrist as output. However, the chest-mounted prototype violates the parameters of Zeagler’s [19] body map locationsfor wearable; since the position near breast could be uncomfortable for wearers especially for non-male users, whichmight affect its social-acceptability.Inspired by these designs, in this work, we explore a different form factor to sense and share one’s heart rate withthe aim of increasing its comfort level and social-acceptability. We adopt PPG sensing to detect heart rate as it has beendemonstrated to be a promising wearable heart rate sensing approach [2]. Through the combination of headband andwristwatch, we strive to provide a comfortable wearing experience and an intuitive interaction.
We followed an iterative design process to finalize the key design considerations for our wearable devices. The designconsiderations were based on - i) trade-offs between the comfort level and the detection accuracy of the heart ratesensor, ii) ambient notification, iii) socially-acceptable designs.
Trade-offs between the Comfort Level and the Detection Accuracy of the Heart Rate Sensor:
Before concludingthe sensor’s placement, we tested various feasible locations, specifically the finger-tip and the ear-lobe, to wear theheart rate sensor to get accurate heart rate. We observed that the finger-tip position added noise to the heart ratedue to hand-movements, whereas the sensor’s placement on the ear-lobe reduced noise in the recorded heart rate.Consequently, we decided to use the ear-lobe for the sensor’s placement. However, the batteries’ weight pulls thesensor down when placed on the earlobe, impacting the wearer’s comfort and the sensor’s accuracy. To overcome thischallenge, we brainstormed a solution that could support the batteries’ weight to keep the heart rate sensor’s positionintact on the ear lobe, providing a seamless experience to the wearer. We deliberated the feasibility of multiple designalternatives such as earrings , neckwear , hair clips , and headbands . The headband prototype, Figures 2, offers the bestweight distribution of the batteries than the competing alternatives; the encased batteries are attached to the headband,which rests on the head, providing stable heart rate sensor positioning. Ambient Notification:
The next design consideration was to design an unobtrusive notification, which the wearer caneasily follow without getting overwhelmed with the constant influx of the sender’s heart rate. According to Hanssonand Ljungstrand [4] colored LED lights are less intrusive methods of notification systems than other forms such assound. Therefore, we devised three different LED lights to indicate the heart rate range; considering that the normalheart rate range is between 60 to 100 for an adult, according to Mayo Clinic [6]. In the Figures 4, we see the bluelight for heart rate less than 60, the green light for the heart rate between 60 and 100, and red to highlight the heartrate beyond 100. In case there is a constant high heart rate of the sender (beyond 100), the receiver (wearer) will bealerted by a high pitched sound. Moreover, we also provide an option for the user to control the sender’s heart rate voK: Connecting loved ones through Heart Rate sharing transmission by pressing a button on the receiver’s prototype. We abandoned the idea of incorporating haptic feedbackto the prototypes, as it may be intrusive and dysfunctional for users to interact with the prototype. Socially-acceptable Designs:
Social acceptability of a wearable device directly depends on its placement on the user’sbody [19]. Our wearable designs are conceptualized according to the socially-acceptable body locations suggestedby Zeagler [19] that provide comfort and confidence to the wearer in public. Therefore, after referring to the bodymap [19], we chose the two areas – the head and the wrist that offer comparatively better affordance for the deviceplacement, and thus we select the form-factors of a headband for the sender and a wrist-watch for a receiver. Suchform-factors, (headband and wrist-watch), Figures 2 and 3, are intuitive, user-friendly and easy to interact with as theseare familiar form-factors to the users. We carefully determined these designs as these are gender-neutral and to themost extent used by many. However, we did not conduct user research to assess the designs’ social acceptance.
Fig. 1. This is the final stage, where we work to consolidate all the sensors in a usable and functional wearable design. Here the heartrate sensor is a headband design for the sender and when heart rate will be sensed from a specific range then, the data will be sharedin the form of visuals with music to the receiver which is in the form of a wrist watch. For different heart ranges, there are specificnotifications, please refer Figure 4 for full explanation on design notifications.
Our system consists of two parts, the sender and the receiver. The sender’s heart rate will be detected and sent to thereceiver, and the receiver will get different visual and audio notifications according to the received heart rate value.To conceptualize the sender part, we used a pulse rate sensor to detect the heart rate. The pulse rate sensor wasconnected to the micro: bit and the code for calculating the heart rate was downloaded to the micro: bit. The pulse ratesensor could be placed on the fingertip or earlobe. We tested both these two placements and found that placing thesensor on the earlobe gave us more stable signals. Also, considering the convenience of a user wearing the wearabledevice, the sensor should be intact while working out, performing some activity, and resting state. Thus, we decided toput the sensor on the earlobe. We tried to put the sensor on the earlobe and connected it to the micro: bit. We foundthat the gravity of the micro: bit would exert a great downward force on the earlobe, which could cause ear discomfort.We needed to put the micro: bit in a supportive position. Our initial idea was to put it in the cloth pocket; however, notall tops had pockets. Then we turned to body parts and found the head would be a good choice to put the micro: bit on.We wanted to make our device portable and could be used by anyone. The idea of the hairpin was abandoned becauseit did not apply to people with short hair. Our idea was to attach the micro: bit to a headband Figure 2. To compact handilya and Wang, et al. Fig. 2. The final design for the sender - Sender’s headband: a wearable headband banded with a 3D printing compact box, includesbattery, Micobits, connector and linked with pulse sensor. the battery and connector, we used the 3D printer to print a small box so that all components can get packed in oneFigure 2. When the sender uses this device, they need to put the headband on and place the pulse rate sensor on theearlobe, as shown in the Figure 1. After wearing the device, it needs to collect one or two minutes of data before gettingthe wearer’s normal heart rate.For the receiver part, we designed some feedback and interactions for the receiver. The receiver also needed to wearthe micro: bit to receive the data from the sender. To make it easier to see the heart rate value and feedback, we madethe device in the form of a wristwatch. We purchased a somatosensory control development board with a wrist band,an RGB led light, a buzzer, and a speaker, Figure 3. The micro: bit was connected to the board Figure 3. We tried toutilize the buzzer to provide haptic feedback; however, the micro: bit was not powerful enough to implement real-timehaptic feedback and caused long delays. In our design, we only used the led light and speaker to provide visual andaudible feedback, the description can be seen in Figure 4. When the sender’s heart rate was below the normal range[6], which was less than 60 per minute, the light would be blue. When the heart rate value was in the normal rangebetween 60 and 100, the light would be green. When the heart rate was beyond the normal range, which meant over100 per minute, the light would turn red, accompanied by a beep sound. The normal range of 60 to 100 was only usedfor our test. Users could set their own heart rate normal range. Additionally, to provide more flexibility, the receivercould control whether to receive the data or not. They could press the left button on the micro: bit to stop receiving thedata and feedback and press the left button again to resume receiving the data and feedback.
We present novel prototype designs with feedback for sharing heart rates to alleviate people’s social isolation from theirloved ones. This work presents a demonstration with a pair of wearable devices that mainly use micro:bit processor anda heart rate sensor. We exclusively made our designs wearable to ensure continuous connectivity through heart ratesharing and indicating the user’s physical and mental well-being.Future work should investigate users’ attitudes and perception towards the proposed way of sensing and sharingheart rates and elicit feedback to further improve its comfort level and social acceptability. Moreover, it is worthconducting a long-term in-the-wild study to reveal both technical issues, such as battery life, and practical issuesemerging from a wide range of daily scenarios. Lastly, as heart rate patterns may vary from person to person and can voK: Connecting loved ones through Heart Rate sharing Fig. 3. Receiver’s wristwatch: contains a somatosensory control development board which enables a user to interact with the prototypethrough audio, visual and haptic feedback.Fig. 4. Three ranges of heart rate with corresponding feedback: Blue LED represents less than 60. Green LED represents the normalrange between 60 and 100. Red LED and alarming sound represents over 100. According to Mayo Clinic [6] , the normal resting heartrate of an individual ranges between 60 and 100; other factors such as, age, fitness level, emotions could also influence the heartrate. The heartbeat value of the sender is displayed on the Microbit’s screen. Note: The heart rates displayed are two and three digitnumbers. One single digit is shown at a time, and the digits are moving from right to left. The first value on the receiver’s device isthe first digit 3 of 30 with a blue LED light, second value is 5, the last digit of value 65 with green LED light, and the last value is 1, thefirst digit of 130 with red LED light. carry important health- or emotion- related information, it is worth exploring the characteristics of such patterns anddesigning prototypes to capture and communicate them among loved ones.
REFERENCES [1] Ismail Hussein Amzat and Pamita Jayawardena. 2016. Emotional loneliness and coping strategies: A reference to older Malaysians at nursing homes.
Journal of Population Ageing
9, 3 (2016), 227–247.[2] Jérémy Frey. 2014. Heart rate monitoring as an easy way to increase engagement in human-agent interaction. arXiv preprint arXiv:1412.1772 (2014).[3] Jérémy Frey, May Grabli, Ronit Slyper, and Jessica R Cauchard. 2018. Breeze: Sharing biofeedback through wearable technologies. In
Proceedings ofthe 2018 CHI Conference on Human Factors in Computing Systems . 1–12.[4] Rebecca Hansson and Peter Ljungstrand. 2000. The reminder bracelet: subtle notification cues for mobile devices. In
CHI’00 Extended Abstracts onHuman Factors in Computing Systems . 323–324.[5] Louise C Hawkley and John T Cacioppo. 2010. Loneliness matters: A theoretical and empirical review of consequences and mechanisms.
Annals ofbehavioral medicine handilya and Wang, et al. [7] Fannie Liu, Mario Esparza, Maria Pavlovskaia, Geoff Kaufman, Laura Dabbish, and Andrés Monroy-Hernández. 2019. Animo: Sharing Biosignals ona Smartwatch for Lightweight Social Connection.
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
3, 1 (2019),1–19.[8] Gilad Lotan and Christian Croft. 2007. ImPulse. In
CHI ’07 Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA) (CHI EA’07) . Association for Computing Machinery, New York, NY, USA, 1983–1988. https://doi.org/10.1145/1240866.1240936[9] Brett Marroquín, Vera Vine, and Reed Morgan. 2020. Mental health during the COVID-19 pandemic: Effects of stay-at-home policies, social distancingbehavior, and social resources.
Psychiatry research
293 (2020), 113419.[10] Hyeryung Christine Min and Tek-Jin Nam. 2014. Biosignal sharing for affective connectedness. In
CHI’14 Extended Abstracts on Human Factors inComputing Systems . 2191–2196.[11] Simone Pettigrew and Michele Roberts. 2008. Addressing loneliness in later life.
Aging and Mental Health
12, 3 (2008), 302–309.[12] Rafael Ramirez and Zacharias Vamvakousis. 2012. Detecting emotion from EEG signals using the emotive epoc device. In
International Conferenceon Brain Informatics . Springer, 175–184.[13] Petr Slovák, Joris Janssen, and Geraldine Fitzpatrick. 2012. Understanding heart rate sharing: towards unpacking physiosocial space. In
Proceedingsof the SIGCHI Conference on Human Factors in Computing Systems . 859–868.[14] Andriy Temko. 2017. Accurate heart rate monitoring during physical exercises using PPG.
IEEE Transactions on Biomedical Engineering
64, 9 (2017),2016–2024.[15] Christiana Tsiourti, Emilie Joly, Cindy Wings, Maher Ben Moussa, and Katarzyna Wac. 2014. Virtual assistive companions for older adults: qualitativefield study and design implications. In
Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare . 57–64.[16] Jenny Waycott, Amee Morgans, Sonja Pedell, Elizabeth Ozanne, Frank Vetere, Lars Kulik, and Hilary Davis. 2015. Ethics in evaluating a sociotechnicalintervention with socially isolated older adults.
Qualitative health research
25, 11 (2015), 1518–1528.[17] Julia Werner, Reto Wettach, and Eva Hornecker. 2008. United-Pulse: Feeling Your Partner’s Pulse. In
Proceedings of the 10th International Conferenceon Human Computer Interaction with Mobile Devices and Services (Amsterdam, The Netherlands) (MobileHCI ’08)