Study of Vital Data Analysis Platform Using Wearable Sensor
社社団法人 電子情報通信学会
THE INSTITUTE OF ELECTRONICS,INFORMATION AND COMMUNICATION ENGINEERS 信学技報
TECHNICAL REPORT OF IEICE.SC2016-34 (2017-03) ウェアラブルセンサーを用いたバイタルデータ分析プラットフォームの検討 山登 庸次 † † NTT ソフトウェアイノベーションセンタ東京都武蔵野市緑町 † [email protected] あらまし 本稿では,バイタルデータを活用しリアルタイムなアクションに繋げるための,ウェアラブルセンサーデータを用いた分析プラットフォームを提案する.近年 IoT 技術が進展しているが,ヘルスケア分野では,分析したバイタルデータを元に,リアルタイムなアクションを行うアプリケーションは十分検討されていない.その原因としては,ストリーム処理 / マイクロバッチ処理の適切な使い分け,ネットワークコストの検討が不十分であるからと考える.既存の課題を解決するため,私達はバイタルデータ分析プラットフォームを提案する.提案は,ウェアラブルバイタルセンサーを一つの例に用いて,着用者の心電や加速度のバイタルデータを取得して,スマートホン及びクラウドで姿勢,疲労度,緊張度等の分析を行い,危険な姿勢や疲労度推移等を取得可能とする.プラットフォームを実装し,トライアルの準備を行っている. キーワード IoT, ウェアラブルセンサー,クラウド , Spark Streaming, リアルタイム処理 , Study of Vital Data Analysis Platform Using Wearable Sensor
Yoji YAMATO † † Software Innovation Center, NTT Corporation3-9-11, Midori-cho, Musashino-shi, Tokyo 1808585 JapanE-mail: † [email protected] Abstract
In this paper, we propose a vital data analysis platform which resolves existing problems to utilize vitaldata for real-time actions. Recently, IoT technologies have been progressed but in the healthcare area, real-timeactions based on analyzed vital data are not considered sufficiently yet. The causes are proper use of analyzingmethods of stream / micro batch processing and network cost. To resolve existing problems, we propose our vitaldata analysis platform. Our platform collects vital data of Electrocardiograph and acceleration using an example ofwearable vital sensor and analyzes them to extract posture, fatigue and relaxation in smart phones or cloud. Ourplatform can show analyzed dangerous posture or fatigue level change. We implemented the platform and we arenow preparing a field test.
Key words
IoT, Wearable Sensor, Cloud Computing, Spark Streaming, Real-Time Processing,
1. Introduction
Recently, IoT (Internet of Things) technologies have beenprogressed. IoT is the technology to attach communicationfunctions to physical things, connect things to networks, ana-lyze things data to enable automatic control. IoT applicationareas are wide such as manufacturing, supply chain [1] [2],maintenance which Industrie4.0 [3] and Industrial Internet [4]target and also health care, agriculture, energy. To utilize IoT data, IoT platforms also appeared to de-velop and operate IoT applications effectively. AWS IoT[5] is a platform to analyze IoT data on a cloud by inte-grating several Amazon Web Services. For example, Ama-zon Kinesis collects and delivers IoT data by MQTT(MQTelemetry Transport) [6] protocol to a cloud and AmazonMachine Learning analyzes those data by machine learningalgorithms. To integrate IoT data and other services, thereare some service coordination technologies such as [7]- [13].— 1 —n manufacturing or maintenance area, there are appli-cations of appropriate timing maintenance actions based onmonitored business machine statuses (e.g., KOMTRAX [14]),but in the healthcare area, real-time actions based on ana-lyzed vital data are not considered sufficiently yet. Of course,there are applications to show daily statistical informationsuch as calorie consumption using wearable sensor data suchas amount of movement acquired by list band sensor, how-ever it has not been able to utilize vital data to real-timeactions.There are two main causes. The first is proper use of an-alyzing methods. To utilize vital data in real-time, not onlybatch processing but also stream processing for continuousdata and micro batch processing for bulk data of short pe-riod are needed, but it is not considered to apply health careindustry sufficiently. The second is network cost. Becausevital data is continuously generated, bandwidth to transferthem to a cloud is large.Based on these backgrounds, in this paper, we proposea vital data analysis platform which resolves existing twoproblems based on open source HortonWorks Data Platform(HDP) [15] architecture to utilize vital data. Our platformcollects workers’ vital data of Electrocardiograph and accel-eration using wearable vital sensor (e.g., [16]) and analyzesthem to extract posture, fatigue and relaxation in smartphones or cloud. Our platform can show analyzed dangerousposture or fatigue level change.The rest of this paper is organized as follows. In Section 2,we review existing IoT technologies. In Section 3, we proposeand design a vital data utilization platform which resolvesexisting problems. We summarize the paper in Section 4.
2. Overview of IoT data technologies
Because IoT technologies include a lot of topics such assensor, actuator, big data, platform, communication proto-col and so on, this section only introduces existing platformtechnologies and wearable sensor for IoT vital data analyzingapplications.To utilize IoT data collected by sensing technologies, AWSIoT [5] is a major platform. Amazon IoT can integrate eachservice of Amazon Web Services for IoT processing flow.Amazon Kinesis [17] can deliver IoT data to Amazon cloudby MQTT [6] protocol. To analyze IoT big data delivered byAmazon Kinesis, Amazon Machine Learning provides ma-chine learning functions such as regression or classification.NTT DOCOMO and GE release an IoT solution whichprovides GE’s industrial wireless router Orbit (MDS-Orbitplatform) with NTT DOCOMO’s communication module in2015 [18]. Companies can collect operation statuses of facili-ties such as bridge, electric and gas by setting Orbit. More- over, companies can develop IoT applications on Toami [19]which is an IoT cloud platform provided by NTT DOCOMOand enables visualization of collected data easily.IoT use cases not only analyzing and visualizing IoT databut also taking appropriate actions fast such as automaticrepair orders are increased. Therefore, IoT data analysisof stream processing such as Storm [20] or Spark Stream-ing [21] becomes more popular though batch processing suchas MapReduce [22] was major conventionally. Stream pro-cessing of IoT data enables fast actions based on real timesituation change. HDP [15] is a data processing platformwith all Open Source Software stack. HDP provides dataanalysis modules of batch and stream processing based onHDFS [23] and YARN. Users can analyze data by MapRe-duce, Spark Streaming or so on, and can store data to HBase,Hive or so on.Regarding to sensors for acquiring vital data, wearable ter-minals have been spread. There are various terminals suchas watch type, list band type, eyeglass type, T-shirt typeand so on. Apple Watch [24] is a watch type computer, con-tains heartbeat sensor, acceleration sensor and can collect vi-tal data continuously. Sony SmartEyeglass [25] is a eyeglasstype computer and can collect acceleration and luminancedata. Hitoe [16] is a T-shirt type wearable sensor NTT andToray develop and can collect Electrocardiograph(ECG) and3-axis acceleration data by wearing hitoe shirt.In this way, platforms and sensors have been progressedfor vital data analysis. However, when we consider to uti-lize vital data and take real-time business actions such assubstitute member assigns, existing technologies have someproblems.In AWS IoT, to analyze IoT data, users need to collect alldata to a cloud and need network cost for many sensors inmultiple regions. For example, a satellite communication isused to collect business vehicle’s sensing data [14] and net-work cost is huge. When users analyze collected data, Ama-zon Machine Learning or Amazon Lambda or other serviceson a cloud are used, however how to use each service for hugecontinuous vital data is not considered sufficiently.[18]’s IoT applications developed on Toami are mainly vi-sualize applications of collected data by batch processing.Therefore, applications which take real-time actions such asrepair parts orders based on analyzed data are not consid-ered.Though there are technologies for micro batch or streamprocessing such as Spark Streaming and Storm, current typ-ical applications are sequential data analysis of SNS posts oroperation log analysis. There are few applications for vitaldata analysis and proper use of micro batch and stream pro-cessing to extract necessary data is not discussed sufficiently.— 2 —ere, we summarize existing problems. The first is properuse of analyzing methods. To utilize vital data in real-time,not only batch processing but also stream processing for con-tinuous data and micro batch processing for bulk data ofshort period are needed, but it is not considered to applyhealth care industry sufficiently. The second is network cost.Because vital data is continuously generated, bandwidth totransfer them to a cloud is large.
3. Proposal of vital data analysis platformwhich resolves existing problems
In this section, we propose vital data analysis platformwhich resolves existing problems. In 3.1, we explain ap-proaches to resolve existing problems. In 3.2, we show adesign of platform to analyze vital data based on HDP.
3. 1 Approaches to resolve existing problems
Our platform collects workers’ vital data via wearable sen-sor such as hitoe, analyzes them and stores on a cloud. Ourplatform can store workers’ health statuses such as fatiguelevel change, alerts for emergent situation such as dangerousposture.To resolve existing problems, we propose following twoideas for our platform.The first is semi real-time analysis of fatigue and relax-ation by micro batch processing of vital data using SparkStreaming on a cloud.The second is stream processing of hitoe data on smartphones to extract primary processed data from raw data andto detect dangerous posture.Thanks to these ideas, we have following merits.Smart phones only send primary processed data (e.g.,heartbeat interval RRI) from huge continuous data (e.g.,ECG) to a cloud. This can reduce network cost. Smartphones also analyze posture in streaming processing and canimmediately notify alerts of dangerous postures during work-ing such as pick up things even if a mobile network is dis-connected.A cloud analyzes bulk primary processed data in a shortperiod (e.g., 1 minute RRI data) and extracts high level in-formation such as fatigue or relaxation. Because analyzingfatigue or relaxation needs complex analyzing logic, rich com-putation resources of cloud are used. And for high accuracyanalysis of high level information, analysis of bulk data withcertain period is needed. Therefore, we adopt micro batchprocessing which repeats storing and analyzing in a shortperiod.
3. 2 Design of vital data analysis platform
Figure 1 shows system image based on above ideas. Fig-ure 1 also shows processing steps of vital sensor data analysisusing HDP where sensor stream data is analyzed by Spark Streaming and analyzed data is stored to HBase. ThoughHDP provides various modules of Open Source Software suchas batch processing and SQL processing, we only use modulesfor target micro batch processing for high level health infor-mation. And HDP can be built on cloud middleware such asOpenStack (e.g., OpenStack cloud services such as [26]- [30]and cloud provisioning such as [31] [32]).Wearable sensor hitoe sends workers’ vital data to smartphones. Hitoe is an example of wearable sensor which NTTand Toray develop and collects ECG and 3-axis acceleration.Smart phones analyze vital data simply to extract primaryprocessed data of RRI from ECG and posture from accelera-tion and sends them to a cloud via REST style. This analysiscan be done by hitoe SDK [33]. Because raw data of ECG ishuge, primary processed data such as RRI is sent to a cloud.If smart phones detect dangerous postures such as pickingup things during working, smart phones notify to workers.Vital data is sent to a cloud and collected data is deliveredby messaging system Apache Kafka by publish/subscribemethod. Spark Streaming Dispatcher subscribes collecteddata to Kafka. The Dispatcher stores acquired data to HBaseand publishes cleansed data to Kafka. Spark StreamingAnalysis job subscribes cleansed data to Kafka. The Anal-ysis job analyzes cleansed data sets of with defined windowsize in a micro batch processing and extracts fatigue and re-laxation level. Window sizes of micro batch are configurablefor each extracted data type. Fatigue level is calculated us-ing RRI change (e.g., [34]) and relaxation level is calculatedwith cardiac vagal index (CVI) [35]. Lastly, analyzed data isstored to HBase.High level data such as fatigue and relaxation is utilizedvarious ways such as to analyze deeply with other sensordata. When we add other sensor data, we can use cloud con-figuration technologies of [36] and automatic regression testtechnologies of [37] [38] for configuration change. And if weneed computation power for high level data extraction, wecan use GPU to compute such as [39]. And for other sys-tems coordination, Web services [40]- [45] or other technolo-gies such as [46]- [50] can be used.We implemented our platform using Spark and HBase.Figure 2 shows a screen image of implemented platform andhitoe. We are preparing a field test. We will report the resultof field test in another paper.
4. Conclusion
In this paper, we proposed a vital data analysis platformwhich resolves existing problems based on HDP architectureto utilize vital data for real-time actions.Our platform has two characteristics. The first is semireal-time analysis of fatigue and relaxation by micro batch— 3 — 図 processing of vital data using Spark Streaming on a cloud.The second is stream processing of raw vital data on smartphones. This can reduce network cost to filter huge ECGdata and can notify dangerous posture immediately. Ourplatform can manage workers’ health information in real-time with low cost and enables real-time actions.We will propose our platform to actual industry companiesto conduct a field test. 文 献 [1] Y. Yamato, Y. Fukumoto and H. Kumazaki, “ Predic-tive Maintenance Platform with Sound Stream Analysis inEdges, ” Journal of Information Processing, Vol.25, 2017.[2] Y. Yamato, Y. Fukumoto and H. Kumazaki, “ Proposal ofShoplifting Prevention Service Using Image Analysis andERP Check, ” “ Service Elementsand Service Templates for Adaptive Service Compositionin a Ubiquitous Computing Environment, ” The 9th Asia-Pacific Conference on Communications (APCC2003), Vol.1,pp. 335-338, Penang, Sep. 2003.[8] Y. Yokohata, Y. Yamato, M. Takemoto and H. Sunaga, “ Service Composition Architecture for Programmabilityand Flexibility in Ubiquitous Communication Networks, ” IEEE International Symposium on Applications and the In-ternet Workshops (SAINTW’06), pp.145-148, Phoenix, Jan.2006.[9] 山登庸次,中辻真,須永宏,“ユビキタス環境で動的にサービス実現するためのサービス合成技術 , ” 情報処理学会論文誌 , Vol.48,No.2, pp.562-577, Feb. 2007.[10] 山登庸次,須永宏,“ユビキタスサービス合成に用いる抽象サービスシナリオ生成手法の検討 , ” 電子情報通信学会論文誌 ,Vol.J91-B, No.10, pp.1220-1230, Oct. 2008.[11] Y. Yamato, Y. Nakano and H. Sunaga, "Study and Evalu-ation of Context-Aware Service Composition and Change-Over Using BPEL Engine and Semantic Web Techniques,"IEEE Consumer Communications and Networking Confer-ence (CCNC 2008), pp.863-867, Jan. 2008.[12] Y. Yamato, H. Ohnishi and H. Sunaga, “ Development ofService Control Server for Web-Telecom Coordination Ser-vice, ” IEEE International Conference on Web Services(ICWS 2008), pp.600-607, Beijing, Sep. 2008.[13] H. Sunaga, Y. Yamato, H. Ohnishi, M. Kaneko, M. Iio andM. Hirano, “ Service Delivery Platform Architecture for theNext-Generation Network, ” “ Fastand Reliable Restoration Method of Virtual Resources onOpenStack, ” IEEE Transactions on Cloud Computing, DOI: — 4 — “ Evaluation ofAgile Software Development Method for Carrier Cloud Ser-vice Platform Development, ” IEICE Transactions on Infor-mation & Systems, Vol.E97-D, No.11, pp.2959-2962, Nov.2014.[30] Y. Yamato, S. Katsuragi, S. Nagao and N. Miura, “ Soft-ware Maintenance Evaluation of Agile Software Develop-ment Method Based on OpenStack, ” IEICE Transactionson Information & Systems, Vol.E98-D, No.7, pp.1377-1380,July 2015.[31] Y. Yamato, Y. Nishizawa, M. Muroi and K. Tanaka, “ De-velopment of Resource Management Server for ProductionIaaS Services Based on OpenStack, ” Journal of InformationProcessing, Vol.23, No.1, pp.58-66, Jan. 2015.[32] Y. Yamato, “ Performance-Aware Server Architecture Rec-ommendation and Automatic Performance VerificationTechnology on IaaS Cloud, ” Service Oriented Computingand Applications, Springer, DOI: 10.1007/s11761-016-0201-x, Nov. 2016.[33] hitoe transmitter SDK web site, https://dev.smt.docomo.ne.jp/?p=docs.api.page&api_name=iot_control&p_name=api_usage_scenario[34] K. Yokoyama and I. Takahashi, "Feasibility Study on Es-timating Subjective Fatigue from Heart Rate Time Series,"IEICE Transactions on Fundamentals of Electronics, Com-munications and Computer Sciences, Vol.96, No.11, pp.756-762, 2013. (in Japanese)[35] Stephen. W. Porges, "Cardiac vagal tone: a physiologicalindex of stress," Neuroscience and Biobehavioral Reviews,Vol.19, No.2, pp.225-233, 1995.[36] Y. Yamato, M. Muroi, K. Tanaka and M. Uchimura, “ Development of Template Management Technology forEasy Deployment of Virtual Resources on OpenStack, ” Journal of Cloud Computing, Springer, 2014, 3:7, DOI:10.1186/s13677-014-0007-3, 12 pages, June 2014.[37] Y. Yamato, “ Automatic verification technology of soft-ware patches for user virtual environments on IaaS cloud, ” Journal of Cloud Computing, Springer, 2015, 4:4, DOI:10.1186/s13677-015-0028-6, 14 pages, Feb. 2015.[38] Y. Yamato, “ Automatic system test technology of user vir-tual machine software patch on IaaS cloud, ” IEEJ Trans-actions on Electrical and Electronic Engineering, Vol.10, Is-sue.S1, pp.165-167, Oct. 2015.[39] Y. Yamato, “ Optimum Application Deployment Technol-ogy for Heterogeneous IaaS Cloud, ” Journal of InformationProcessing, Vol.25, No.1, pp.56-58, Jan. 2017.[40] Y. Nakano, Y. Yamato, M. Takemoto and H. Sunaga, “ Method of creating web services from web applications, ” IEEE International Conference on Service-Oriented Com-puting and Applications (SOCA 2007), pp.65-71, NewportBeach, June 2007.[41] 山登庸次,大西浩行,須永宏,“
Web- テレコム連携のためのサービス実行制御技術の研究開発 , ” 電子情報通信学会論文誌 ,Vol.J91-B, No.11, pp.1417-1427, Nov. 2008.[42] 中野雄介,山登庸次,武本充治,須永宏,“ Web アプリケーション