SICE Annual Conference August -,, Akita Uniersity, Akita, Jaan Non-contact Resiration Measurement Using Structured Ligt -D Sensor Hirooki Aoki, Masaki Miyazaki, Hidetosi Nakamura, Ryo Furukawa, Ryusuke Sagawa and Hirosi Kawasaki 4 Graduate Scool of Information Sciences, Hirosima City Uniersity, Hirosima, Jaan (Tel : +8-8-8-686; E-mail: aoki @irosima-cu.ac.j) Medical Scool, Keio Uniersity, Tokyo, Jaan Intelligent Systems Researc Institute, AIST, Tsukuba, Jaan 4 Graduate Scool of Science and Engineering, Kagosima, Kagosima Uniersity, Jaan Abstract: We ae examined a non-contact resiration measurement using Kinect, tat is D sensor based on structured ligt, in order to decrease te burden of te ulmonary function test. In our roosed metod, te resiratory waeform is roduced from te olume cange of toracoabdominal art of examinee measured by te tree-dimensional measurement function of Kinect. In tis aer, we examine te alidity of our roosed metod by simultaneous measurement wit te exiration gas analyzer and te flow Te exerimental results imlies tat quantitatie resiration measurement witout contact-tye sensor is realized by Kinect. Keywords: Resiration measurement, non-contact ysiological measurement, -D measurement, structured ligt.. INTRODUCTION In te ulmonary function test, te function of resiratory gas excange and air infiltration is examined. Te ulmonary function test by te sirometer is generally conducted. In tis test, te resiratory comromise is diagnosed by te forced exiratory olume.% and % ital caacity for screening te cronic obstructie ulmonary disease (COPD). Te examinees, esecially cildren, women and older eole, sense te restraint sensation, because te moutiece and te nasal lug must be attaced to te face of te test subject under te test. Measurement failures by leaking air-flow aen occasionally. And, it is necessary to cange a moutiece at eery measurement. Terefore, te oeration of te ulmonary function test is difficult in a general medical examination to intend for many subjects. We ae studied te resiration measurement using te dot-matrix attern rojection in order to decrease te burden of te ulmonary function test to te examinee []-[]. In our roosed system, te attern ligt rojector irradiates te torax wit te dot-matrix attern ligt. And te CCD camera takes a time-series images of te dot-matrix attern ligt. Te -D sae of te torax surface can be calculated by te distribution of te dot-matrix attern. Te resiratory waeform is calculated by te time-series olume cange of te body surface [4]. Te cange of te -D sae calculated by our system sould be te alternate index of te ital caacity for te realization of te non-restraint ulmonary function test. We ad examined te relationsi between te resiratory flow olume and te olume cange of te torax surface. In te comaratie exeriments wit te sirometer, we comared te olume cange of te torax surface wit te flow olume. As a result of comaring te olume cange of te torax surface wit te resiratory flow olume, te olume cange is linearly related to te resiratory flow olume [5]. In our roosed metod, te ligt rojector of te dot-matrix attern is consist of te Fiber diffraction Grating (FG), wic is roosed by Macida et al. [6]. Te -D sensor wit te FG are not mass-roduced, and te sensor must be assembled by our and. Te deeloment of sensor by and is unfaorable wit deeloment seed and cost. Difficulty of te deeloment is obstruction of te collaborations wit medical researcers. wile, Microsoft cororation released Kinect wic was te -D sensor by using te structured ligt as like our roosed sensor wit FG in December,. Kinect is a low-riced interface for ome-use game console. Te Software Deeloment Kit for Kinect is reared from Microsoft cororation and we can use Kinect by a PC connected by te USB connector. Kinect attracts attention in arious researc fields, since rocurement and use are easy. We excet tat te COPD screening at te general medical examination and at te ome are realized if a non-contact resiration measurement wit Kinect in substitution for our roosed sensor wit FG is ossible. In tis aer, we roose a new non-contact resiration measurement by using Kinect and examine te alidity of our roosed metod by simultaneous measurement wit te exiration gas analyzer and te flow. METHOD. System configuration In tis researc, te non-contact resiration measurement is executed wit te measurement system sown in Fig.. During te measurement, te examinee sit back so as not to aear te body moement excet te resiratory moement. Kinect is arranged in te front of sitting examinee. Te distance between te bearing surface end and Kinect is set at,mm. Te -64- PR//- 64 4 SICE
installation eigt of Kinect is set at,mm. Te eigt of te bearing surface is 5mm. Kinect is mainly consist of a color camera, an infrared camera, and a attern ligt rojector. Kinect simultaneously acquires det image of QVGA wit color image of VGA. Te det information acquired by Kinect is bits information and te det alue at eac ixel is calculated wit millimeter order. Te secifications of Kinect is sown to Table I [7]. In tis system, Kinect is connected to te PC by USB cable. We deeloed te measurement software wit Kinect for Windows SDK Beta distributed by Microsoft Researc. Kinect sensor Processing unit Examinee. Non-contact resiration measurement by Kinect Te rocedure for measuring resiration witout contact by using Kinect is sown below. First, te examinee sits down in front of Kinect. Ten, te system acquires te det image, and te body area of examinee is detected till te examinee sits down. te skeleton is simultaneously extracted from te detected body area. Kinect ossesses te function for detecting te body area and extracting te skeleton. Te skeleton consists of joints suc as "HEAD", "SHOULDER CENTER", "SHOULDER RIGHT", "SHOULDER LEFT", "ELBOW RIGHT", "ELBOW LEFT", "WRIST RIGHT", "WRIST LEFT", "HAND RIGHT", "HAND LEFT", "SPINE", "HIP CENTER", "HIP RIGHT", HIP LEFT", "KNEE RIGHT", "KNEE LEFT", "ANKLE RIGHT", "ANKLE LEFT", "FOOT RIGHT", "FOOT LEFT" as sown in Fig.. In our roosed metod, te domain contained by "SHOULDER CENTER", "SHOULDER RIGHT", "SHOULDER LEFT", "HIP RIGHT" and "HIP LEFT" is defined as abdominal toracoabdominal domain corresonding to te Triod stand Fig. System configuration. TABLE I SPECIFICATION OF KINECT Item Secification Range of color and det stream. to.5 meters - Range of skeletal tracking. to.5 meters - 4 degrees ertical/ Viewing angle 57 degrees orizontal Mecanical tilt range 7 degrees (ettical) frame rate frames er second Resolution, det stream QVGA( x 4) Resolution, color stream VGA(64 x 48) (i) Color image. SHOULDER RIGHT ELBOW RIGHT WRIST RIGHT HAND RIGHT SPINE HIP RIGHT HEAD SHOULDER CENTER SHOULDER LEFT WRIST LEFT HAND LEFT HIP CENTER HIP LEFT ELBOW LEFT (ii) Det image and detected body. KNEE RIGHT KNEE LEFT ANKLE RIGHT FOOT RIGHT ANKLE LEFT FOOT LEFT (iii) Toracoabdominal domain Fig. Joints of skeleton. Fig. Determination of toracoabdominal domain. -65-
torax art and te abdomen art of te examinee, as sown in Fig.. And, resiratory waeform is calculated by finding a olume cange wit te breating of te toracoabdominal domain. Fig. 4 sows te -D sae of torax surface. Te relationsi between eac ixel in te det image and -D coordinate (X, Y, Z) in te real sace is sown as te next equation based on te triangulation. ( x / ) tan( θ / ) X = ( Y = Z = z / / y ) tan( θ / ) z / z () were x is orizontal coordinate of te ixel, y is ertical coordinate of te ixel, z is det alue of te ixel, is number of orizontal ixels, is number of ertical ixels, θ is orizontal iewing angle of te det sensor, and θ is ertical iewing angle of te det sensor. Te olume of te toracoabdominal domain is found by calculating te numerical integral of -D coordinate corresonding eac ixel in te toracoabdominal domain. And, te resiratory waeform is roduced by calculating te time-series cange of te olume of toracoabdomial domain. Here, -D coordinates in te toracoabdominal domain distribute discretely in te -D sace. Terefore, te -D sae is calculated by alying te Delaunay triangulation wit linear interolation [8].. EXPERIMENT AND RESULT We examined te alidity of our roosed metod by te simultaneous measurement wit te exiration gas analyzer (AEROMONITOR AE-8S manufactured by Minato Medical Science Co., Ltd.). Te exiration gas O Y Z X Fig.4 -D sae of body surface. - - - - - - - - - - 6 8 6 8 6 8 6 8 Fig.5 Resiratory waeform obtained by Kinect..5.5.5.5.5.5 6 8 6 8 6 8 6 8 Fig.6 Results of simultaneous measurement. -66-
analyzer is generally used for te measurement of te resiratory flow olume, and it is widely utilized in te medical institution. Te resiratory flow olume is measured by te exiration gas analyzer, wit te mask-tye sensor attaced on te face of examinee. Examinees are 4 ealty males (age: 4+/-4 years old, body weigt: 7.4+/-6. kg). Prior to te measurement, we obtained te consent document on te measurement execution from te examinees. In te measurement, te examinees wear T sirt. Te measurement time is set at 8 seconds. Examinees cange te resiratory flow olume ariously. Fig. 5 sows resiratory waeforms obtained by Kinect. Te eak-to-eak alue of waeform corresonds to te tidal olume cange wit exiration / te tidal olume cange wit insiration in te toracoabdominal domain. Fig.6 sows te cange of te olume cange obtained by Kinect and te exiratory flow olume measured by te exiratory gas analyzer. Bot are almost similar in cange trend. Fig. 7 sows te correlation between te olume cange by Kinect and te exiratory flow olume by te exiratory gas analyzer. We calculated te regression line and te correlation coefficient. Fitted line is a straigt line troug origin. All gras sow te olume cange is igly correlated wit te air flow olume. Tere was te ig correlation oer 98 between te olume cange and te air flow olume in all examinees. Fig. 8 sows Bland Altman lots about te olume cange and te exiratory flow olume. In eac subject, a sligt addition error was recognized, but te roortion error did not aear. We tink tat te cause of te addition error is te automatic determination of te toracoabdominal domain by Kinect for Windows SDK. It will be necessary to examine te influence of te body-build difference of te subjects in te setting of toracoabdominal domain by te SDK in near future. Moreoer, we make comarison between measurement waeforms of our roosed metod and te flow Te flow meter like te exiratory gas analyzer is used for measuring resiration wit attacing a mask tye flow sensor on te nose and mout. Te flow meter can measure te resiratory flow waeform, unlike te exiratory gas analyzer. Here, we used FM- manufactured by Arco-system ltd. as te flow Te exerimental result for examinee A is sown in Fig. 9. Te dimension of waeform obtained by our roosed metod is litter. Terefore, te dimension of obtained waeform is translated to letter er second tat is te measurement dimension of te flow As te result of te dimension translation, te waeform include noisy comonent, and te resiratory eriodicity is lost on te surface, as sown in Fig.9 (i). Accordingly, te moing-aerage metod is alied to te waeform. Te moing-aerage alue of target datum oint is calculated as te un-weigted mean of Y =. * X R =.9946 ( <.) Y =.5 * X R =.996 ( <.) Y =.968 * X R =.9896 ( <.) Y =.884 * X R =.99 ( <.) Fig. 7. Correlation between air flow olume and olume cange..5 -.5 -.5 -.5 -.5 -.5 -.5 -.5 - +.96SD -.96SD.5.5.5 +.96SD -.96SD.5.5.5 +.96SD -.96SD.5.5.5 +.96SD -.96SD.5.5.5 Fig. 8. Bland-Altman lot about air flow olume and olume cange. -67-
5 datum oints, tat are te target datum oint and te reious 7 and te following 7 datum oints. Te resiratory eriodicity is actualized by alying te moing- aerage metod, as sown in Fig. 9(ii). Fig. 9(iii) sows te result of simultaneous measurement wit our roosed metod and te flow Tere is ig similarity between bot waeforms. Howeer, te waeform of our roosed metod include ig-frequency noise, and tere is artially difference between two waeforms. of toracoabdominal domain of toracoabdominal domain of toracoabdominal domain - - - - - - 6 (i) Dimension translated waeform. 6 (ii) Moing-aeraged waeform. Proosed metod (Kinect) Flow-meter 6 (iii) Comarison between roosed metod and flow Fig.9 Result of comaratie exeriment wit flow 4. CONCLUSION We roosed te non-contact resiration measurement by using Kinect. And, we examined te alidity of our roosed metod by simultaneous measurement wit te exiration gas analyzer. As te results, te olume cange of toracoabdominal surface of examinee obtained by our roosed metod is igly correlated wit te exiratory air flow olume measured by te exiration gas analyzer. Tis result imlies tat our roosed metod realizes quantitatie resiration measurement witout contact-tye sensor. And, as te result of simultaneous measurement wit te flow meter, ig similarity between waeforms obtained by our roosed metod and te flow meter is indicated clearly. Terefore, we exect te realization of low restrictie ulmonary function test, as a substitute of te sirometer, by using Kinect. Te motion-cature function of Kinect become te focus of ublic attention, but te result of tis aer indicate tat Kinect roides -D measurement of ig recision. ACKNOWLEDGEMENT Tis work was suorted in art by Strategic Information and Communications R&D Promotion Programme (SCOPE) No.7, KAKENHI No./No. 7576, Funding Program for Next Generation World-Leading Researcers No. LR, and Grant rograms of Hori Science and Art Foundation in Jaan. REFERENCES [] H. Aoki et al : All-nigt Resiration Monitoring of Elderly Peole by Using New Non-restraint Resiration Monitoring System, Slee and Biological Rytms, Jaanese Society of Slee Researc, Vol.,,.- [] H. Aoki et al : A non-contact and non-restricting resiration monitoring metod for a sleeing erson wit a fiber-grating otical sensor, Slee and Biological Rytms, Jaanese Society of Slee Researc, Vol.,,.49-5 [] H. Aoki et al : Non-restricting Resiration Monitoring System Using Near-infrared Brigt Sots Matrix Irradiation, Tecnical Digest of 4 ICO International Conference Otics & Potonics in Tecnology Frontier, 4,.55-56 [4] Y. Mizobe, H. Aoki and K. Kosiji: Proosal on Nonrestraint Pulmonary Function Test Using Actie D Measurement for Body Surface, World Congress on Medical Pysics and Biomedical Engineering 6, 6,.747-75 [5] Y. Mizob, H. Aoki and K. Kosiji: Basic Study on Relationsi Between Resiratory Flow Volume and Volume Cange of Torax Surface, Proceedings of International Secial Toic Conference on Information Tecnology Alications in Biomedicine 7, 7,.9 - [6] H. Macida et al : Hig-efficiency fiber grating for roducing multile beams of uniform intensity, Al. Ot., Vol., No.,.- (984) [7] Microsoft Researc : Programming Guide - Getting Started wit te Kinect for Windows SDK Beta from Microsoft Researc -,,. [8] D.T. Lee, and B.J. Scacter,: Two Algoritms for Constructing a Delaunay Triangulation,, International Journal of Comuter and Information Sciences, Vol. 9, No., 98,. 9-4 -68-