A Projector-based Movable Hand-held Display System

Transcription

1 A Projetor-base Movable Ha-el Disla Sstem Ma Cue Leu, Kai Ki Lee 2, Ki Ho Wo, Miael Mi Yue Ca 2 Det. of Comuter Siee a Eieeri Det. of Iformatio Eieeri 2 Te Ciese Uiversit of Ho Ko {mleu, wo}@se.u.eu. {lee6, ma}@ie.u.eu. Abstrat I tis aer, we roose a movable a-el isla sstem wi uses a rojetor to rojet isla otet oto a oriar arboar wi a move freel witi te rojetio area. Su a sstem a ive users reater freeom of otrol of te isla su as te viewi ale a istae. At te same time, te size of te arboar a be mae to a size tat fits oe s aliatio. A rojetor-amera air is alibrate a use as te trai a rojetio sstem. We reset a visio base aloritm to etet a oriar arboar a tra its subsequet motio. Disla otet is te re-ware a rojete oto te arboar at te orret ositio. Exerimetal results sow tat our sstem a rojet oto te arboar i reasoable reisio.. Itroutio I tis aer *, we roose a movable a-el isla sstem wi uses a rojetor to rojet isla otet oto a oriar wite arboar. Te setti of our sstem is sow i Fiure. Usuall a erso uses a fixe moitor as te isla evie, or uses a rojetor to rojet te isla otet oto a wall or sree. I tese ases, te islas are at fixe ositios. Alterativel, a a-el isla evie ives users reater freeom of otrol su as te viewi ale a istae. However, te srees of most a-el isla evies are small. Ireasi te size of te srees of tese evies will mae tem eav a ot suitable for a-el urose. Tou eletroi aer is lit, it is still ot available at low ost. So, a alterative solutio is eee. J.Summet a R. Sutaar [] a J. C. Lee et al. [2] ave exlore te use of rojetor to rojet isla otet oto a movable surfae a sowe ifferet aliatios * Tis wor was suorte b a iret Grat of Projet Coe 254 from te Fault of Eieeri, Te Ciese Uiversit of Ho Ko. Wite arboar Projetor-amera air Fiure : Setti of our a-el isla sstem. of su movable a-el islas. J. Koiez et al.[3] also built a rojetio sstem to rojet isla otet oto a flexible surfae. However, all tese aroaes ee to atta sesors oto te movable surfaes. So, seiall mae rojetio surfaes are require. Comuter visio is a more oveiet aroa to etet a tra te movable surfae as we o ot ee to a sesors oto it. Hee, a oriar aer a be use as te movable isla. We use a rojetor-amera air to serve as te iut a outut evies. A oriar wite arboar is use as te movable isla surfae. A omuter visio aloritm is roose to etet a tra te arboar i real-time. Disla otet is rojete oto te arboar. Our aroa a be ivie ito tree mai arts. Te first art is te alibratio of te rojetor-amera air. Tis is to fi te eometri relatiosi of te rojetor a te amera. R. Rasar a P. Bearsle [4] roose a aloritm to alibrate a rojetor-amera air usi te eiolar eometr. Tis meto a ive te itrisi arameters of te rojetor a te relative ose of it wit reset to te amera. However, te alulatio ste is rater omliate. I our sstem, we o ot ee to ow all te arameters. Tus, we roose a simler alibratio meto base o 3D to 2D oit orresoees. Te seo art is te etetio a trai of te arboar. We use lie features extrate from te amera

2 Carboar size, Camera arameters Offlie alibratio of rojetor-amera air imaes to etet te arboar. Te artile filter is use to tra its subsequet motio. Several aers [ ] ave emostrate te robustess of usi Kalma a artile filter i ose trai tass. I tis aer, we roose a quarale etetio a trai aloritm usi lie features a artile filter. Te arboar is etete we it is lae i te view of te amera. Te we ee tra of te ose of te amera to te arboar Te tir art is to rojet te isla otet oto te arboar. Usi te alibratio a trai result, we re-war te isla otet orresoi to te ositio a orietatio of te arboar i te 3D sae so tat it a be rojete oto te arboar orretl. Fiure 2 sows te overview of our roose sstem. I [], te Uiversal Meia Boo also use a omuter visio aroa to tra movable bla aers i te form of a boo a rojet re-ware otet oto tem. Te sstem is iitialize b lai te boo to a ositio aori to a rojete retaular outlie. Feature oit orresoees betwee te framebuffer imae a te rojete imae ature b te amera are use to tra te aer. However, tis meto is uable to tra i-lae rotatios or traslatios of te isla lae. Our roose aroa oes beo te Uiversal Meia Boo i two was: () we o ot ee a iitializatio ste; (2) our aroa suorts bot i-lae a out-of-lae rotatios a traslatios of te isla lae. 2. Projetor-amera air alibratio I our sstem, we use a alibrate amera, so te itrisi arameters A of te amera are ow. We ee to rojet te isla otet oto te wite arboar. Beause te rojetor a amera o ot sare te same rojetio eter, tere is a relative ose betwee tem. I orer to rojet o te arboar reisel, we ee to alibrate te rojetor-amera air a fi teir relatiosi first. 2.. Projetive eometr of a rojetor G Real-time quarale etetio a trai Pre-war isla otet Pre-ware isla otet Projetio Camera ose Fiure 2: Overview of our roose sstem. Te 2D-3D rojetio moel of a rojetor is similar to tat of a amera. I a ieal situatio, it is ersetive rojetio. Te ifferee betwee a rojetor a a amera is te iretio of rojetio. I our roose sstem, we efie te worl ooriate sstem as te sstem of te amera. Ea 3D oit i te sae is exresse as [ X Y Z ] T P If we treat te rojetor as a amera, P will be rojete oto a oit I o te imae lae of te rojetor. P a I are relate b te rojetio matrix G wi esribes te itrisi a extrisi matrix of te rojetor. G 2 f u 2 f v 3 4 u v [ R T ] Here, f u a f v are te foal let of te rojetor, a ( u, v ) is te otial eter. R a T are te relative rotatio a traslatio of te rojetor to te amera. A 3D oit i te amera ooriate sstem is relate to a 2D imae oit of te rojetor b te relatiosi liste i (3). I G P I our sstem, we ee to fi G, but it is ot eessar to fi te itrisi a extrisi matries. So, a simler alibratio meto a be use Calibratio meto Te mai iea of te alibratio meto is to fi 3D to 2D orresoees i te amera a rojetor ooriates resetivel. We use a arboar wit ow size as te rojetio surfae. Te a ross at imae oit I [u v ] T is rojete oto te arboar. Te imae is ature b te amera. Sie te arboar is a alibrate objet, we a easil fi te 3D ositio of te four orers i amera ooriate b Za s meto []. Kowi te 3D ositios of te 4 orers, we a ostrut te lae equatio of te arboar. We eote te lae as (4). Te 3D ositio of te ross mar rojete oto te arboar is eote as P i (). Substituti X, Y a Z i (4) b X, Y, a Z resetivel, we et (5). ax + by + Z + ax + by + Z + At te same time, tis oit is ature i te amera imae at a oit I [u v ] T. Usi te itrisi arameters of te amera A, we ave () (2) (3) (4) (5)

3 [ ] T (3) (a) (b) Fiure 3: (a) Detete lie semets usi Hou Trasform; (b) Detete quarale. U V S X A Y Z f u f v u X v Y Z f u a f v are te foal let a ( u, v ) is te otial eter of te amera. From (6), we a et te 2D imae oit as U fux + uz u S Z V fvy + vz v S Z Cosieri equatios (5), (7) a (8), we ave 3 uows a 3 ostrais. So, P a be solve. From (3), P a I are relate b G. So we ave U V S X Y Z B osieri te imae ooriate, we ave v u U S V S X + 2Y + 3Z + 4 X + Y + Z + 2 X X + Y + + Y + Z Z + + Divii () b () a rearrai te equatio, we et (2). v ( X + Y 2 + Z 3 + 4) u ( X + Y + Z + 2 ) B olleti ( >8) umbers of su 3D to 2D orresoi oits, we a rearrae te equatios ito te form B, were B is a x8 matrix. Te 8x vetor as sow i (3) a be omute b usi Siular Value Deomositio of B ito BUDV T. Similarl, eq. () a be rearrae to form (4). (6) (7) (8) (9) () () (2) u ( X X + Y + Z + ) + Y 2 + Z 3 +,, a a be omute b usi 2 ( 2 >4) oit orresoees. I tis wa, G a be full alibrate. 3. Quarale etetio a trai I our sstem, we roose a real-time quarale etetio a trai aloritm to etet a tra te wite arboar. 3.. Automati quarale etetio Lie semets are use i our sstem as te feature for bot etetio a trai roesses. A set of lie semets L {l, l 2,, l m } are extrate from te imae usi te lie etetor base o Hou Trasform i OeCV [2]. Fiure 3(a) sows a imae wit etete lie semets. From te lie semets obtaie, we oose a set of lo oes to be examie. Four lies are osiere as formi a quarale if te fulfill te followi riteria: Ea sie lie is loer ta a tresol value; Oosite sies are of similar lets; Ea ale is witi te rae from 3 to 5 ; Te overlai ratios at ea sie are at least.7. Defiitio of te overlai ratios at ea sie is as follows: s : let of te etete lie semet; s 2 : let of te sie lie of te forme quarale; s 3 : Overlai let of te lie semet a te sie lie. Te two overlai ratios assoiate wit ea sie are s 3 r 2 s Te last riterio is eessar as it esures tat te quarale etete is iee rereseti a real objet. Fiure 3(b) sows te etete arboar wit ree lies o te borers Quarale trai usi artile filter From te etetio result, we ave 2D ositios of te four orers of te quarale. Sie we are usi a arboar wit ow size, a te amera is alibrate, we a omute te relative rotatio a traslatio of te amera to te arboar. Te, our trai state at frame is reresete b tis relative ose i te followi form r 4 s s 3 2 (4) (5)

4 Iitial samle set Re-rojetio usi ose Closest lie mati Criteria ei Cotet relaemet Weit uate Samle uate b raom wal q [ r r r t t t ] x Were r x, r a r z are te rotatio ales alo te x,, a z axis resetivel. A t x, t a t z are te traslatio alo te x, a z axis resetivel. Partile filter is use to estimate te osterior esit for tis amera ose. Te ose is reresete as a set of isrete artiles. Ea artile as a weit to iiate ow ofiet it is to rereset te amera ose. Te two mai omoets of a artile filter are te state ami moel a te observatio moel. Te state ami moel etermies ow te artiles roaate from frame to frame. Tat is, efii te esit (q q - ). Te observatio moel etermies ow weits are assie to artiles rovii te observatio at tat frame. Let te observatio i te urret frame be a te sequee of observatio u to te urret frame be :, te artile filter [3] erforms te trai b reursivel aroximati te osterior esit (q : ). Te outut of te artile filter is a set of te weite samles S S { q w },..., { q w } were S is te artile umber. Te flow iaram te artile filter use i our rojet is sow i Fiure State ami moel. Sie te quaraular surfae is i free motio, a simle raom wal moel base o a uiform esit U about te revious state [5] is use. Te variable e reresets te uertait about te movemet of te etete quarale. ( q q ) U ( q e, q z Fail + e x Webam Lie features extrate usi Hou Trasform Observatio Observatio moel State Dami Moel Fiure 4: Flow iaram of te artile filter use i our rojet. z ) (6) (7) Observatio moel. Te observatio i our aloritm is (8). It reresets te lie semet feature ma obtaie from te urret imae frame. { l l l } L,...,, 2 To evaluate te lielioo of ea artile, we first re-rojet te four orers of te quarale oto te imae lae aori to te ose reresete b te artile. Te, a lie mati aloritm base o oit-lie istaes a aular ifferees is use to fi te best mati lies to te four sie lies of te quarale i te urret frame. Te, if all sie lies of te quarale reresete b te artile a mat to a lie semet, we a form a quarale b te mate lie semets. Tis quarale is ee usi te first tree riteria i te quarale etetio aloritm. If a of te sie lies aot mat to a lie semet i te lie feature ma or te forme quarale aot ass te ei roeure, a ver small weit will be assie to te artile. Oterwise, te artile a mat to a real quarale i te urret imae. To ive a more reise trai result, we itroue a relaemet seme ito our observatio moel. For artiles tat asse te lie mati a quarale riteria ei roeures, te four mate lies form a real quarale i te see. Te, te four orers of te quarale are alulate from te four lies. Tese orers will te relae te oes i te artile. I tis wa, all artiles wi survive from te evaluatio roeures will rereset a real quarale i te see. Sie te orers are alulate from lies, te trai reisio a be i sub-ixel level. Te lielioo is te alulate aori to te sum of overlai ratios: ( q ) 4 t m r r t t 2 Te fial weits of te artiles are ive b ormalizi te sum of te weits to : w S ( ( q ) q ) 4. Projetio oto te arboar From te trai result of setio 3, we ow te relative ose of te amera to te arboar at ea frame. From tis trae ose, we a omute te 3D loatio of te four orers of te arboar i amera ooriate. We eote te four 3D orers as: Q [ Q Q Q Q ] (8) (9) (2) (2) B te rojetio matrix G alibrate i setio 2, we a fi te four orresoi 2D oits o te rojetor

5 Fiure 5: A rojetor a a webam are fixe toeter to form a rojetor-amera air. imae lae aori to equatios (9) to (). We eote te 2D orers as te vetor i i (). i [ i i i i ] B treati te rojetor as a amera, te arboar wit te four orers Q are rojete oto te quaraular area forme b te four imae oits i. B reversi te roess, tis meas tat isla otet witi te quaraular area forme b i will be rojete oto te arboar. For ea frame, we a omute te four orers of te quaraular area from te arboar trai result. Te isla otet is re-ware ito te quaraular area before rojetio. I orer to erform te re-wari roeure, we ee to fi te omora matrix H H were Te omoeous form of te ooriates of te soure orer oits is eote b J [ j j2 j3 j4], rereseti te uer left, uer rit, lower left a lower rit orer of te sree resetivel. Te vetor C is te omoeous form of i. Te omora matrix H a be solve b (25). H CJ C HJ T JJ T Trou te above roeures, te isla otet a be rojete oto te arboar orretl. Te arboar will at as a lit movable a-el isla. 5. Exerimetal results We ave built a rotote sstem. I te sstem, a rojetor wit resolutio 28x a a webam wit resolutio x are fixe toeter to form a rojetor-amera air as sow i Fiure 5. We use a wite arboar wit size of 35mm x 3mm as te isla ( ) () () () (25) Fiure 6: Trai result of our aloritm usi lie features a artile filter wit olusio. Fiure7: Trai result i ese lutter. surfae. We alibrate te rojetor-amera air aori to te alibratio meto esribe i setio imaes are ature a use to alibrate te trasformatio matrix. A omuter wit 2.6GHz ual ore roessors agb memor is use as te testi latform for our exerimets. For te trai art, we set te umber of artile to be 8 a aieve a stable trai result wit roessi time about 3ms. For te wole trai a rojetio roess, te roessi time is about 5ms. So, our sstem a aieve a frame rate of about 2 wi is oo eou for a real time aliatio. We arrie out several exerimets to test te robustess of te etetio a trai aloritm uer ifferet situatios su as olusio a ese lutter. Fiure 6 sows some itures ature from te trai roess. Te first row sows te istributios of te re-rojetio result of te artiles. Te seo row sows te trae arboar. Tis exerimet sows tat our trai aloritm wors well uer severe olusio. Fiure 7 sows te trai result we te barou is i ese lutter. Tere are ma strait lies at te barou tat ma affet te trai result. However, te result sows tat our aloritm a tra te arboar reisel. To test te rojetio erformae, we ave arrie out a umber of real exerimets. I te first exerimet, we rojet a stati imae oto te arboar wi is move freel witi te rojetio area. Fiure 8 sows 4 imaes ature uri te exerimet. Te result sows tat te imae a be re-ware a rojete oto te arboar reisel. I te seo exerimet, we tr to rojet a movie li oto te arboar to test te stabilit of our meto i ami rojetio otet. Fiure 9 sows 4 imaes ature uri te exerimet. Te result sows

6 tat our sstem wors well wit ami rojetio otet. Fiure sows aoter two exerimetal results i ifferet barous. 6. Colusio We ave roose a rojetor-base a-el isla sstem usi omuter visio. A rojetor-amera air is use as te iut a outut evie. A simle alibratio aloritm is itroue i tis aer to fi te 3D to 2D trasformatio matrix of te amera a rojetor ooriates. Lie features are use to etet a wite arboar a artile filter is use to tra te amera ose relative to te arboar. Usi te alibratio a te trai result, isla otet a te be re-ware a rojete oto te arboar orresoi to its 3D loatio. No sesors are eee to be ae oto te arboar i our aroa. Tus, a oriar arboar a be use. Exerimetal results sow tat our sstem a suessfull rojet isla otet oto te arboar wit reasoable reisio a stabilit. Referees Fiure8: Exerimetal result wit a stati imae. [] J. Summet a R. Sutaar, Trai Loatios of Movi Ha-Hel Dislas Usi Projete Lit, Proeeis of Pervasive 25. Mui, Germa [2] J. C. Lee et al. Moveable Iterative Projete Dislas Usi Projetor Base Trai, Proeeis of te ACM Smosium o User Iterfae Software a Teolo, Otober 25, [3] J. Koiez et al. A Hael Flexible Disla Sstem, Proeeis Visualizatio [4] R. Rasar a P. Bearsle, A Self-Correti Projetor. Proeeis of Comuter Visio a Patter Reoitio (CVPR) 2. [5] M. Puilli a A. Calwa, Real-Time Camera Trai Usi a Partile Filter, Pro. Britis Maie Visio Coferee, 25. [6] M. Puilli a A. Calwa, Real-Time Camera Trai Fiure 9: Exerimetal result wit ami rojetio otet. Fiure : Oter exerimetal results i ifferet barous. Usi Kow 3D Moels a a Partile Filter, 8t Iteratioal Coferee o Patter Reoitio (ICPR'6). [7] F. Ababsa a M. Mallem, Robust lie trai usi a artile filter for amera ose estimatio, Proeeis of te ACM smosium o Virtual realit software a teolo, 26, [8] Y. K. Yu, K. H. Wo, S. H. Or a J. Z. Ce, "Cotrolli Virtual Cameras Base o a Robust Moel-free Pose Aquisitio Teique", IEEE Trasatios o Multimeia, Vol., No., PP. 84-9, Ja 29. [9] Y. K. Yu, K. H. Wo, S. H. Or, a M. M. Y. Ca, "Robust 3-D Motio Trai From Stereo Imaes: A Moel-Less Meto", IEEE Trasatio o Istrumetatio a Measuremet, Vol. 57, No. 3, PP. 6-63, Mar 28. [] S. Guta, C. Jaes, Te Uiversal Meia Boo: Trai a Aumeti Movi Surfaes wit Projete Iformatio. IEEE/ACM Iteratioal Smosium o Mixe a Aumete Realit (ISMAR), 26, [] Z. Za. A flexible ew teique for amera alibratio. IEEE Trasatios o Patter Aalsis a Maie Itelliee, (): 4, 2 [2] Itel, Oe Soure Comuter Visio Librar, tt:// [3] S. Arulamalam et al, A tutorial o artile filters for o-lie/o-aussia Baesia trai, IEEE Tras. o Sial Proessi, 5(2): 74-88,