REAL-TIME TRACKING OF NON-RIGID OBJECTS USING MODIFIED KERNEL-BASED MEAN SHIFT AND OPTIMAL PREDICTOIN

Transcription

1 REAL-TIME TRACKING OF NON-RIGID OBJECTS USING MODIFIED KERNEL-BASED MEAN SHIFT AND OPTIMAL PREDICTOIN A Merat Sarf Uversty of Tecology Departet of Electrcal Egeerg P.O.Box , Tera, Ira erat@er.sarf.ed Sore Kasae Sarf Uversty of Tecology Departet of Copter Egeerg P.O.Box , Tera, Ira skasae@sarf.ed ABSTRACT A effcet scee for real-te color-based trackg of o-rgd objects s proposed. Te cetral coptatoal odle s based o ea sft teratos. It coptes te ost probable target posto te crret frae, wle te predcto of te ext target locato s copted sg a Kala flter. Te dsslarty betwee te target odel ad te target caddates s expressed by a etrc based o te Battacaryya coeffcet. I ts work, we ave adapted te kerel profle (sed calclatg te featre stogra) wt a bary ask geerated by proposed adaptve backgrod sbtracto scee. Te odfed kerel calclates te featre stogra oly for foregrod pxels ad prevets backgrod pxels fro casg te estato process to devate. Te adaptve backgrod sbtracto algort ay fal der varyg llato ad sadow codtos. To overcoe ts proble, we ave decoposed te cog age to ts trsc copoets (llace ad reflectace), ad ave desged a adaptve backgrod sbtracto scee sg te reflectace age. Te experetal reslts sow te capablty of te proposed tracker to adle real-te partal occlsos, sgfcat cltter, ad also target scale varatos.. INTRODUCTION Object trackg s a task reqred by dfferet copter vso applcatos, sc as perceptal ser terface [3], tellget vdeo copresso [7], ad srvellace []. To aceve robstess to ot-of-plae rotatos of te target, te color dstrbto of te target odel s eployed stead of te raw age pxels. Te locato of te target te ew frae s predcted based o te past trajectory ad te a searc s perfored ts egborood to detere te age regos (target caddates) wose dstrbto s slar to tat of te odel. I sgle ypotess trackg scees te best atc deteres te ew locato estato; owever, ore coplex strateges also exst to for ltple ypoteses []. Te exastve searc te egborood of te predcted target locato for te best target caddate s, owever, a coptatoally tesve process. As a solto to ts proble, a color-based trackg etod based o ea sft teratos [4, 5] s proposed. Ts etod works real te; as t s based o te gradet ascet optzato rater ta te exastve searc. Te easreet vector s derved based o ea sfts, wle te predcto of te ext target locato s copted by a Kala flter. Fgre sows te block dagra of te a coptatoal odles of te proposed trackg algort. Te fast target localzato s based o te ea sft teratos ad te state predcto sg Kala flterg. Te oto of te target s assed to ave a velocty tat dergoes slgt cages, odeled by a zero-ea wte ose tat affects te accelerato. Fgre. Block dagra of te a coptatoal odles of te proposed trackg algort.

2 I ay statos, de to te slarty betwee te target ad backgrod colors, te ea sft vector devates to a backgrod rego. Ts, ere we se a bary ask to odfy te kerel sed [4] (calclatg te wegted stogra of te regos) to prevet te ea sft vector to devate to te backgrod regos. Te orgazato of paper s as follows. Secto presets te eployed slarty easre, te ea sft-based target localzato wtot kerel odfcato ad te Kala flter. Secto 3 dscsses te proposed algort wc cossts of te kerel odfcato ad te llato varace backgrod sbtracto processes. Experetal reslts are descrbed Secto 4. Te paper s coclded Secto 5.. KERNEL-BASED MEAN SHIFT TRACKING.. COLOR BASED SIMILARITY MEASURE Gve te predcted locato of te target crret frae ad ts certaty, te easreet task asses te searc of a cofdece rego for te target caddate tat s te ost slar to te target odel. Te developed slarty easre s based o color forato. Te featre z represetg te color of te target odel ad s assed to ave a desty fcto q z, wle te target caddate cetered at locato y as te featre dstrbted accordg to p z (. Now, te proble s to fd te dscrete locato y wose assocated desty p z ( s te closest to te target desty q z. Or easre of te dstace betwee te two destes s based o te Battacaryya coeffcet, wose geeral for s defed by []: ρ ( ρ[ p(, q] p ( q dz. () z z Propertes of te Battacaryya coeffcet sc as ts relato to te Fser easre of forato, qalty of te saple estate, ad ts explct fors for dfferet dstrbtos are dscssed [6, ]. Te dervato of te Battacaryya coeffcet fro saple data volves te estato of te destes p ad q, for wc te stogra as bee eployed. Te dscrete desty q ˆ { q ˆ } (wt ˆ... q ) s estated fro te -b stogra of te target odel, wle p ˆ ( { p ˆ ( } (wt p ˆ... ) s estated at a gve locato y fro te -b stogra of te target caddate. Terefore, te saple estate of te Battacaryya coeffcet s gve by: ˆ( ρ[ˆ p(, ] (. ρ () Based o eqato () te dstace betwee two dstrbtos s defed as: ( ρ[ (, ]. d (3) Te statstcal easre (3) s a etrc vald for arbtrary dstrbtos, beg early optal (de to ts lk to te Bayes error []) ad varat to scale of te target. It s terefore speror to oter easres sc as stogra tersecto [], Fser lear dscrat [9], or Kllback dvergece... TARGET LOCALIZATION Ts secto explas ow to effcetly ze (3) as a fcto of y te egborood of a predcted locato. I cotrast to object trackg based o exastve searc a cofdece rego [, 8, ], te optzato trog ea sft teratos s faster sce t explots te spatal gradet easre (3).

3 ... Wegted Hstogra Coptato x.let b : R {... } be te I) Target Model: Te pxel locatos of te target odel cetered at are deoted by * { }... * fcto tat assocates to te pxel at locato x ad te dex b( x * ) of te stogra b correspod to te color of tat pxel. Te probablty of te color te target odel s derved by eployg a covex ad ootoc decreasg fcto k : [, ) R wc assgs a saller wegt to te locatos tat are farter fro te target ceter. Ts wegt creases te robstess of te estato; sce te perperal pxels are te least relable, beg ofte affected by occlsos (cltter) or backgrod areas. By assg tat te geerc coordates x ad y are oralzed by x ad y, respectvely, we get: C k * [ b( x ) ] * x δ (4) were δ s te Kroecker delta fcto. Te oralzato costat C s derved by posg te codto q ˆ C k x *, te sato of delta fctos for..., beg eqal to oe., fro were: II) Target Caddates: Let deote te pxel locatos of te target caddate, cetered at y te crret frae, by{ x }.... Eployg te sae wegtg fcto k, te probablty of occrrece of te color te target caddate s gve by: ( ) C k y x y δ [ b( x ) ]. (6) Te scale of te target caddate (.e., te ber of pxels) s detered by te costat wc plays te sae role as te badwdt (rads) te case of te kerel desty estato [5]. By posg te codto tat p ˆ, we obta te oralzato costat as: C y x k Note tat C does ot deped o y; sce te pxel locatos x are orgazed a reglar lattce ( y deotes te lattce ode). Terefore, tec ca be precalclated for a gve kerel wt dfferet vales of. (5) (7)... Dstace Mzato Te searc for te ew target locato te crret frae starts at te predcted locato ŷ of te target copted by te Kala flter (Fgre ). Ts, te color probabltes { p ˆ (ˆ y )} of te target caddate at locato ŷ... crret frae ave to be copted frst. Te zato of te dstace (3), beg eqvalet to te axzato of te Battacaryya coeffcet (), s started wt te Taylor expaso of ˆ( ), ˆ], wc yelds: ˆ ρ[ p y q arod te vales p (ˆ y )

4 [ˆ( p, ] + ( y ) ρ (ˆ )ˆ ˆ y q p ( (8) By sbstttg (6) (8) we get: [ˆ( p, ] C y x + (ˆ y ) w k ρ (9) were, w (ˆ y [ b( x ) ] δ (). ) Hece, to ze te dstace (3), te secod ter eqato (9) as to be axzed (te frst ter s depedet to. Te secod ter represets te desty estate copted wt kerel profle k at y te crret frae, wt te data beg wegted by w (). Te axzato ca be effcetly aceved based o te ea sft teratos (see [5]), sg te followg algort. To axze te Battacaryya coeffcet ρ [ (, ], gve te dstrbto { q ˆ } of te target odel ad te predcted... locato ŷ of te target:. Copte te dstrbto{ p ˆ (ˆ y ) }, ad evalate:... ρ [ p(ˆ y ), ] ( y ). w.... Derve te wegts { } 3. Derve te ew locato of te target [5]: y ˆ yˆ x xw g yˆ x w g p ˆ (ˆ) y... Update { } accordg to ().., ad evalate: ρ [ (ˆ y ), ] ( y ) ˆ. q 4. Wle ρ [ (ˆ y ), ˆ] [ˆ(ˆ ), ˆ q < ρ p y q], Do: y ˆ (ˆ ˆ y +. y ˆ yˆ < 5. If ε, stop ˆ y Oterwse, set yˆ ad go to step. Te above optzato eploys te ea sft vector Step 3 to crease te vale of te approxated Battacaryya ~ coeffcet ρ ( y ). Sce ts operato does ot ecessarly crease te vale of ˆ ρ (, te test clded Step 4 s eeded to valdate te ew locato of te target. However, practcal experets (trackg dfferet objects, for log perods of te) sowed tat te Battacaryya coeffcet copted at locato defed by eqato () was alost always lager ta te coeffcet correspodg

5 to ŷ. Less ta.% of te perfored axzatos yelded cases were Step 4 was ecessary. Te terato tresoldε sed Step 5 s derved by costrag te vectors represetg ŷ ad ŷ to be wt te sae pxel...3. Measreet Ucertaty Te certaty te target localzato aly cased by te age ose, te slarty betwee te target colors ad te backgrod/cltter colors, ad te percetage of occlso. However, te pertrbato sorces also flece te ax vale of te Battacaryya coeffcet ad te crvatre arod te ax. Sce tese two paraeters (te ax vale ad te crvatre arod ax) ca be evalated real te, a lookp-table tat relates te ax vale ad te srface crvatre to certaty locato estate trog te Mote-Carlo slatos as bee derved. As a reslt, after eac ea sft optzato tat gves te target easred locato, te certaty of te estato ca be copted..3. KALMAN PREDICTION Te proposed tracker eploys two depedet Kala flters, oe for eac of te x ad y drectos. Te target oto s assed to ave slgtly cagg velocty ([, p.8]) odeled by zero-ea, low varace (.) wte ose tat affects te accelerato. Te trackg process for eac frae cossts of rg te ea sft based optzato (wc deteres te easreet vector ad ts certat, followed by te Kala terato (wc gves te predcted posto of te target ad a cofdece rego). Tese ettes are sed tr to talze te ea sft optzato for te ext frae. 3. PROPOSED ALGORITHM 3.. KERNEL MODIFICATION Te above etoed etod for real te trackg asses tat te Battacaryya srface fored fro te Battacaryya coeffcets pxels arod te predcted locato s soot ad as oe ax wc correspods to te real locato of te target. Bt t s see te experets tat de to te slarty of te backgrod colors (stogra) to te target colors, te Battacaryya srface as ore ta oe ax ad soetes te vale of te oter axs s larger ta te ax real locato. Ts cases te ea sft to coverge to a local ax. I ts work, we ave odfed te kerel sed Secto to calclate te stogra of te odel ad te caddate, so tat t prevets backgrod pxels to be copted te stogra. I order to odfy te kerel, a bary ask tat s obtaed by a adaptve backgrod sbtracto s sed. Ts bary ask specfes weter a pxel belogs to te backgrod or te foregrod. By sg ts bary ask, te caddate rego stogra s oly copted foregrod pxels prevetg backgrod pxels to be copted stogra ad also te devato of ea sft teratos to a local ax te Battacaryya srface. Te bary ask te t frae s gve by: Bary Bary _ Mask : R {,} () Mask ( x ) f x foregrod f x backgrod _ () I order to se ts bary ask, te forer kerel s ltpled by bary ask defed above. Te ew odfed kerel wegts te pxels te caddate rego ot oly de to te dstace to te rego ceter, bt also de to te lkelood of te foregrod pxels. By ts ltplcato, te target odel stogra wll be: * * Ck( x ) δ [ b( x ) ] Bary_ Mask ( x ) (3) wc te oralzato costat (C) s gve by: C * k( x ) Bary _ Mask ( ), x Te stogra of te caddate rego s copted by: (4)

6 y x ( C k were, C δ [ b( x ) ] Bary _ Mask ( x ), (5), _ ( ) y x k Bary Mask x I coptg te ew locato, te wegts w are gve by: (6) w ˆ [ b( x ) ] Bary _ Mask( x ) p (ˆ ) y δ (7) Oter forlas wll be te sae as Secto. Te teratos are perfored sg te ew forlas. 3.. ILLUMINATION INVARIANCE BACKGROUND SUBTRACTION Te bggest proble wt backgrod sbtracto teds to be ts falre der codtos of varyg llato ad sadows. To overcoe ts proble, we propose to decopose a cog age to ts trsc copoets; llace ad reflectace []. A backgrod sbtracto rote tat s very easly fooled by weater/llato cages wold be brttle f t were appled o a trsc age as opposed to te orgal age. However, recoverg te trsc copoets of a sgle color age s a ll-defed proble [3]. I ost practcal srvellace applcatos, t s jstfable to asse tat te scee llace vares sootly. Sce te ew age odel s reqred to be varat ot oly to te global cage lgtg bt also to te soot varatos of te dstrbto, te llace copoet sold be acqred as local as possble. Terefore, a local Gassa low-pass flter s eployed or fraework descrbed below. Ts process s kow as te ooorpc flterg [4] ad reslts separatg te reflecto copoets fro te age. Deotg te pt age by I ( x,, te reflectace age by R( x, ad te llato age by L ( x,, we ave: Te: ad, also, I ( x, L( x, R( x,. (8) l I ( x, l L( x, + l R( x, (9) l L G3 * 3 l I () r () exp(li l L) were G 3* 3 s te 3x3 Gassa ask, I s te age, ad deotes te covolto operato. Applyg () to eac color cael yelds: r exp(l I l L) () r r R G exp(l I R (3) B exp(l I G B l L) (4) l L)

7 Cobg te reflectace copoets of all tese caels, we get te reflectace age correspodg to te orgal color age. After ts preprocessg step, we apply te reglar adaptve backgrod sbtracto tecqe. 4. EXPERIMENTAL RESULTS Te proposed algort was sed to track te objects cotaed te test seqeces of or database. Te database cossts of 5 color seqeces cotag ore ta srvellace objects (as ad vecles). Te sze of seqeces fraes s We ave r te algort sg Vsal C++ o a. GHz PC. A typcal seqece s sow Fgre. It s wort to eto tat oter seqeces reslted slar reslts. I ts seqece te pedestras are tracked. To so ts, frst a rectaglar patc s selected as te odel of a pedestra. Te target stogra s derved te RGB space wt bs. Te algort rs cofortably at 3 fps. Ts fgre sows 3 saples fro a seqece (fraes,, 7) sg ea sft trackg wtot kerel odfcato by bary ask (a) ad wt kerel odfcato by bary ask (b). As t s see, de to te slarty betwee te backgrod colors (stogra) ad te target colors (stogra), te rectaglar patc red left (a) s devated to soe oter place sadow. Bt de to te kerel odfcato by bary ask, ts devato does ot occr (b). Fgre 3 sows te Battacaryya srface for pxels arod te predcted locato frae for two cases wt kerel odfcato (a) ad wtot kerel odfcato (b). As see Fgre.3, te srface for te case wtot kerel odfcato by bary ask (b) as ay local axa ad te global ax does ot correspod to te real locato of te target. I cotrast to ts case, te srface obtaed for te case wt te kerel odfcato by te bary ask as a global ax tat correspods to te real locato of te target ad te srface as soe sall vales wc correspod to te backgrod. Or algort perforace s c better coparso srvellace trackg algorts [, 5]. 5. CONCLUSION I ts paper, we proposed a trackg algort tat proves te ea sft object trackg perforace by odfcato of te kerel sg a bary ask obtaed fro or proposed adaptve backgrod sbtracto scee. As a reslt, te probablty of devato of te algort to soe local axa as bee decreased. Experetal reslts sow te speror perforace of te proposed algort for srvellace applcatos. I sc applcatos de to ll llato codtos, te stogra of soe correct locatos s very slar to tat of te target stogra ad ts te proposed etod ca effcetly track te real locato of te objects. Also, order to ake te backgrod sbtracto scee ore robst der varyg llato, te ooorpc flterg for recoverg te reflectace age s sed tat frter proved te perforace.

8 (a) (b) Fgre.Trackg a pedestra wtot kerel odfcato (a) ad wt kerel odfcato (b). (a) (b) Fgre 3.Battacaryya srface for cases wt kerel odfcato (a) ad wtot kerel odfcato (b).

9 ACKNOWLEDGMENT Ts work was part spported by a grat fro ITRC. 6. REFERENCES [] Y. Bar-Salo, T. Forta, Trackg ad Data Assocato, Acadec Press, Lodo, 988. [] S. Brcfeld, Ellptcal Head Trackg sg Itesty Gradets ad Color Hstogras, IEEE Cof. o Cop. Vs. ad Pat. Rec, Sata Barbara, 3 37, 998. [3] G.R. Bradsk, Copter Vso Face Trackg as a Copoet of a Perceptal User Iterface, IEEE Work. O Applc.Cop. Vs., Prceto, 4 9, 998. [4] D. Coac, V. Raes, P. Meer, Real-Te Trackg of No-Rgd Objects sg Mea Sft, To appear, IEEE Cof. o Cop. Vs. ad Pat. Rec., Hlto Head Islad, Sot Carola,. [5] D. Coac, P. Meer, Mea Sft Aalyss ad Applcatos, IEEE It l Cof. Cop. Vs., Kerkyra, Greece, 97 3, 999. [6] A. Djoad, O. Sorraso, F.D. Garber, Te Qalty of Trag-Saple Estates of te Battacaryya Coeffcet, IEEE Tras. Patter Aalyss Mace Itell,. :9 97,99. [7] A. Elefterads, A. Jacq, Atoatc Face Locato Detecto ad Trackg for Model-Asssted Codg of Vdeo Telecoferece Seqeces at Low Bt Rates, Sgal Processg- Iage Cocato, 7(3): 3 48, 995. [8] P. Fegt, D. Terzopolos, Color-Based Trackg of Heads ad Oter Moble Objects at Vdeo Frae Rates, IEEE Cof. o Cop. Vs. ad Pat. Rec, Perto Rco, 7, 997. [9] K. Fkaga, Itrodcto to Statstcal Patter Recogto, Secod Ed., Acadec Press, Bosto, 99.T. Kalat, Te Dvergece ad Battacaryya Dstace Measres Sgal Selecto, IEEE Tras. Co. Tec., COM-5:5 6, 967. [] T. Kalat, Te Dvergece ad Battacaryya Dstace Measres Sgal Selecto, IEEE Tras. Co. Tec., COM-5:5 6, 967. [] A.J. Lpto, H. Fjyos, R.S. Patl, Movg Target Classfcato ad Trackg fro Real-Te Vdeo, IEEE Worksop o Applcatos of Copter Vso, Prceto, 8 4, 998. [] H.G. Barrow ad J. Teeba, Recoverg Itrsc Scee Caracterstcs fro Iages, Acadec press 978. [3] E.H. Lad ad J.J. McCa, Lgtess ad Retex Teory, Joral of te Optcal Socety of Aerca, 6 -, (97). [4] Adrzej J. Kassk ad Alaa M. Hady, Segetato based o ooorpc flterg ad proved seeded rego growg for oble robots trackg age seqeces, Mace Grapcs & Vso Iteratoal Joral,, , (). [5] O. Masod, ad N. P. Papakolopolos, "A ovel etod for trackg ad cotg pedestras real-te sg a sgle caera". IEEE Tras. Veclar Tecology, 5:67-78,.