Mapping Tridiagonal Solvers to Linear Recurrences

Transcription

1 Mppg Trgol Solvers to Ler Reurrees L- We Chg We- me Hwu {lhg w- IMPACT Tehl Report IMPACT- - Uversty of Illos t Ur- Chmpg Ceter for Relle Hgh- Performe Computg Septemer 8 Revso: Septemer Astrt I ths report we summrze estg prllel lgorthms for trgol solvers propose ovel trgol solver lgorthm lle LUL- UBD lgorthm. We pot out ll estg lgorthms e vewe s ler reurrees or eteso of ler reurrees. We further te eessry optmzto strteges for hgh- performe mplemetto of eh lgorthm SIMD rhtetures. We lso lyze the performe of estg lgorthms usg omputto omplety the requre umer of prllel steps the requre umer of vso opertos the umer of memory ess requests. Our propose lgorthm hs the mml umer of memory ess requests the mml umer of vso opertos ler omputtol omplety. Key terms: Cyl Reuto Blo Cyl Reuto Prllel Cyl Reuto SPIKE lgorthm Trgol Solvers LU- eomposto Ler reurree. Itrouto The trgol solver s very mportt ulg lo we rge of egeerg setf ppltos lug omputer grphs [][][] flu yms [][][] Posso solvers [] preotoers for tertve solvers [] u sple terpolto [8] sem- orseg [9][] for mult- gr methos. I orer to heve hgh throughput severl prllel trgol lgorthms were propose lug Cyl Reuto (CR) [] Prllel Cyl Reuto (PCR) [] Reursve Doulg (RD) [] the SPIKE lgorthm [][]. I ths report we frst use the frst- orer seo- orer ler reurree to epl the ove lgorthms. We show ll estg trgol lgorthms re le to e vewe s ler reurrees or re relte to the ler reurrees. Bse o tht we further propose trgol lgorthm usg oly ler reurrees. The we ompre the propose lgorthm wth the estg lgorthms show the effetveess of the propose lgorthm y usg omputto omplety the requre umer of prllel steps the requre umer of vso opertos the umer of memory ess

2 requests. The reso usg the requre umer of vso opertos s tht vso s the most epesve operto trgol solver the reso usg memory ess requests s tht the trgol solver s memory- ou pplto moer SIMD rhtetures. I ths report we o ot trget lgorthms o y rhteture le GPU or CPU. We te to me our lyss susso geerl to ft y SIMD rhtetures. Se o every operto hs equvlet ost the reltoshp of osts mght vry from rhteture to rhteture optmzto strteges mght vry from rhteture to rhteture. I our lyses omputto omplety the requre umer of prllel steps the requre umer of vso opertos the umer of memory ess requests re ll use for omprg performe.. Bgrou for Relte Estg Algorthms The trgol solver s solver to f soluto of A where A s - y- mtr trgol mtr s - etry vetor s lste Eq. Here we summrze fve lssl lgorthms lug the Thoms lgorthm [] CR PCR RD SPIKE lgorthms. Whle oly the Thoms lgorthm s sequetl lgorthm the rest re prllel lgorthms. A (). Thoms Algorthm The Thoms lgorthm s reue se of Guss elmto (or LU- eompostosolver) for trgol mtr ossts of two phses forwr reuto wr susttuto s show Eq.. The forwr reuto elmtes the lower gol sequetlly whle the wr susttuto solves uow vrles sequetlly usg the resultt upper m gols ()

3 ... + ().. Cyl Reuto The CR lgorthm s o- eve reuto ow s two- wy elmto for trgol mtr. There re lso two phses forwr reuto wr susttuto CR. I eh step of forwr reuto eh o (or eve) equto s elmte y usg jet two equtos fter removg reut uow vrles zeros hlf- sze system s forme of the resultt ew equto. Fg. llustrtes oe step of forwr reuto - y- mtr. I ths se!! the equto e! s elmte y e! e! usg Eq. ut!! s propgte from! e!. After tht ew row e!! s forme. Smlrly e!! s geerte y elmtg e! wth e!. After tht e!! e!! form ew mtr wth the hlf sze. Fgure. A forwr reuto step of CR - y- mtr stre stre stre stre stre stre stre stre where () I eh step of the wr susttuto uow vrles e solve y susttutg solutos of the smller system usg Eq.. stre stre + () Fgure. The grph represetto of CR for 8- y- 8 mtr.

4 Fg. shows the grph represetto of CR for 8- y- 8 mtr. Here we tetolly use olors re lue to lel eges. It mght e surprsg tht ll re eges form Bret- Kug rut [] (Fg. ) ll lue eges form other reverse Bret- Kug rut. I Fg. we tetolly splt re lue eh step to me t eser to e oserve. However t s ot oee to hve Bret- Kug rut CR. A trgol mtr e splt to oe lower gol mtr upper gol mtr esly. A lower gol solver e esly represete s frst- orer ler reurree upper gol mtr e osere s reverse frst- orer ler reurree. The Bret- Kug rut s ommo lgorthm for frst- orer ler reurree. Therefore CR e osere s spelze eteso of Bret- Kug rut. Fgure. The Bret- Kug rut wth 8 oes Fgure. The grph represetto of CR for 8- y- 8 mtr wth splt re lue eges

5 .. Prllel Cyl Reuto The PCR lgorthm s mofto of CR. I otrst to CR PCR performs oly reutos usg Eq. o oth eve AND o equtos. Fg. shows emple for step PCR for - y- trgol mtr. After reutos two epeet sets of eve o equtos e osere s two epeet su- trgol mtres. Most prevous wors [][8] use ths hrterst to splt lrge trgol mtr to multple smller trgol mtres for more prllelsm. Fgure. A forwr reuto step of PCR - y- mtr Fg. shows the grph represetto of PCR for 8- y- 8 mtr. Here s we o for CR we lso use olors to lel eges. As epete we see ll re eges form Kogge- Stoe rut [9] (Fg. ) ll lue eges form other t- reversl Kogge- Stoe rut. Coserg the Kogge- Stoe rut s lso ommo lgorthm for frst- orer ler reurree t seems ovous. Therefore PCR e osere s spelze eteso of the Kogge- Stoe rut. Fgure. The grph represetto of PCR for 8- y- 8 mtr. Fgure. The Kogge- Stoe rut wth 8 oes.. Reursve Doulg The RD lgorthm s prllel trgol solver y formultg trgol mtr to seo- orer ler reurree (Eq. ) wth uow vrle ( ). By pplyg ler reurree solver the

6 reltoshp α β etwee uow vrle e eterme e lso solve. After s solve uow vrle e esly solve y pplyg susttuto wth. Ths susttuto e osere s rost se ll uow vrles s ' requre. ().. SPIKE lgorthm The SPIKE lgorthm s om eomposto lgorthm t eomposes e mtr to set of sjot mtres tht e solve epeetly. We esre spelze SPIKE lgorthm for trgol mtres. I the trgol solver (Fg. 8) AXF the mtr A e prttoe severl gol los A off- gol sgle elemets B C. Smlrly X F e prttoe to orrespog prts. A A A A B B B C C C A X X X X X F F F F F. Fgure 8. Prtto for trgol mtr the SPIKE lgorthm Further we efe D S stsfyg ADS where D s just the gol los of A S hs the struture Fg. 9. A A A A D I I I I V V V W W W S. Fgure 9. Defto of D S the SPIKE lgorthm

7 Here V W stsfy Eq.. (). Therefore AXF e esly solve y resultg DYF SXY. Se D s lo- gol eh lo e solve epeetly. SXY e solve y ut lo trgol solver. Most estg wors use spelze Blo Cyl Reuto (BCR) for solvg SXY. Oly two vrles eh lo re requre to e solve for other los. A reue mtr s requre to e solve. We refer reers to the wors of Pollz [][]. I ths report we lso propose ltertve usg Blo Prllel Cyl Reuto (BPCR) to solve the ut lo trgol solver wth the reue mtr ppe. We pot out the SPIKE lgorthm shres lot of smlrtes wth the group- struture lgorthms [X] frst- orer ler reurrees. The group- struture lgorthms lso eompose frst- orer ler reurree to severl epeet lol frst- orer ler reurrees. After solvg ll lol frst- orer ler reurrees group- struture lgorthms lso solve reue frst- orer ler reurree.. Propose Algorthm We propose trgol lgorthm usg oly ler reurrees. As we metoe trgol mtr e splt to two gol mtres eh gol solver e vewe s frst- orer ler reurree. Therefore we splt the trgol mtr to two gol mtres solve just usg two ler reurrees oe fter the other. Here prllel LU- eomposto s pple for geertg the lower upper gol mtres. We oser ALU L U re efe Eq. 8. L U (8) A e esly solve y resultg Ly Uy. Smply multplyg L U we hve Eq. 9.

8 (9). () By pplyg otue frtos wth seo- orer ler reurree we efe the followg wth seo- orer ler reurree (Eq. Eq. ). (). () Our LU- eomposto tully s otue- frto LU (CF- LU) eomposto [] wth ut upper gol mtr. After eh s ompute we smply solve Ly sve o- the- fly. More lerly we fuse CF- LU wth the gol solver for Ly. Coserg oly s eee for omputg oth Ly we smply ompute to sve vso opertos. After Ly solve e smply solve y solvg Uy. Here we use oe seo- orer ler reurree (Eq. ) two frst- orer ler reurrees (Ly Uy). However se erel fuso s pple oly oe seo- orer ler reurree oe frst- orer ler reurree re relly ompute. Here we ll our seo- orer ler reurree LUL (LU- eomposto wth lower gol solver) the frst- orer ler reurree UBD (upper gol solver). Our metho s me s LUL- UBD. Compre to the RD lgorthm our LUL- UBD lgorthm hs severl eefts. Frst RD uses vso opertos wth s '. Se my e zero or muh smller th or geerl mtr eve golly omt mtr RD tes to hve more umerl errors. Seo Whle RD lues oe seo- orer ler reurree oe rost LUL- UBD ots oe seo- orer ler reurree two frst- orer ler reurrees. It seems tht LUL- UBD mght e slower th RD. However LUL- UBD oe frst- orer ler reurree e fuse wth the seo- orer ler reurree to vo reut memory ess. If we oser frst- orer ler reurree s memory- ou most moer SIMD rhteture ts performe s smlr to the performe of rost whh s lso memory- ou. The CF- LU lgorthm hs ts ow lmttos. Se gol vlues Eq. re ll s whh re ot epeet o oeffets of trgol mtr t mght use overflow whe those oeffets re

9 very g. For emple for gve we smply multply y very lrge umer. Ths operto oes ot fluee the soluto of trgol solver ut mght use overflow of CF- LU. However t s lso very smple to vo ths overflow ssue y performg ormlzto of eh row. Therefore geerl LUL- UBD s more roust th RD.. Tlg Optmzto Algorthm Alyss I ths seto we suss out tlg optmzto whh s eessry for hgh- performe mplemetto most SIMD rhtetures. Before we me etle susso we efe our SIMD rhteture the prolem sze to gurtee fr omprso. Here we ssume the sze of mtr s. The vetor legth of lo step SIMD s M. M s muh smller th. We lso oser luster of vetor proessors. The umer of vetor proessor s P. P s lso muh smller th. The lrgest sze for t hg s L (L tme t types). For emple f our t type s oule oule s 8 Byte our lrgest sze for t hg s 8L Byte. Here we lso ssume L s muh smller th ut L s lrger th M (se the sze of prvte he s usully lrger th the vetor legth). No multthreg ests our efe vetor proessor for smplfyg lyses.. Thoms Algorthm The Thoms Algorthm s sequetl lgorthm t ot eeft from SIMD rhtetures. The umer of (sequetl) steps of Thoms lgorthm s - the omputtol omplety s O(). The umer of vso s lso -. The umer of memory ess requests lues res wrtes the forwr reuto res wrtes the wr susttuto. I the Thoms lgorthm wth lrge sze se the forwr reuto eeutes wth fferet orer of the wr susttuto t use the forwr reuto re less lely to e he for the wr susttuto. (For t the tl of forwr reuto t hg mght e fesle.) Therefore the totl umer of memory ess requests rou 9.. Cyl Reuto If - legth vetor mhe s fesle the requre umer of (prllel) steps s log(). However prte t mght ot e fesle. I our efe SIMD rhteture we oser M- legth vetor proessor P umers of vetor proessors. Se t s luster of proessors tlg optmzto s eessry for mmzg the umer of memory requests. Fg. shows the grph represetto of tle CR. The ol lue les re ter- tle ommuto of forwr reuto were lso propose s splt opertos []. Smlrly we lso efe the ol re les s ter- tle ommuto for wr susttuto fter glol CR ross ll tles. The umers of ter- tle ommuto of forwr reuto wr susttuto re oth (T- )log(t) where T s the tle sze.

10 Coserg the lrgest sze for t hg s L oeffets re requre for oe equto T s equl to L. Se tle CR oes ot hge the omputto ptter the omputtol omplety of tle CR s stll O() the umer of vso s stll - log()-. The requre umer of (prllel) steps eh tle s log(t) + log(t) + f T- legth vetor mhe s fesle. Whle - M legth vetor mhe s pple M s smller th T the requre umer of (prllel) steps s log T + +!!!"# (!) el(!!!! ). The umer of (prllel) steps for the glol CR e osere s other!!! CR wth T prolem sze. The umer of memory requests lues 8 res 8+8T wrtes 8(T - )log(t) res for ter- tle ommuto the memory requests of glol CR whh s other CR wth T prolem sze. The lower ou of memory requests s the upper ou s 8. glol CR Fgure. The grph represetto of tle CR for 8- y- 8 mtr wth tles of - y- su- mtres Glol BCR or BPCR for SPIKE Fgure. The grph represetto of tle CR wth the SPIKE lgorthm for 8- y- 8 mtr

11 Fg. shows other possle tlg strtegy whh e osere s hyr of SPIKE CR. I eh tle omplete CR s performe. After tht BCR or BPCR whh re metoe the seto of SPIKE lgorthm s pple for reue mtres. The rost opertos re pple for ompletely solvg eh tle (eh lo lo trgol). Here we leve the lyss the lyss of SPIKE lgorthm.. Prllel Cyl Reuto If - legth vetor mhe s fesle PCR requres log() (prllel) steps O(*log ()) omputtol omplety log()- + vso opertos. However our efe SIMD rhteture tlg s lso eessry for hgh- performe mplemetto. Fg. shows the grph represetto of tle PCR t e osere s hyr of SPIKE PCR. Smlr to the hyr of SPIKE CR eh tle omplete PCR s performe. After tht BCR or BPCR s pple for reue mtres. The rost opertos re pple for ompletely solvg eh tle. Here we leve the lyss the lyss of SPIKE lgorthm. Glol BCR or BPCR for SPIKE Fgure. The grph represetto of tle PCR wth the SPIKE lgorthm for 8- y- 8 mtr. Reursve Doulg I RD the omputtol omplety the umer of steps the umer of memory requests re ll epeet o lgorthms for ler reurrees. The mml omputtol omplety s O() t hppes most ler reurree lgorthms. The mml umer of vso s t lso hppes most ler reurree lgorthms. The umer of steps vres lot from lgorthm to lgorthm. The mml umer of steps s log()+ t requres the Kogge- Stoe rut wth - legth vetor mhe. I prte tle ler reurree hs rou TM + log(m) steps steps of the glol ler reurree wth T prolem sze where T s tlg sze s L. The glol ler reurree s usully mplemete wth sequetl lgorthm wth ter- tle ommuto to mmze the umer of memory requests for hgh- performe mplemetto. The mml umer of memory requests of seo- orer ler reurree ots res - wrtes (- wrtes for oeffets wrte for ) whle the mml umer of memory requests rost ots - res - wrtes. Therefore RD hs rou 9 memory ess requests mmum.

12 . SPIKE lgorthm The omputtol omplety the umer of steps the umer of memory request SPIKE lgorthm s ll epeet o the prllel lgorthm for solvg su- mtres (eh tle). Three lgorthms whh re the Thoms lgorthm CR (Fg. ) PCR (Fg. ) re ofte pple. The mml omputtol omplety s rou O() t hppes whe the omplety of the su- mtr solver s lso ler suh s CR or the Thoms lgorthm. The SPIKE lgorthm wth Thoms lgorthm hs +T vso opertos whle the SPIKE lgorthm wth CR hs - (T)*log(T)- T vso opertos. Whe PCR s pple for the su- mtr solver the omplety eomes O(log(T) + (T)*log(T) ) where T s tlg sze s equl to L. The SPIKE lgorthm wth PCR hs log(t)- +T vso opertos. Here to me more lerly the tlg sze (T) of Thoms lgorthm mght e fferet from the tlg sze of CR or PCR. The umer of steps vres lot from lgorthm to lgorthm lug oth lgorthms for the su- mtr solver the lo trgol solver. Whe the Thoms lgorthm s pple for the su- mtr solver T steps re requre for eh tle wth T sze. Se the Thoms lgorthm oes ot ee t tlg we smply mmze prllelsm. Therefore the mml umer of steps s MP + log(m*p) + t hppes whe BPCR re pple for the reue lo trgol solver. Whe the Thoms lgorthm s pple for the su- mtr solver log(t) steps re requre for eh tle wth T sze. Therefore the mml umer of steps s log(t)+log(t) + lso BPCR s pple. Whe PCR s pple the mml umer of steps s log(t)+log(t) + lso BPCR s pple. The mml umer of memory requests of ll su- mtr solvers lues res wrtes. The umer of memory requests for the lo trgol solver vres from to + T where T s tlg sze.. LUL- UBD lgorthm I LUL- UBD smlr to RD the omputtol omplety the umer of steps the umer of memory requests re ll epeet o lgorthms for ler reurrees. The mml omputtol omplety smlr to RD s O() t hppes most ler reurree lgorthms. The mml umer of vso s lso t lso hppes most ler reurree lgorthms. I prte tle LUL whh s seo- orer ler reurree fuse wth frst- orer ler reurree hs rou T M + log(m) steps steps of the glol ler reurree wth T prolem sze where T s tlg sze T s L. The UBD whh s frst- orer ler reurree hs T M + log(m) steps steps of the other glol ler reurree wth T prolem sze where T s tlg sze T s L. Smlr to RD the glol ler reurrees re usully mplemete wth sequetl lgorthm wth ter- tle ommuto to mmze the umer of memory requests for hgh- performe mplemetto. The mml umer of memory requests of LUL ots res wrtes the oe of UBD ots res wrtes. Therefore LUL- UBD hs rou 9 memory ess requests mmum.

13 . Coluso We propose ovel trgol lgorthm lle LUL- UBD whh oly reles o ler reurrees. We further show ll estg lgorthms re relte to ler reurrees. We lso ompre LUL- UBD wth ll estg lgorthms to emostrte LUL- UBD hs mml umer of memory requests vso opertos. Referee [] M. Kss A. Lefoh J. Owes Itertve epth of fel usg smulte ffuso o GPU Teh. Rep. #- Pr Amto Stuos J.. [] M. Kss G. Mller Rp stle flu yms for omputer grphs Proeegs of the th ul oferee o Computer grphs tertve tehques SIGGRAPH 9 (New Yor NY USA) pp. 9 ACM 99. [] S. Segupt M. Hrrs Y. Zhg J. D. Owes S prmtves for GPU omputg Grphs Hrwre pp. 9 Aug.. [] N. Shryh Trgol solvers o the GPU ppltos to flu smulto NVIDIA GPU Tehology Coferee Septemer 9. [] N. Shryh Effet trgol solvers for ADI methos flu smulto NVIDIA GTC Sep.. [] R. W. Hoey A fst ret soluto of Posso s equto usg Fourer lyss J. ACM vol. pp. 9 J. 9. [] A. Greeum Itertve Methos for Solvg Ler Systems. Phlelph: SIAM 99. [8] L. Chg M. Lo N. Assr K. Hsu N. Hug W. Hwu Prllel mplemetto of mult- mesol esemle emprl moe eomposto Itertol Coferee o Aousts Speeh Sgl Proessg My. [9] M. Preto R. Stgo D. Esps I. M. Llorete F. Tro Prllel multgr for sotrop ellpt equtos J. Prllel Dstr. Comput. vol. pp. 9 Jury. [] D. Göee R. Strzo Cyl reuto trgol solvers o GPUs pple to me- preso multgr IEEE Trstos o Prllel Dstrute Systems vol. pp.. [] S. D. Cote C. W. D. Boor Elemetry Numerl Alyss: A Algorthm Approh. MGrw- Hll Hgher Euto r e. 98. [] R. W. Hoey C. R. Jesshope Prllel omputers : rhteture progrmmg lgorthms. Hlger Brstol 98. [] H. S. Stoe A effet prllel lgorthm for the soluto of trgol ler system of equtos J. ACM vol. pp. 8 J. 9. [] E. Polzz A. H. Smeh A prllel hyr e system solver: The SPIKE lgorthm Prllel Computg vol. o. pp. 9. Cte By (se 99):. [] E. Polzz A. Smeh SPIKE: A prllel evromet for solvg e ler systems Computers Flus vol. o. pp.. Cte By (se 99):. [] R. P. Bret H. T. Kug A Regulr Lyout for Prllel Aers IEEE Trs. Comput. Mr 98

14 [] A. Dvso Y. Zhg J. D. Owes A uto- tue metho for solvg lrge trgol systems o the GPU Proeegs of the th IEEE Itertol Prllel Dstrute Proessg Symposum My. [8] H. Km S. Wu L. Chg W. Hwu A slle trgol solver for gpus Prllel Proessg (ICPP) Itertol Coferee o pp. Sept.. [9] Peter M. Kogge Hrol S. Stoe. A Prllel Algorthm for the Effet Soluto of Geerl Clss of Reurree Equtos IEEE Trs. o Comput. v..8 p.8-9 Aug. 9 [] Ömer Eğeoğlu LU Ftorzto A Prllel Evluto Of Cotue Frtos Proeegs of IPDPS 998 [] F. Argüello D.B. Hers M. Bóo J. Lms- Roríguez The splt- - merge metho geerl purpose omputto o GPUs Prllel Computg Volume 8 Issues Jue July pp Appe The Blo Prllel Cyl Reuto (BPCR) lgorthm s eteso of PCR for lo trgol solver. Fg. A shows the lssl PCR whle Fg. A shows tle PCR orrespog to the grph represetto of Fg.. I Fg. A - y- reue mtr s solve for 8- y- 8 lo trgol mtr wth lo sze s - y-. Fgure A. A forwr reuto of PCR 8- y- 8 mtr Fgure A. A forwr reuto of tle PCR 8- y- 8 mtr I orer to me more ler we oser the stuto wth more tles. Fg. A shows tles tle PCR. Se oly two equtos est tle Fg. A e lso vewe s emple of the reue lo trgol solver of SXY the SPIKE lgorthm. I SXY ll s re oes fter eh step of BPCR

15 ew s re lso ormlze to oes. Therefore BPCR e pple for solvg SXY the SPIKE lgorthm. Fgure A. A forwr reuto of BPCR 8- y- 8 mtr