ORIGINAL SCIENTIFIC PAPER 59 Modelling of Apple Fruit Growth y Appliction of Imge Anlysis Denis STAJNKO 1 Zltko ÈMELIK SUMMARY The possiility of RGB imge processing nd nlysis for modelling of the development nd growth of pple fruits ws investigted during the two sesons under the orchrds experiment in the four-yers old Golden Delicious nd Gl vriety. The fruit detection depended significntly on the size nd fruit s colour of ech growing stge, thus the correltion coefficients were continuously incresing from r=.71 (1) nd r=.73 () fter fruit tinning in June, up to r=. (1) nd r=.9 () t hrvesting in Septemer respectively. The yield t hrvest ws estimted with the ccurcy of 9% nd 1% for Golden Delicious nd with % nd 9% for Gl respectively, whenever sed on imges cptured on June 1 nd June. Therefore, the imge lgorithm ws proved to e equl or even etter method for estimting the yield t hrvest thn the common Prognosefruit method (ccurcy 1% nd 77% for Golden Delicious nd 7% nd 3% for Gl respectively) KEY WORDS RGB imge, pple, yield, forecst 1 University of Mrior, Fculty of Agriculture Vrnsk 3, Mrior, Sloveni E-mil: denis.stjnko@uni-m.si University of Zgre, Fculty of Agriculture Svetošimunsk 5, Zgre, Croti Received: Ferury, 5 ACKNOWLEDGEMENTS This reserch represents n integrted prt of the Ph.D. Thesis tht ws successfully defended on the Fculty of Agriculture University of Zgre. Agriculture Conspectus Scientificus, Vol. 7 (5) No. (59-)
Denis STAJNKO, Zltko ÈMELIK INTRODUCTION Modelling of pple fruit development nd growth represents one of the most interesting topics for the scientists for the long period. However, in the recent decdes iologicl simultion models hve grown in populrity s sustitute for lrge-scle orchrd experiments due to incresed computer cpcity (Oride nd Dillon, 1997). Despite of ll methods developed y the horticulturists, the mechnistic Prognosfruit (Winter, 19) remins the most ccepted method y the Europen pple nd per producers (Lmrechts, 1, Rmos nd Lieerz, 3). However, the time consuming mesurements of required prmeters void to pply the method on single orchrd level, therefore new pproch for effective dt cquisition nd imge nlysis lgorithms hs een developing nd investigting in order to determine the numer nd dimeter of fruits for estimtion the yield t hrvesting (Stjnko et l., ) In the lst decdes the implementtion of computer vision hs een widely dopted for uilding the fruit detection lgorithm pplied in the fruit grding procedure on the pckging lines. Severl industril lgorithms for processing the min steps of the imge nlysis lgorithm under the rtificil lighting in chmers utomticlly were represented y mny uthors (Jimenez et l. 1999). However, whenever operting in the open field with sunlight s n illumintion source the imge qulity decresed significntly, thus the controlled illumintion ws suggested nd pplied in most cses. Alredy with the first pple-picking root MAGALI, the detection of different vrieties of pple fruits ws possile only under drk ckground ssured y protective coverge (Grnd D Esnon et l., 197). Furthermore, Peterson et l. (1999) instlled specil fire-reinforced drpery on the pple-hrvesting root to lock the influence of nturl light conditions. Also the citrus root for hrvesting ornges (Juste nd Sevill, 1991) nd strwerry hrvesting roots (Kondo et. l., 199) needed n rtificil light sources lthough the ripen fruits were red colour. The ccurcy of pple fruit detection lgorithms ws crucil prmeter whenever evluting the efficiency of the root hrvesting. However, the reported precision vried gretly from 1% (Kssy, 199) to 5% (Ktok et l., 1999) nd 95% (Petersen et l., 1999) depending significntly on the fruit colour, the pplied filters nd thresholding techniques. In our reserch the numer of fruits ws determined lso prior the hrvesting period, when the colours of the fruits did not differ sustntilly from the colour of leves. Thus the min ojective of this pper is to demonstrte nd evlute the pplicility of the method for predicting the numer nd the dimeter of the pple fruits needed for modelling the current nd hrvested yield in the pple orchrd. MATERIALS AND METHODS During the vegettion period June-Septemer 1 nd, ten pple trees (Mlus domestic Borkh. of the Golden Delicious nd ten of the Gl vriety were exmined in the Fculty s orchrds (lt. o 3 N, long. 15 o 33 5 E). Four yer-old pple trees were trined s super spindle nd plnted t spcing of, x,7 m. All trees were grfted on the M9 rootstock nd the rows were oriented from Est-North to West- South. In oth yers five developing stges of pple fruits were selected for cpturing imges during the fruit s growth nd ripening (Tle 1). For cpturing imges CCD OLYMPUS 33 cmer with the Flsh setting progrm ws used with three different resolutions x153, 1x1 nd 1x9 pixels from distnce of 1,m nd the ngle of 9 o towrds the tree row. Tle 1. Cpturing pln Fruit detection lgorithm The pplied five-step pple fruit lgorithm sed on colour nd shpe detection. However, the fruits of the chosen pple vrieties chnged their colour ccording to the growing stge significntly, thus roust nd djustle lgorithm ws developed. As seen from the Figure 1, ll the fruits could not e detected on the originl RGB imges t once, thus smple imge from ech series ws first trined y dividing it into three sis plnes (R imge, G imge nd B imge). Whenever ny of those imges did not fulfill the required contrst etween the ojects, dditionl trnsformtion to the imges of I (illumintion), H (hue) or S (sturtion) proceeded. After tht on the sis of the histogrm nlysis, the most fitting imge ws selected for ech fruit developing stge seprtely. For tht reson, in the Golden Delicious vriety the R imge ws chosen for further processing in the first stge (My 3 rd 1 nd My th ) nd the G imge for ll other stges (Figure ). Contrry, in the Gl the G imge ws selected in the first stge nd the R imge for ll lter stges. In the second stge the selected imge ws first filtered y pplying of specified size of kernels (3x3 pixels) to remove the noise nd the connectivity- function, which divided the order pixels elonging to one or nother oject. Then, y pplying of precise threshold vlues the inry imge ws creted for
MODELLING OF APPLE FRUIT GROWTH BY APPLICATION OF IMAGE ANALYSIS 1 Figure 1. Originl RGB imge showing the Golden Delicious () nd Gl tree () c d Figure. Trnsformed G () nd R () imge of the Golden Delicious nd G (c) nd R (d) imge of the Gl Figure 3. Binry imge processed from the Golden Delicious () nd the Gl imge () ech developing stge (Figure 3). After tht, in the third stge, two-step oject detection proceeded utomticlly y using of the ellipse templte in the first step nd the whole pple fruit templte in the second step. Finlly, the remining ojects (Figure ) were counted nd the stndrd morphologicl
Denis STAJNKO, Zltko ÈMELIK Figure. Detected fruits of the Golden Delicious () nd the Gl imge () chrcteristics (longest segment, mjor xis, minor xis, re, perimeter, compctness nd elongtion) were nlysed on ech oject seprtely. In the lst stge the current yield ws estimted on ech imge y pplying of derived Mitchell s (19) eqution for oth vrieties s follows:, 9 N D Y =, 5 (1) GoldenD N D Y =, 59 Gl, 9 () where Y GoldenD nd Y Gl represents the yield per tree in kg, N the numer of fruits per tree nd D the verge fruits dimeter of specified developing stge. For performing the ove-descried lgorithms our own code ws developed in the IMAQ Vision 5.1 nd Lview. Pckge Progrm. RESULTS The estimted numer of pple fruits per tree y the imge nlysis s well s mnully counted fruits is represented in the Tle. As seen, the estlished correltion coefficient vried in the Golden Delicious etween.71 nd.9 in 1 nd from.73 to.9 in respectively. Very close correltion ws lso estlished for the Gl vriety wheres in 1 the coefficients vried from.73 in June to.9 t hrvest nd from.7 in June to.91 t hrvest in respectively. A close correltion etween detected nd mnully counted numer of fruits t the hrvesting were lredy reported from Kondo et l. (199) whenever investigting the lgorithms for the strwerry hrvester nd Ktok et l. (1999) for the pple hrvester. However, our fruit detection lgorithm ws proved to e successful toll lso for predicting the numer of fruits prior hrvesting, which ws n importnt prmeter for estimting the yield. On the other hnd, whenever evluting the estimted verge fruit s dimeters per tree with mnul mesurements t different developing stges of pple fruits (Tle 3), it my e seen, tht it ws prcticlly equl to the mnul mesurements t ll developing stges during the vegettion in oth vrieties. However, in the Golden Delicious the correltion coefficient vried gretly from.19 to.55 in 1 nd from.3 to.79 in respectively while in the Gl the coefficient rised from.77 to. (1) nd from.3 to. (). The first reson for lower correltion coefficients lies in the fruit detection lgorithm, which is sed on the Tle. Numer of pple fruits estimted y imge nlysis nd mnully counted
MODELLING OF APPLE FRUIT GROWTH BY APPLICATION OF IMAGE ANALYSIS 3 Tle 3. Averge dimeter (mm) of pple fruits estimted y imge nlysis nd mnully counted Tle. Averge yield per tree (kg) estimted y imge nlysis nd mnully counted longest segment mesurement. The second one is due to very levelling dimeter mong ll smpled trees cused y chemicl tinning, so even the smll devition of the dimeter resulted in the gret fll of the correltion. The current yield per tree ws estimted y pplying of equtions 1 ( Golden Delicious ) nd ( Gl ). The correltion coefficient etween mnul mesurements nd imging estimtion of the prticulr developing stge is represented in the Tle. As seen, in the Golden Delicious coefficients vried from.55 to.7 in 1, while in remined very stle ltering only from.91 to.97. The ovious difference etween oth yers ws due to the erly frost dmges nd summer hets in 1. For the Gl the coefficients vried gretly in oth yers, lthough the negtive effect of the wether conditions hs lrger influence in the first yer. Contrry, whenever modelling the development of the fruit yield on the sis of imge nlysis nd compring it with mnul mesurements, lmost identicl growing curves were estlished for the Golden Delicious (Figure 5) nd the Gl (Figure ) respectively. With the very close correltion for oth vrieties in ll yers, the imge lgorithm hs show good possiility for modelling of the yield t hrvest lredy fter the fruit tinning in June. Averge yield (kg/tree) Averge yield (kg/tree) 1 1 1 () Golden Delicious (1) r=. 9 June June Mnul mesurement Imge nlysis Averge yield (kg/tree) mnully Averge yield (kg/tree) imge nlysis June July 1 () Golden Delicious () r=. 9 June July 1 Sep 1 Sep 1 Figure 5: Growing curves of the Golden Delicious : () 1 nd () experiment
Denis STAJNKO, Zltko ÈMELIK Averge yield (kg/tree) Averge yield (kg/tree) CONCLUSION () G l (1) r=. 9 June June () G l () r=. 9 June Mnul mesurement Imge nlysis Averge yield (kg/tree) mnully Averge yield (kg/tree) imge nlysis June July 1 July 1 Figure. Growing curves of the Gl () 1 nd () experiment A new pproch for modelling of pple fruit development nd estimtion of the hrvested yield under orchrd conditions ws reserched in our investigtion. Bsed on cptured RGB imges, nd grown through the severl processing nd nlysis procedures, the presented lgorithm shows gret possiility for modelling of yield development in the Golden Delicious nd Gl vriety during the vegettive period. However, future work should e focused on improving the lgorithm, so it is le to detect lso prtilly hidden sphericl ojects. REFERENCES Jimenez A.R., Jin A.K., Ceres R., Pons, J.L. (1999). Automtic fruit recognition: survey nd new results using Rnge/ Attenution imges, Pttern Recognition 3, 1719-173 Juste F., Sevill F. (1991). Citrus: A Europen project to study the rootic hrvesting of Ornges, in: Proc. 3 rd Int.Symp. Fruit, Nut nd Vegetle Hrvesting Mechniztion, Denmrk, Sweden, Norwy, 331-33 Kssy L. (199). Hungrin rootic pple hrvester, ASAE Pper No. 9-7 Ktok T., Bulnon D.M, Hirom T., Ot Y. (1999). Performnce of Rootic Hnd for Apple Hrvesting, ASAE Pper No.9933, 11 s. MI.:ASAE Kondo N., Hised K., Mont M. (199). Development of Strwerry Hrvesting Rootic Hnd, ASAE Pper No.93117,ASAE St. Joseph, MI-USA, p.1-7 Lmrechts G. (1). Apple EU 1 Forecst 1, Prognosfruit 1, p. 1- http://cmlg.fgov.e/dg/fr/communictions/prognos-lgemeen.pdf Grnd D esnon, A., Rtel G., Pellenc R. (197). Mgli: self-propelled root to pick pples, ASAE pper No. 7-37, St.Joseph, MI 95-959 Mitchell P.D. (19). Per fruit growth nd the use of dimeter to estimte fruit volume nd weight, Hort Science 1,, 3-5 Oride C.A., Dillon C.R. (1997). Developments nd iophysicl nd io economic simultion of griculturl systems: rewiev. Agriculturl Economics 17, 5-. Peterson D.L., Anger W.C., Bennedsen B.S.,Wolford S.D. (1999). A system pproch to rootic ulk hrvesting of pples. ASAE Pper No. 99-75 Rmos K., Lieerz S.M. (3). Prognosfruit 3 Europen Crop Forecst Convention, 7 s., http://www.fs.usd.gov/ ginfiles/3/15957.doc Stjnko D, Lkot M, Hoèevr M. () Estimtion of numer nd dimeter of pple fruits in n orchrd during the growing seson y therml imging. Comput. electron. Agric., 31-. Welte H.F. (199). Forecsting hrvest fruit size during the growing seson. Act Horticulture 7, 75- Winter F. (19). Modelling the iologicl nd economic development of n pple orchrd. Act Horticulture, 1, 353-3. cs7_