Giuseppe Pellizzi Prize 2018

Transcription

1 Giuseppe Pellizzi Prize th Members Meeting of the Club of Bologna November 10, 2018 (SPACE RESERVED FOR PHOTO CONCERNING THE THESIS) INTELLIGENT VISION SENSING SYSTEMS FOR PRECISION AGRICULTURE PRACTICES IN FLORIDA CITRUS PRODUCTION Daeun Dana Choi, Ph.D. Assistant Professor, Dept. Ag and Bio Engineering Penn State University, Pennsylvania, U.S.A.

2 BIOMECHATRONIC ENGINEERING B. E. at Sungkyunkwan University, Seoul, South Korea Summa Cum Laude Research Intern: Intelligent Robotics & Mechatronics System Lab ECONOMICS B. A. at Sungkyunkwan University, Seoul, South Korea Study Abroad, Vrije University, Amsterdam, Netherlands AGRICULTURAL AND BIOLOGICAL ENGINEERING DAEUN DANA CHOI, PH. D. Education and Experiences M. S. and Ph.D. at University of Florida, Gainesville, FL Research Assistant in Precision Agriculture Laboratory Master Thesis: Detecting and counting citrus fruit on the ground using machine vision Dissertation: Intelligent sensing systems for precision agriculture practices in citrus industry AGRICULTURAL AND BIOLOGICAL ENGINEERING Assistant Professor at Pennsylvania State University, University Park, PA, U.S.A. Lab Supervisor at Smart Agriculture Laboratory Photo credit to Emiel Molenaar

3 Intelligent Vision Sensing System for Florida Citrus Production slide #3 Introduction Production (1000 tons) Orange production in top five leading countries during Citrus Production Decline in the United States Year Brazil United States China Source: Citrus: world markets and trade, USDA Foreign Agricultural Service (2017) 9852 Citrus? High-valued specialty crops in international trade markets Fresh market Processed Product Citrus Production million tons per year ( ) Major crop: Oranges (69.2 million tons per year, 59%) Source: Citrus fruit statistics 2015, Food and Agriculture Organization of the United Nations (2016) Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 3

4 Intelligent Vision Sensing System for Florida Citrus Production slide #4 Challenges in the Florida Citrus Industry Orange production in the United States Florida Citrus Industry Incomparably dominant producer more than 80% of the oranges in US Economic impacts $8.9 billion in the season $10.8 billion in the season Orange production decrease Production dropped to 73% in the , 68% in and 59% in Orange Production (1000 tons) Florida California USA Canker & Hurricanes Year Widespread of HLB Source: Florida citrus statistics , USDA-NASS (2016) Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

5 Intelligent Vision Sensing System for Florida Citrus Production slide #5 Precision Agriculture Optimization of Farming Input Efficient farming by site-specific crop management Increase crop yields Maximize profit for growers Reduce environmental impact Citrus grove in Lake Wales, Florida Intelligent Sensing System Automatic data acquisition + machine Intelligence Crucial for agricultural automation, robotics, precision agriculture management Reduce production costs and increase productivity Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

6 Intelligent Vision Sensing System for Florida Citrus Production slide #6 Intelligent Vision Sensing to support precision agriculture in Florida citrus production Early Stage of Crop Development Immature citrus detection system for yield forecasting using a Kinect depth sensor and a gradient vector based method Pre-harvest to Harvest Premature citrus fruit drop detection using regular RGB cameras and an adaptive image correction algorithm 03 Post-harvest Precise citrus defect inspection using regular RGB cameras, deep learning, and parallel computing techniques Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

7 Intelligent Vision Sensing System for Florida Citrus Production slide #7 Immature Citrus Detection System for Yield Forecasting Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

8 Intelligent Vision Sensing System for Florida Citrus Production slide #8 Immature Citrus Detection for Yield Forecasting Forecasting citrus production Early decisions for planning operations and marketing Estimating the number of fruit by manually surveying sampled areas Difficult to produce consistent, accurate and timely agricultural market data and forecasts Accurate estimation of fruit count in individual tree Kinect V2 Truck Hand-held Golf cart Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

9 Intelligent Vision Sensing System for Florida Citrus Production slide #9 How many citrus can you find? Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

10 Intelligent Vision Sensing System for Florida Citrus Production slide #10 Answer: 18 citrus False positive warning area! Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

11 Intelligent Vision Sensing System for Florida Citrus Production slide #11 Behind the scene Near-Infrared Image Computers may see better than human does. Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

12 Intelligent Vision Sensing System for Florida Citrus Production slide #12 Gradient Vector Field on the Citrus Surface ügradient Vector of depth (D) V = D = D x, D y = D *, D, üdirection of the Gradient Vector v = V V = D D = 1 D D *, D, Depth map of a citrus surface Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 12

13 Convex function Concave function 2D Representation of 3D Surfaces ü Divergence: the amount of spreading vectors Div v = 3 v = x, y 3 1 D D *, D, = 4 56 ( * + 86 ; 8, ) Divergence

14 Intelligent Vision Sensing System for Florida Citrus Production slide #14 2D Representation of 3D Surfaces Convex function Rotating vectors in 2D GVF plot in clockwise direction with fixed origin. ü Vorticity: the amount & direction of microscopic circulation ω = v = x, y, z D *, D,, D B Concave function = 86 ; 8* , k ω = D, x D * y v = D *, D, v = D,, D * Vorticity Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 14

15 Intelligent Vision Sensing System for Florida Citrus Production slide #15 2D Representation of 3D Surfaces Convex function Concave function v = D *, D, v = D,, D * Vorticity Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 15

16 Intelligent Vision Sensing System for Florida Citrus Production slide #16 Characteristics of the Citrus Surface Contour map of the citrus surface v = D *, D, v = D,, D * Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 16

17 Intelligent Vision Sensing System for Florida Citrus Production slide #17 Characteristics of the Citrus Surface Contour map of the citrus surface Divergence Vorticity Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 17

18 Intelligent Vision Sensing System for Florida Citrus Production slide #18 Spherical Object Detection Algorithm Condition 1 The center pixel of a sphere has the maximum divergence value among other pixels Condition 2 All pixels of a sphere maintain non-negative divergence and vorticity values Divergence Vorticity Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 18

19 Intelligent Vision Sensing System for Florida Citrus Production slide #19 Classification (SVM and Random Forest) Feature description Number of features Roundness 1 Number of edges/area in NIR 1 Mean NIR 1 Mean divergence 1 Mean vorticity 1 Mean divergence after Gabor filtering 8 Mean vorticity after Gabor filtering 8 Mean magnitude of gradient vectors after Gabor filtering 8 Mean of synthetic texture after Gabor filtering 8 Total 37 Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

20 Intelligent Vision Sensing System for Florida Citrus Production slide #20 Results Result will be displayed on RGB Image or Near-infrared Image for comparison Day Image Image with Strong Sunlight Statistics Night Image with an External Light Correct Identification = Correctly Identified Citrus Fruit/Actual Number of Fruit False Positives = Incorrectly Detected Object/Actual Number of Fruit Night Image with a Partial External Light Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

21 Night image

22 Night image

23 Intelligent Vision Sensing System for Florida Citrus Production slide #23 Daytime image Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

24 Intelligent Vision Sensing System for Florida Citrus Production slide #24 With strong sunlight Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

25 Intelligent Vision Sensing System for Florida Citrus Production slide #25 With strong sunlight Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

26 Intelligent Vision Sensing System for Florida Citrus Production slide #26 Results Using all 37 Features Actual # of Fruit Random Forest Support Vector Machine CI (%) FN (%) FP (%) CI (%) FN (%) FP (%) Day (88.7) 38 (11.3) 23 (6.9) 275 (82.1) 60 (17.9) 15 (4.5) Night (89.8) 31 (10.2) 17 (5.6) 278 (91.2) 27 (8.9) 15 (4.9) Sum (89.2) 69 (10.8) 40 (6.3) 553 (86.4) 87 (13.6) 30 (4.7) Selected Features using Random Forest Actual # of Fruit RF Top 5 features RF Top 3 features CI (%) FN (%) FP (%) CI (%) FN (%) FP (%) Day (87.2) 43 (12.8) 26 (7.8) 281 (83.9) 54 (16.1) 38 (11.3) Night (86.9) 40 (13.1) 21 (6.9) 264 (86.6) 41 (13.4) 21 (6.9) Sum (87.0) 83 (13.0) 47 (7.3) 545 (85.2) 95 (14.8) 59 (9.2) Dana Choi Intelligent Vision Sensing System for Florida Citrus Production 26

27 Intelligent Vision Sensing System for Florida Citrus Production slide #27 Unit:mm Dana Choi Intelligent Vision Sensing System for Florida Citrus Production

28 Acknowledgements Dissertation Supervisors Dr. Won Suk Lee Dr. John Schueller Dr. Reza Ehsani Dr. Fritz Roka Dr. Corey Toler-Franklin Dr. Dorota Haman Dr. Mark Ritenour Dr. Alireza Pourreza Mr. Hao Gan Dr. Yao Zhang Mr. Chenglin Wang Dr. Kezhu Tan Mr. Arumugam Kalaikannan Mr. Mike Zingaro

29 THANK YOU Thank you for coming Have a nice day Image source: Orange tree, oil painting by Alan Flood