Instance segmentation method of street trees based on Mask R-CNN

LU Qingyu;LI Qiujie;TONG Yuekai;WANG Mingxia;YUAN Pengcheng;College of Mechanical and Electronic Engineering, Nanjing Forestry University;

street trees resources investigation;;street trees instance segmentation;;deep learning;;transfer learning;;Mask R-CNN

To solve the problems of segmentation of street tree images in the investigation of street tree resources, a method of the instance segmentation of street trees was proposed based on Mask R-CNN. The management of conventional street tree resources has mainly adopted manual measurements combined with the sampling investigation method, which requires large amount of labor force and low efficiency. In this study, a new instance segmentation strategy of street trees was designed with the application of deep learning methods, and experiments verified that the proposed model achieved high accuracy and efficiency. The overall procedures of the proposed method are as follows: image data was firstly collected and labeled, and image dataset of street trees was established. Images were then preprocessed through the size transformation and data expansion to mitigate the over-fitting problem which may occurred in network training. To initialize the dataset, the transfer learning was adopted by transferring pre-trained network parameters in COCO large image dataset to the instance segmentation model of street trees. Lastly, the model was trained until its convergence. In experiments, street trees images of different tree species were collected to verify the generalization abilities for the model, including Cinnamomum camphora, Platanus, Magnolia grandiflora, Salix, Ginkgo biloba, and Trachycarpus fortunei. The number of images in dataset was 1 482, and dataset was partitioned into training set, validation set and test set. The number of images in training set, validation set, and test set was 894, 294 and 294, respectively. In this study, Tensorflow was chosen as the deep learning framework for training and testing. The preprocessed model was trained on this deep learning framework for 40 000 iterations. Afterwards, a total of 294 images in test dataset were tested. The results were also compared with ground truth data to calculate relating evaluating indexes. The evaluating indexes were calculated for each street tree species, and for all the images in test set, the index of mean intersection of union was about 80%; both the average precision and average recall were over 95%; the average of the image testing time was 0.476 s, which could satisfy the requirements for the investigation of street tree resources. The experimental results indicated that the model had very good performance on test dataset and effectively obtain fine segmentation of street tree instances of different tree species.