Application of Deep Learning in Segmentation of Cell Image by Optical Microscope

JIA Ce,CAO Guang-Fu,WANG Xiao-Feng,ZHANG Xiang

(1)Guangzhou University,Guangzhou 510006,China;2)Institute of Biophysics,Chinese Academy of Sciences,Beijing 100101,China;3)South China Agricultural University,Guangzhou 510642,China)

Abstract Objective In order to analyze the number and morphology of cells in the process of cell culture conveniently.Methods In this paper,we introduce a cell counting method which can directly count cells in culture dish from images of commercial optical microscope by applying deep learning technology.Results In order to implement cell segmentation and counting,labeling and training is carried out on the image of adherent cells and suspension cells by a U-Net structure network.The cell growth curve is plotted and the inhibition rate of inhibitor is calculated by this algorithm,which shows the practicability of the algorithm.Conclusion It is feasible to do cell segmentation in dish by deep learning method.

Key words deep learning,cell segmentation

The analysis of cell number and morphology by optical microscope is widely used in medical and biological research[1-3].Cell counter for counting and analyzing cells is also available in some instruments.However,this kind of instrument often needs a special counting plate or dye[4].Cells need to be transferred from the culture dish to counting plate,which increases the extra labor and the consumption of the laboratory.Therefore,a method that can take pictures from commercial optical microscope and directly count cells in the culture dish is necessary for practical usage.At present,there are many cell image recognition algorithms,including threshold based segmentation[5],image edge based segmentation[6],histogram based segmentation[7],image region based segmentation[8],wavelet based segmentation[9]and so on.These segmentation algorithms have been successfully applied in some cell segmentation scenes and tasks,however,segmentation effect still needs improvement[10-12].Deep learning is a machine learning technology developed in recent years.It simulates the structure of human brain to establish a multilayer neural network model for research,which has been successfully applied to the field of image recognition and segmentation and achieve many results[13-15].The convolution neural network has greatly accelerated the development of this field.Therefore,this paper attempts to apply the deep learning technology to the recognition and counting of cell images taken by commercial optical microscope,at the same time,the algorithm is used to plot cell growth curve and calculate the inhibition rate of inhibitors,in order to prove the practicability of the algorithm.

1 Methods

1.1 Network structure

The network structure of U-Net is used for segmentation[16].U-Net is an improved algorithm developed on the basis of full convolution neural network(FCN),which has been proved to be suitable for medical image segmentation.The U-Net network structure adopts the encoder and decoder structure.In the decoder,the features in the image are extracted by convolution and merging.In the decoder,the segmentation results are gradually constructed while maintaining their features.The network structure is shown in Figure 1.

Fig.1 Network structure

This network is mainly composed of two parts:an encoder structure on the left,and a decoder structure on the right.The encoder is a typical convolutional neural network structure.The input image doubles the number of feature channels by two 3×3 convolutions(conv 3×3).Each convolution is followed by a Rectified linear unit(Relu)and then the length and width of the feature image are halved by a 2×2 max pooling operation.After five groups of such operations,the length and width of the feature map become 1/16 of the original,and the number of feature channels become 1 024.The decoder is a reverse operation.It uses 2×2 up sampling to double the length and width of the feature map,and combines it with the corresponding feature map from the encoder.Then it uses two 3×3 convolutions followed by a linear correction unit to halve the feature channel.In the final layer,a 1×1 convolution followed by a sigmoid function is used to map the feature image to the segmentation labeled image.

1.2 Training parameters

1.2.1 Data set

A large image was cut into some small images of 512×512 pixels.We selected 30 small images and labeled the cells for training,and 6 images unlabeled for validation.

1.2.2 Optimer

Adam algorithm[17]with 0.000 1 learning rate was choose for optimizing.

1.2.3 Loss function

Binary cross entropy was chosen for training,which was defined as,

whereMis output image size,piis the labeled value,qiis the predictive value.

1.3 Cell culture and image acquisition

We used HeLa cells for adherent cells and hybridoma cells for suspension cells in the experiment.The two kinds of cells were cultured in a dish containing 10% FBS(fetal bovine serum)and DMEM(hyclone,GE,America).Then images were taken by a microscope(szx16,Olympus,Japan)equipped with a digital camera(D760,Nikon,Japan).

2 Results

2.1 Adherent cell segmentation

We used Tensorflow to write programs which ran on the computer with CPU E5-1650@3.6 GHz,RAM 32 GB,TITANxp GPU and operating system of windows 10/64 bit.Due to the limitation of GPU memory size,a large image was cut into some small images of 512×512 pixels.We selected 30 small images and labeled the cells,and then send them into the neural network for 100 times of iterative training.After the training,we selected another image of the adherent cells for recognition test.We selected two commonly used indicators(precision and recall rate)to evaluate the detection effect.The definition of precision and recall rate is as follows.

Where TP is True Positive,FP is False Positive,FN is False Negative.

We counted the precision and recall rate from Figure 2b and the results are shown in Table 1.From Table 1,we can see that the segmentation precision and recall rate of the algorithm reach 99.6% and 99.5% respectively.This accuracy index is quite high and basically meets the actual application requirements.

Fig.2 Adherent cell image and segmented image

Table 1 Segmentation accuracy index of adherent cells

2.2 Suspension cell segmentation

Similar to the same method above,we cut a large cell image into a small image of 512×512 pixels,and randomly selected 30 small images for marking and then sent them to the neural network for 100 times of iterative training.After the training,we chose another image of the suspension cells for recognition test.We selected two commonly used indicators,precision and recall,to evaluate the detection effect.

We counted the precision and recall rate from Figure 3b and the results are shown in Table 2.From Table 2,we can see that the segmentation precision and recall rate of the algorithm are 98.1% and 98.7%respectively,and the result is slightly lower than that of adherent cells.

Fig.3 Suspended cell image and segmented image

Table 2 Segmentation accuracy index of suspension cells

2.3 Cell growth curve plotting

The hybridoma cells were inoculated in a dish and cultured in DMEM medium containing 10%FBS.The images of cells were captured on days 0,1.5 and 3,respectively,with a microscope equipped with a digital camera.

Through our algorithm to count the cells,we can plot the cell growth curve(Figure 4).The number of hybridoma cells was 15 when they were first inoculated,and they proliferated to 123 after 1.5 days and 1 128 after 3 days.Therefore,we can quantitatively analyze the cell proliferation by algorithm recognition statistics.

Fig.4 Cell growth curve

2.4 Cell changes before and after inhibitor addition

We inoculated hybridoma cells in a dish and cultured them in DMEM/FBS medium with 1 μm chestnut spermine without inhibitor.After three days of culture,the images of cells were captured by a microscope with a digital camera.Compared with the images without inhibitor,the inhibition rate of chestnut spermine inhibitor on cells can be calculated.The calculation formula of inhibition rate is as follows.

In the formula,NC0andNY0represent the number of cells in the control group and experimental group on day0,andNCnandNYnrepresent the number of cells in the control group and experimental group on dayn,respectively.

As shown in Figure 5,through our algorithm to count the number of cells,the initial cell number of the control group and the experimental group were 355 and 245 respectively,and the cell number of the control group and the experimental group were 1 727 and 332 respectively after 3 days.According to these data,we can calculate that the inhibition rate of chestnut spermine is 93.6%.

Fig.5 The comparison of cell number for cells with or without inhibitor

3 Conclusion

Deep learning has been widely used in many fields.In this paper,the classic U-Net network structure was used to segment the adherent cells and suspension cells imaged by optical microscope,and the precision and recall rate were calculated respectively.The precision and recall rate are 99.6%and 99.5% for adherent cells,and 98.1% and 98.7%for suspension cells.The two indexes are relatively high in cell segmentation field,which can meet the actual application requirement.At the same time,we used the algorithm to plot the cell growth curve and calculate the inhibition rate of inhibitor,which showed the practicability of the algorithm further.However,in our research,we also found that when cells grew too dense,the accuracy of segmentation with this algorithm would be greatly reduced,which needs to be solved by improving the network structure or combining with other more advanced segmentation algorithms.This is our next research direction.