The digitalization of medical images has allowed in recent years the automation of many tasks with the aim of supporting the clinicians in his diagnostic work.

Histological images, obtained through tissue samples, have also been digitized thanks to the powerful scanning microscopes available on the market, which are gradually adopted by the pathological anatomy services in hospitals. In this context, software packages that facilitate the daily operation of pathologists emerge.

The Computer Vision group in Tecnalia is currently working in diagnosis support systems for the pathologists, such as the histological images search engine based on similar visual content and clinical relevance, which deserved 1st EARTO Innovation Prize in 2014.

Under this strategic approach, one of the most tedious tasks that pathologists must perform is mitosis counting in a tissue sample that turns out to be tumoral. The lower or higher proliferation of mitosis in a tissue is one of the most relevant prognostic indicators in some pathologies, such as in invasive breast carcinoma. This proliferation is expressed as the number of mitosis per tissue area. A high rate of mitosis proliferation reveals the aggressiveness of the tumor and is a relevant indicator of prognosis. The number of mitoses is routinely evaluated every day in pathological anatomy laboratories around the world, usually after visual examination on the microscope, configured with a large zoom, usually × 40, on samples stained with hematoxylin and eosin (H&E). The standard clinical procedure recommends selecting an area that includes the most invasive zone of the tumor, where the highest cellularity is observed. The identification and counting of mitosis is done manually.

Automatic detection of mitosis can provide with a precision that is crucial to identify the severity of the disease. However, the automatic characterization of mitotic elements is a complicated problem since a mitotic figure maybe be visually similar to other elements present in the image. Moreover, mitosis has four main phases and each phase present different shape and texture. Therefore, there is no simple way to detect mitosis in terms of colour, shape and texture features, although there are several approaches in the literature that have addressed it in this way.

Example of mitosis (green arrows) in histological image together with other elements that present visual similarity (TUPAC Challenge 2016)

The analysis of highly complex images is a challenge that has been tackled in image processing discipline. Up to now, acceptable results have been obtained in highly controlled environments using the standard image processing procedure: preprocessing, feature extraction, classification. However, this feature extraction presents high dependence on the application.

In recent years, techniques based on convolutional neural networks integrated in deep architectures have been popularized. This is referred as Deep Learning. These architectures allow feature extraction in the first layers and an adaptation to them by the final classifier simultaneously. It simulates the human visual system. This allows solving problems of classification, detection and image segmentation that had not been solved until now, with a much greater generalization capacity than was previously feasible.

Tecnalia’s Computer Vision group is addressing mitosis detection and counting by means of these Deep Learning techniques.