AZTI has developed, and is operating for more than 10 years, videometry systems for monitoring sandy shores and bathing areas, equipped with cameras that capture images from a fixed point of view at known time intervals. The products derived from the treatment of the images obtained and the referencing of this information using photogrammetric techniques provide direct information of great value, for the continuous understanding and monitoring of different processes and uses of the coastline. In this sense, AZTI has developed its own technology called KostaSystem® (www.kostasystem.com).
The Marine Technologies group of AZTI has collaborated for years with TECNALIA’s Computer Vision Research Group. Specifically, in the development of advanced image analysis algorithms that KostaSystem cameras collect, such as the “Run-up” and “Rebase” algorithms described in the blog post: “AUTOMATIC INFORMATION ANALYSIS OF COASTAL EMERGENCY INFORMATION BY VIDEOMETRY”.
TECNALIA has also developed an Automatic Beach Segmentation and Occupancy Estimation algorithm for AZTI. The Hondartza software aims, given an image of a beach, the classification of each of the pixels between person or non-person with the goal of calculating their occupation. In the field of Computer Vision, this process of separating an image into two or more regions pixel by pixel is called segmentation.
The decision of whether a pixel corresponds to a person or not may seem trivial but implementing an algorithm that automatically performs this task is complex, mainly due to the difficulty of processing outdoor images with changing light and weather conditions. To address this challenge, it was decided to apply Machine Learning algorithms, belonging to the world of artificial intelligence, for supervised segmentation. These algorithms are provided with a series of examples, so that it is possible to learn relevant characteristics and then extrapolate them to new images that have not been used to learn. This learning process is called training, while the process of extrapolation to new images is called prediction.
And that is the strategy used by Hondartza software. The procedure implemented consists of two steps: first, the training of the statistical model and second, the application of the model for the segmentation of beach images.
In order to be able to “teach” the software to determine if a pixel is person or beach, it is first required to study what characteristics or quantities associated with that pixel can represent it best. In this case, it has been decided to use the response of each pixel to a filter bank defined by Leung and Malik. From all the statistical descriptors available then, an exhaustive selection has been made, since, for computational efficiency, it is not possible to use all of them. Each of the training images must have a ground-truth, that is, a manual segmentation made (or supervised) by a person. Analysing the response to this preselection of training image filters and taking into account whether the pixel corresponds to a person or beach, a global statistical model is constructed, using the AdaBoost algorithm. The model is then passed on to the part of the software dedicated to prediction so that, based on the knowledge included in the model, it is possible to apply exactly the same calculation of statistical descriptors in a new image. Depending on the responses of the pixels in this image and comparing them with the responses of the training images, Hondartza software is capable of efficiently classifying the pixels as corresponding to person or beach.
Once we have a “map” of responses to pixels that we know what their true classification is, we can apply it to the detection and segmentation of people on beaches. This step also corresponds to the AdaBoost algorithm, and whose result has been improved using a segmentation refinement algorithm, called a guided filter.
To get a better performance and not process areas of the image that are known to be non-sand, masks are defined with the regions in which to apply the algorithm. The result is the one shown in the following figure:
Hondartza software was developed prior to the explosion of Deep Learning and convolutional neural networks, and during this time it has been generating valuable information on the use of beaches. In the future, it is planned to incorporate convolutional network technology into the algorithm that allows the extraction of additional and more accurate information from the coastal environment.
The Environmental Directorate of the Provincial Council of Gipuzkoa has developed a videometry network that applies the KostaSystem technology and that covers 10 sandy areas throughout the territory (Ondarbeltz, Deba, Saturraran, Itzurun, Santiago, Gaztetape, Malkorbe, Antilla, Zarautz and Hondarribia). The Gipuzkoa Provincial Council has signed an agreement with AZTI to exploit the information on the network. The type of information provided by these videometry systems is useful to know the capacity and increase safety for the bath and in general improve the management of the uses that occur on the beaches, generating useful information for the people. Likewise, on a broader time scale, they provide information on the processes that govern coastal dynamics, with a high value at the level of scientific knowledge that supports the management of the sand.
Due to the current health crisis caused by the COVID-19, coastal municipalities, as managers of the use of the sand in their municipalities, have the need to regulate access and limit the density of use of their beaches in based on the minimum social distance recommended by the competent authorities. The Provincial Council of Gipuzkoa, in order to make available to the public the best possible information regarding the enjoyment of the beaches during the summer season in a responsible and safe manner, contacted AZTI to use all these tools and infrastructure available to know in a simple way the density of occupation of the beaches in real time.
In this context, it has been decided to design a system that automatically and continuously applies the occupancy calculation algorithms on the beaches of the Gipuzkoa coastline and displays it through a mobile application. For this, an information exchange structure has been set up in which the images of the beaches with their configuration files and data to be displayed are stored, an application has been developed, that executes the occupation algorithm at defined intervals and that generates cuts of the images and additional information for a mobile application, NIK HONDARTZAK, developed by TOKITEK, to share all this information with the users. The following figure shows a brief description of the process.
This application is an example of how Computer Vision is applied to our daily life and helps to find solutions to the challenges that are presented to us. It is also clear that the collaboration between Technological Centers and Administrations benefits the entire society.