Common search engines like Google or Bing still just rely on text-based analysis, providing real time results with great accuracy. However, sometimes, the page rank implemented by these search engines does not provide the web-pages that we are looking for. Sergio Rodriguez-Vaamonde, researcher at Tecnalia Computer Vision Area, under the supervision of Prof. Lorenzo Torresani from Dartmouth College and in collaboration with Microsoft Research have proved that image information can be included in text-based search engines to increase the search accuracy.

In order to test this hypothesis, Sergio developed a search engine that analyses the page rank returned by a text search and extracts the visual information associated to each web-page image. The algorithm uses computer vision and artificial intelligence techniques to properly describe and categorize the images and use this information to re-rank the results of the text search engine.

Performed test on the TREC datasets reveal that the estimated precision of the search when incorporating this visual information is increased from 48.2% up to 64.5% in one of the top performing search engines. They presented their results at ACM SIGIR Conference on Research and Development in Information Retrieval in Dublin this summer, being received with great interest by key players such as Google, Microsoft and Baidu.

5 years ago, Tecnalia started an Image Understanding and Recognition research line devoted to the extraction, analysis and structuring of the visual information contained in images. This area has developed content-based image retrieval (CBIR) and analysis systems in different fields, such as automated media content analysis and tagging, tourism, medical imaging, steel industry and biology.

More information:


[Research paper]: What Can Pictures Tell Us About Web Pages? Improving Document Search using Images

[The Dartmouth]