Tomáš Grošup

Image exploration using online feature extraction and reranking

We present an image meta-search engine that allows content-based exploration of the results obtained from various sources (mostly based on keyword query). The online feature extraction and the particle physics model are the two key features of our demo application that shows very promising results.

Towards efficient multimedia exploration using the metric space approach

In this paper, we investigate the content-based multimedia exploration techniques benefiting from the metric space indexing approach. We present two orthogonal approaches for browsing multimedia collections and discuss their strong and weak points. We also provide an implementation of the two approaches in our publicly available demo application where users can try to find as much objects of a predefined class as possible, given a limited time and/or a number of clicks.

MLES: Multilayer Exploration Structure for Multimedia Exploration

The traditional content-based retrieval approaches usually use flat querying, where whole multimedia database is searched for a result of some similarity query with a user specified query object. However, there are retrieval scenarios (e.g., multimedia exploration), where users may not have a clear search intents in their minds, they just want to inspect a content of the multimedia collection. In such scenarios, flat querying is not suitable for the first phases of browsing, because it retrieves the most similar objects and does not consider a view on part of a multimedia space from different perspectives. Therefore, we defined a new Multilayer Exploration Structure (MLES), that enables exploration of a multimedia collection in different levels of details. Using the MLES, we formally defined popular exploration operations (zoom-in/out, pan) to enable horizontal and vertical browsing in explored space and we discussed several problems related to the area of multimedia exploration.

Product Exploration based on Latent Visual Attributes

In this demo paper, we present a prototype web application of a product search engine of a fashion e-shop. Although e-shop products consist of full-text description, relational attributes (e.g., price, type, size, color, etc.) as well as visual information (product photo), traditional search engines in e-shops only provide full-text and relational attributes for product filtering. In our retrieval model, we incorporate also the visual information into the search by extracting visual-semantic features using deep convolutional neural networks. Furthermore, visual exploration of the product space using the visual-semantic features (multi-example queries) is used to dynamically discover latent visual attributes that could enhance the original relational schema by fuzzy attributes (e.g., a floral pattern in product). In the demo, we show how these latent attributes could be used to recommend the user preferred products and even outfits (e.g., shoes, bag, jacket) that fit a certain visual style.

A Web Portal for Effective Multi-model Exploration

During last decades, there have emerged various similarity models suitable for specific similarity search tasks. In this paper, we present a web-based portal that combines two popular similarity models (based on feature signatures and SURF descriptors) in order to improve the recall of multimedia exploration. Comparing to single-model approach, we demonstrate in the game-like fashion that a multi-model approach could provide users with more diverse and still relevant results.

On Scalable Approximate Search with the Signature Quadratic Form Distance

The signature quadratic form distance and feature signatures have become a respected similarity space for effective content-based retrieval. Furthermore, the similarity space is configurable by a parameter alpha affecting both retrieval precision and intrinsic dimensionality, and thus interesting trade-offs can be achieved when a metric index is used for exact search. In this paper we combine such configurable model with state of the art approximate search techniques developed for the M-Index. In the experiments, we show that employing a configuration resulting in the best effectiveness of the measure leads also to very competitive approximate search effectiveness when using the M-Index, regardless the high intrinsic dimensionality of the corresponding similarity space.

SIR: the smart image retrieval engine

We present the Smart Image Retrieval meta-search engine that allows content-based exploration of the results obtained from various sources (mostly based on keyword query). The online feature extraction architecture and exploration models utilizing single-/multi-query approaches are the two key features of our demo application that shows very promising results.

Malware Discovery Using Behaviour-Based Exploration of Network Traffic

We present a demo of behaviour-based similarity retrieval in network traffic data. The underlying framework is intended to support domain experts searching for network nodes (computers) infected by malicious software, especially in cases when single client-server communication does not have to be sufficient to reliably identify the infection. The focus is on interactive browsing enabling dynamic changes of the retrieval model, which is based on a recently proposed statistical description (fingerprint) of a communication between two network hosts and the bag of features approach. The demo/framework provides unique insight into the data and enables annotation of the data and model modifications during the search for more effective identification of infected hosts.

Interactive Product Search Based on Global and Local Visual-Semantic Features

In this paper, we present a prototype web application of a product search engine of a fashion e-shop. Today, e-shop product metadata consist of text description, simple attributes (price, size, color, fabric, etc.) and visual information (product photo). Search engines used in e-shops mostly provide text and attribute/category interface for product filtering. In our model, we focus on the visual information applied in an interactive query-by-example scenario. The global visual descriptors may be often ambiguous and may not correspond well with the intended mental query of the user. Therefore, we proposed and evaluated model and GUI allowing user to guide the query process by selecting image regions (patches) of interest within the query. In the demo evaluation, we show that allowing user to specify relevant image patches led to a significant improvement of the results’ relevance in the vast majority of tested queries.

Evaluating Multilayer Multimedia Exploration

Multimedia exploration is an entertaining approach for multimedia retrieval enabling users to interactively browse and navigate through multimedia collections in a content-based way. The multimedia exploration approach extends the traditional query-by-example retrieval scenario to be a more intuitive approach for obtaining a global overview over an explored collection. However, novel exploration scenarios require many user studies demonstrating their benefits. In this paper, we present results of an extensive user study focusing on the comparison of 3-layer Multilayer Exploration Structure MLES structure with standard flat k-NN browsing. The results of the user study show that principles of the MLES lead to better effectiveness of the exploration process, especially when searching for a first object of the searched concept in an unknown collection.