Christin Seifert

Visualizations of Deep Neural Networks in Computer Vision: A Survey

In recent years, Deep Neural Networks (DNNs) have been shown to outperform the state-of-the-art in multiple areas, such as visual object recognition, genomics and speech recognition. Due to the distributed encodings of information, DNNs are hard to understand and interpret. To this end, visualizations have been used to understand how deep architecture work in general, what different layers of the network encode, what the limitations of the trained model was and how to interactively collect user feedback. In this chapter, we provide a survey of visualizations of DNNs in the field of computer vision. We define a classification scheme describing visualization goals and methods as well as the application areas. This survey gives an overview of what can be learned from visualizing DNNs and which visualization methods were used to gain which insights. We found that most papers use Pixel Displays to show neuron activations. However, recently more sophisticated visualizations like interactive node-link diagrams were proposed. The presented overview can serve as a guideline when applying visualizations while designing DNNs.

Focus Paragraph Detection for Online Zero-Effort Queries: Lessons learned from Eye-Tracking Data

In order to realize zero-effort retrieval in a web-context, it is crucial to identify the part of the web page the user is focusing on. In this paper, we investigate the identification of focus paragraphs in web pages. Starting from a naive baseline for paragraph and focus paragraph detection, we conducted an eye-tracking study to evaluate the most promising features. We found that single features (mouse position, paragraph position, mouse activity) are less predictive for gaze which confirms findings from other studies. The results indicate that an algorithm for focus paragraph detection needs to incorporate a weighted combination of those features as well as additional features, e.g. semantic context derived from the users web history.

Most Important First – Keyphrase Scoring for Improved Ranking in Settings With Limited Keyphrases

Automatic keyphrase extraction attempts to capture keywords that accurately and extensively describe the document while being comprehensive at the same time. Unsupervised algorithms for extractive keyphrase extraction, i.e. those that filter the keyphrases from the text without external knowledge, generally suffer from low precision and low recall. In this paper, we propose a scoring of the extracted keyphrases as post-processing to rerank the list of extracted phrases in order to improve precision and recall particularly for the top phrases. The approach is based on the tf-idf score of the keyphrases and is agnostic of the underlying method used for the initial extraction of the keyphrases. Experiments show an increase of up to 14% at 5 keyphrases in the F1-metric on the most difficult corpus out of 4 corpora. We also show that this increase is mostly due to an increase on documents with very low F1-scores. Thus, our scoring and aggregation approach seems to be a promising way for robust, unsupervised keyphrase extraction with a special focus on the most important keyphrases.

QueryCrumbs: A Compact Visualization for Navigating the Search Query History

Models of human information seeking reveal that search, in particular ad-hoc retrieval, is non-linear and iterative. Despite these findings, todays search user interfaces do not support non-linear navigation, like for example backtracking in time. In this work, we propose QueryCrumbs, a compact and easy-to-understand visualization for navigating the search query history supporting iterative query refinement. We apply a multi-layered interface design to support novices and firsttime users as well as intermediate users. The formative evaluation with first-time and intermediate users showed that the interactions can be easily performed, and the visual encodings were well understood without instructions. Results indicate that QueryCrumbs can support users when searching for information in an iterative manner.