Engin Mendi

Shot boundary detection and key frame extraction using salient region detection and structural similarity

In this paper, we present a novel algorithm for shot boundary detection and key frame extraction from video sequences. Saliency maps representing the attended regions are produced from the color and luminance features of the video frames. Introducing a novel signal fidelity measurement -saliency based structural similarity index- the similarity of the maps is measured. Based on the similarities, shot boundaries and key frames are determined. Proposed algorithm is tested on neurosurgical videos and precision and recall performances are measured. Experimental results validate effectiveness of the proposed shot boundary detection and key frame extraction algorithm. Moreover, the algorithm is robust to dissolving digital video effects used in shot transition.

Sports video summarization based on motion analysis

Non-annotated video is more common than ever and this fact leads to an emerging field called video summarization. Key frame selection using motion analysis can greatly increase the understanding of the video content by presenting a series of frames summarizing the intended video. In this paper, we present an automatic video summarization technique based on motion analysis. The proposed technique defines motion metrics estimated from two optical flow algorithms, each using two different key frame selection criteria. We conducted a subjective user study to evaluate the performance of the motion metrics. The summarization process is threshold free and experimental results have verified the effectiveness of the method.

Food intake monitoring system for mobile devices

In this paper, we introduce a real-time food intake monitoring system for mobile devices. The proposed system gets acceleration data from the sensor placed on the wrist of the user during a meal. The data is then sent to the mobile device via Bluetooth. The system analyses patterns between the motion profile and bite actions by first filtering the data to remove noise effects and then identifying the peaks. Based on detecting peaks, real-time feedback regarding eating trends such as total number of bites, bites-taken rate and eating speed is provided to the user. If the eating is too fast, the system warns the user in the form of both audio and text. The mobile application is implemented on the Android Platform and tested on a subject successfully. The proposed system offers an affordable quick solution that can be used in any place where eating happens. It can help people who are obese or with other eating disorders by monitoring consumption of food intake to control their eating rate in realtime.

HMM based classification of sports videos using color feature

Video content classification is an important element for efficient access and retrieval of video in any media content management system. Categorizing the video segments can help to provide convenience and ease in accessing the relevant video content without sequential scanning. In this paper, we present a Hidden Markov Model (HMM) based classification technique for sports videos. Speed of color changes is computed for each video frame and used as observation sequences in HMM for classification. Experiments using more than 1 hour of 18 training and 18 testing sports videos of 3 predefined genres (golf, hockey and football) give very satisfactory classification accuracy.

Contour-Based Image Segmentation Using Selective Visual Attention

In many medical image segmentation applications identifying and extracting the region of interest (ROI) accurately is an important step. The usual approach to extract ROI is to apply image segmentation methods. In this paper, we focus on extracting ROI by segmentation based on visual attended locations. Chan-Vese active contour model is used for image segmentation and attended locations are determined by SaliencyToolbox. The implementation of the toolbox is extension of the saliency map-based model of bottom-up attention, by a process of inferring the extent of a proto-object at the attended location from the maps that are used to compute the saliency map. When the set of regions of interest is selected, these regions need to be represented with the highest quality while the remaining parts of the processed image could be represented with a lower quality. The method has been successfully tested on medical images and ROIs are extracted.

Content-based management service for medical videos.

Development of health information technology has had a dramatic impact to improve the efficiency and quality of medical care. Developing interoperable health information systems for healthcare providers has the potential to improve the quality and equitability of patient-centered healthcare. In this article, we describe an automated content-based medical video analysis and management service that provides convenience and ease in accessing the relevant medical video content without sequential scanning. The system facilitates effective temporal video segmentation and content-based visual information retrieval that enable a more reliable understanding of medical video content. The system is implemented as a Web- and mobile-based service and has the potential to offer a knowledge-sharing platform for the purpose of efficient medical video content access.

Facial animation framework for web and mobile platforms

In this paper, we present a realistic facial animation framework for web and mobile platforms. The proposed system converts the text into 3D face animation with synthetic voice, ensuring synchronization of the head and eye movements with emotions and word flow of a sentence. The expression tags embedded in the input sentences turn into given emotion on the face while the virtual face is speaking. The final face motion is obtained by interpolating the keyframes over time to generate transitions between facial expressions. Visual results of the animation are sufficient for web and mobile environments. The proposed system may contribute to the development of various new generation e-Health applications such as intelligent communication systems, human-machine interfaces and interfaces for handicapped people.

Text-to-audiovisual speech synthesizer for children with learning disabilities.

Learning disabilities affect the ability of children to learn, despite their having normal intelligence. Assistive tools can highly increase functional capabilities of children with learning disorders such as writing, reading, or listening. In this article, we describe a text-to-audiovisual synthesizer that can serve as an assistive tool for such children. The system automatically converts an input text to audiovisual speech, providing synchronization of the head, eye, and lip movements of the three-dimensional face model with appropriate facial expressions and word flow of the text. The proposed system can enhance speech perception and help children having learning deficits to improve their chances of success.

Agent based pornography filtering system

Internet is an infinite information repository that also contains harmful contents like; pornography, violence, and hate messages. It is very important to obstruct these kinds of contents from underage children not to adversely affect their development. Today, there are many commercial software products developed for this purpose. But the filtering capabilities of these commercial software are limited to text based and image based contents. Different techniques must be used to filter video based contents. This article describes an agent-based system which is developed for the detection of videos containing pornographic contents. Videos on the Internet can be divided into six groups as; anime, commercial, music, sitcom, sports, and porn related. The proposed system uses the Hidden Markov Model (HMM) based classification technique to classify the videos into these predefined categories with intelligent agents. Color features are extracted from each video frame and used as observation sequences in HMM for classification. According to the classification results, the videos, which are closely related to the category of sex and pornography, are filtered to the underage users. The test results obtained indicate that the classification has been satisfactory.

Extraction of human body contours and position analysis

The paper presents a new method for extracting and positioning contours of moving objects. The method is applicable in the surveillance of elderly individuals and facilitates the detection of critical situations when the elderly individuals find themselves in need of immediate help. For this, single frames from the video sequence are extracted in regular time intervals and the position of the human body is analyzed. The position is traced using the body contours. For the surveillance, a static video camera is used. The body contours extraction is performed using two consecutive filters: the first - for the adaptive texture suppression, and the second - for the contours extraction. The main presumption is that the environment is known and this makes the influence of multiple variable parameters (illumination, shadows, etc.) lower. A Fourier descriptor is used for the body position analysis.