Jia-Ching Wang

A Novel Video Summarization Based on Mining the Story-Structure and Semantic Relations Among Concept Entities

Video summarization techniques have been proposed for years to offer people comprehensive understanding of the whole story in the video. Roughly speaking, existing approaches can be classified into the two types: one is static storyboard, and the other is dynamic skimming. However, despite that these traditional methods give brief summaries for users, they still do not provide with a concept-organized and systematic view. In this paper, we present a structural video content browsing system and a novel summarization method by utilizing the four kinds of entities: who, what, where, and when to establish the framework of the video contents. With the assistance of the above-mentioned indexed information, the structure of the story can be built up according to the characters, the things, the places, and the time. Therefore, users can not only browse the video efficiently but also focus on what they are interested in via the browsing interface. In order to construct the fundamental system, we employ maximum entropy criterion to integrate visual and text features extracted from video frames and speech transcripts, generating high-level concept entities. A novel concept expansion method is introduced to explore the associations among these entities. After constructing the relational graph, we exploit graph entropy model to detect meaningful shots and relations, which serve as the indices for users. The results demonstrate that our system can achieve better performance and information coverage.

Intensity Gradient Technique for Efficient Intra-Prediction in H.264/AVC

This study presents an intensity gradient approach for intra-prediction in H.264 encoding system, which enhances the performance and efficiency of previous fast algorithms. We propose a preprocessing stage in which eight orientation features are extracted from a macro block by the intensity gradient filters. The orientation features are utilized to select a subset of prediction modes to be involved in the rate-distortion calculation so that the encoding time can be reduced. The simulation results indicate that the intensity gradient based algorithm for intra-prediction contributes better tradeoff between rate-distorion performance and encoding complexity than the previous algorithms. Compared to H.264 reference software, the proposed algorithm introduces slight PSNR degradation and bit rate increase but saves around 76% of the total encoding time with all intra-frame coding.

Robust Environmental Sound Recognition for Home Automation

This work presents a robust environmental sound recognition system for home automation. Specific home automation services can be activated based on identified sound classes. Additionally, when the sound category is human speech, such speech can be recognized for detecting human intentions as in conventional research on home automation. To attain this ambitious goal, this study uses two key techniques: signal-to-noise ratio-aware subspace-based signal enhancement and sound recognition with independent component analysis mel-frequency cepstral coefficients and a frame-based multiclass support vector machines, respectively. Simulations and an experiment in a real-world environment are given to illustrate the performance of the proposed robust sound recognition system.

Chip design of MFCC extraction for speech recognition

The mel frequency cepstral coefficient (MFCC) is one of the most important features required among various kinds of speech applications. In this paper, the first chip for speech features extraction based on MFCC algorithm is proposed. The chip is implemented as an intellectual property, which is suitable to be adopted in a speech recognition system on a chip. The computational complexity and memory requirement of MFCC algorithm are analyzed in detail and improved greatly. The hybrid table look-up scheme is presented to deal with the elementary function value in the MFCC algorithm. Fixed-point arithmetic is adopted to reduce the cost under the accuracy studies of finite word length effect. Finally, the area-efficient design is implemented successfully into the single Xilinx XC4062XL FPGA.

A novel fast algorithm for intra mode decision in H.264/AVC encoders

This paper presents a fast mode decision algorithm for H.264 intra prediction based on dominant edge strength (DES). In H.264 intra prediction, the computation-extensive rate distortion optimization (RDO) technique with full intra mode search is used to select the best mode for each macroblock. To reduce the computational load in mode decision, the DES which is corresponding to a decision mode is detected first. In accordance with the detected dominant edge, a subset of the prediction modes is then chosen for RDO calculation. The proposed algorithm only searches 4 modes instead of 9 for the 4times4 luma blocks. As for the 16times16 or 8times8 chroma blocks, instead of 4 modes, only 2 modes are required to be searched. Experimental results revealed that the computation time of the proposed fast intra prediction algorithm is averagely reduced to 40% of the full search method with slight PSNR degradation

Environmental Sound Classification using Hybrid SVM/KNN Classifier and MPEG-7 Audio Low-Level Descriptor

In this paper, we present a new environmental sound classification architecture. The proposed sound classifier is performed in frame level and fuses the support vector machine (SVM) and the k nearest neighbor rule (KNN). In feature selection, three MPEG-7 audio low-level descriptors, spectrum centroid, spectrum spread, and spectrum flatness are used as the sound features to exploit their ability in sound classification. Experiments carried out on a 12-class sound database can achieve an 85.1 % accuracy rate. The performance comparison between the HMM sound classifier using audio spectrum projection features demonstrates the superiority of the proposed scheme.

Robust Speaker Identification and Verification

Acoustic characteristics have played an essential role in biometrics. In this article, we introduce a robust, text-independent speaker identification/verification system. This system is mainly based on a subspace-based enhancement technique and probabilistic support vector machines (SVMs). First, a perceptual filterbank is created from a psycho-acoustic model into which the subspace-based enhancement technique is incorporated. We use the prior SNR of each subband within the perceptual filterbank to decide the estimators gain to effectively suppress environmental background noises. Then, probabilistic SVMs identify or verify the speaker from the enhanced speech. The superiority of the proposed system has been demonstrated by twenty speaker data taken from AURORA-2 database with added background noises

Unsupervised speaker change detection using SVM training misclassification rate

This work presents an unsupervised speaker change detection algorithm based on support vector machines (SVM) to detect speaker change (SC) in a speech stream. The proposed algorithm is called the SVM training misclassification rate (STMR). The STMR can identify SCs with less speech data collection, making it capable of detecting speaker segments with short duration. According to experiments on the NIST Rich Transcription 2005 Spring Evaluation (RT-05S) corpus, the STMR has a missed detection rate of only 19.67 percent.

Thermomechanical behaviour of metals in cyclic loading

Temperature variation induced by repeated mechanical cyclic loading on AISI 1045 mild steel was studied.The experimental results of cyclic loading at low stress levels elucidate the coupling phenomena of thermal/mechanical behaviour which causes cooling and/or heating corresponding to the stressed state. The governing factors are thermoelastic effect and viscous dissipation. The thermoelastic effect causes the specimen temperature to go down and/or up which corresponds to the loading and/or unloading in cycling, where the viscous dissipation effect causes heat to generate inside the sample which steadily heats the specimen. As a result, a trend of increasing specimen mean temperature with periodical local fluctuation on temperature history can be observed. The heating rate, due to viscous dissipation, is increased with increasing strain rate. Cyclic loading at high stress levels results in large amounts of heat generation where thermoplasticity predominates. An abrupt temperature rise in the first few cycles, followed by a slow-down in later cycling, is to be seen. The phenomena and results were discussed. In addition, the effect of heat transfer between the specimen and its surroundings should be considered for both cases if the time is sufficiently long or the temperature gradient evolved is of significance.

Motion entropy feature and its applications to event-based segmentation of sports video

An entropy-based criterion is proposed to characterize the pattern and intensity of object motion in a video sequence as a function of time. By applying a homoscedastic error model-based time series change point detection algorithm to this motion entropy curve, one is able to segment the corresponding video sequence into individual sections, each consisting of a semantically relevant event. The proposed method is tested on six hours of sports videos including basketball, soccer, and tennis. Excellent experimental results are observed.