Francisco Zamora-Martínez

Time-Series Forecasting of Indoor Temperature Using Pre-trained Deep Neural Networks

Artificial neural networks have proved to be good at time-series forecasting problems, being widely studied at literature. Traditionally, shallow architectures were used due to convergence problems when dealing with deep models. Recent research findings enable deep architectures training, opening a new interesting research area called deep learning. This paper presents a study of deep learning techniques applied to time-series forecasting in a real indoor temperature forecasting task, studying performance due to different hyper-parameter configurations. When using deep models, better generalization performance at test set and an over-fitting reduction has been observed.

Online learning algorithm for time series forecasting suitable for low cost wireless sensor networks nodes.

Time series forecasting is an important predictive methodology which can be applied to a wide range of problems. Particularly, forecasting the indoor temperature permits an improved utilization of the HVAC (Heating, Ventilating and Air Conditioning) systems in a home and thus a better energy efficiency. With such purpose the paper describes how to implement an Artificial Neural Network (ANN) algorithm in a low cost system-on-chip to develop an autonomous intelligent wireless sensor network. The present paper uses a Wireless Sensor Networks (WSN) to monitor and forecast the indoor temperature in a smart home, based on low resources and cost microcontroller technology as the 8051MCU. An on-line learning approach, based on Back-Propagation (BP) algorithm for ANNs, has been developed for real-time time series learning. It performs the model training with every new data that arrive to the system, without saving enormous quantities of data to create a historical database as usual, i.e., without previous knowledge. Consequently to validate the approach a simulation study through a Bayesian baseline model have been tested in order to compare with a database of a real application aiming to see the performance and accuracy. The core of the paper is a new algorithm, based on the BP one, which has been described in detail, and the challenge was how to implement a computational demanding algorithm in a simple architecture with very few hardware resources.

F-measure as the error function to train neural networks

Imbalance datasets impose serious problems in machine learning. For many tasks characterized by imbalanced data, the F-Measure seems more appropiate than the Mean Square Error or other error measures. This paper studies the use of F-Measure as the training criterion for Neural Networks by integrating it in the Error-Backpropagation algorithm. This novel training criterion has been validated empirically on a real task for which F-Measure is typically applied to evaluate the quality. The task consists in cleaning and enhancing ancient document images which is performed, in this work, by means of neural filters.

Improving a DTW-Based Recognition Engine for On-line Handwritten Characters by Using MLPs

Our open source real-time recognition engine for on-line isolated handwritten characters is a 3-Nearest Neighbor classifier that uses approximate dynamic time warping comparisons with a set of prototypes filtered by two fast distance-based methods. This engine achieved excellent classification rates on two writer-independent tasks:UJIpenchars and Pendigits. We present the integration of multilayer perceptrons into our engine, an improvement that speeds up the recognition process by taking advantage of the independence of these networks’ classification times from training set sizes. We also present experimental results on our new publicly available UJIpenchars2 database and on Pendigits.

Insights on the Use of Convolutional Neural Networks for Document Image Binarization

Convolutional Neural Networks have systematically shown good performance in Computer Vision and in Handwritten Text Recognition tasks. This paper proposes the use of these models for document image binarization. The main idea is to classify each pixel of the image into foreground and background from a sliding window centered at the pixel to be classified. An experimental analysis on the effect of sensitive parameters and some working topologies are proposed using two different corpora, of very different properties: DIBCO and Santgall.

Stacked Denoising Auto-Encoders for Short-Term Time Series Forecasting

In this chapter, a study of deep learning of time-series forecasting techniques is presented. Using Stacked Denoising Auto-Encoders, it is possible to disentangle complex characteristics in time series data. The effects of complete and partial fine-tuning are shown. SDAE prove to be able to train deeper models, and consequently to learn more complex characteristics in the data. Hence, these models are able to generalize better. Pre-trained models show a better generalization when used without covariates. The learned weights show to be sparse, suggesting future exploration and research lines.

Improving isolated handwritten word recognition using a specialized classifier for short words

The aim of this work is to improve the performance of off-line handwritten text recognition systems based on hidden Markov models (HMM) and hybrid Markov models with neural networks (HMM/ANN). In order to study the systems without the influence of the language model, an isolated word recognition task has been performed. The analysis of the influence of word lengths on the error rates of the recognizers has lead to combine those classifiers with another one specialized in short words. To this end, various multilayer perceptrons have been trained to classify a subset of the vocabulary in a holistic manner. Combining the classifiers by means of a variation of the Borda count voting method achieves very satisfying results.

A flexible system for document processing and text transcription

STATE is a flexible system for document processing. It comprises a graphical front-end that can be easily connected to different text recognition back-ends. We comment here the front-end and two backends: one based on nearest neighbors and one based on Hidden Markov Models. Experimentation shows that if the back-end has a moderately low character error rate, productivity gains can be as high as 100% when compared to directly transcribing the text.

Speech interaction in a multimodal tool for handwritten text transcription

STATE is a multimodal tool for document processing and text transcription. Its graphical front-end can be easily connected to different text recognition back-ends. New features and improvements are presented in this work: the interactive correction of one word in the transcribed line has been improved to reestimate the entire transcription line using the user feedback and speech input has been integrated in the multimodal interface enabling the user to also utter the word to be corrected, giving the user the possibility to use the interface according to her preferences or the task at hand. Thus, at the current version of STATE, the user can type, write on the screen with a stylus, or utter the incorrectly recognized word, and then, the system uses the user feedback in any of the proposed modalities to reestimate the transcribed line so as to hopefully correct other errors which could be caused by the mistaken word the user has corrected.

Adding morphological information to a connectionist part-of-speech tagger

In this paper, we describe our recent advances on a novel approach to Part-Of-Speech tagging based on neural networks. Multilayer perceptrons are used following corpus-based learning from contextual, lexical and morphological information. The Penn Treebank corpus has been used for the training and evaluation of the tagging system. The results show that the connectionist approach is feasible and comparable with other approaches.