José Gabriel Rodríguez Carneiro Gomes

Semantic-Friendly Indexing and Quering of Images Based on the Extraction of the Objective Semantic Cues

Abstract image semantics resists all forms of modeling, very much like any kind of intelligence does. However, in order to develop more satisfying image navigation systems, we need tools to construct a semantic bridge between the user and the database. In this paper we present an image indexing scheme and a query language, which allow the user to introduce cognitive dimension to the search. At an abstract level, this approach consists of: (1) learning the “natural language” that humans speak to communicate their semantic experience of images, (2) understanding the relationships between this language and objective measurable image attributes, and then (3) developing corresponding feature extraction schemes.More precisely, we have conducted a number of subjective experiments in which we asked human subjects to group images, and then explain verbally why they did so. The results of this study indicated that a part of the abstraction involved in image interpretation is often driven by semantic categories, which can be broken into more tangible semantic entities, i.e. objective semantic indicators. By analyzing our experimental data, we have identified some candidate semantic categories (i.e. portraits, people, crowds, cityscapes, landscapes, etc.) and their underlying semantic indicators (i.e. skin, sky, water, object, etc.). These experiments also helped us derive important low-level image descriptors, accounting for our perception of these indicators.We have then used these findings to develop an image feature extraction and indexing scheme. In particular, our feature set has been carefully designed to match the way humans communicate image meaning. This led us to the development of a “semantic-friendly” query language for browsing and searching diverse collections of images.We have implemented our approach into an Internet search engine, and tested it on a large number of images. The results we obtained are very promising.

Scanning and processing 3D objects for Web display

We present a case study of scanning 3D objects for the purposes of education and public information. We begin by describing the original design of a 3D scanning system now in use in Cairos Egyptian Museum. The system captures both the geometry and surface color and detail of museum artifacts. We report on the experience using the system in the museum setting, and how practical problems with the system were addressed. We present samples of how the processed 3D data will be used on a Web site designed to communicate Egyptian culture.

A Variational Approach to Recovering a Manifold from Sample Points

We present a novel algorithm for recovering a smooth manifold of unknown dimension and topology from a set of points known to belong to it. Numerous applications in computer vision can be naturally interpreted as instanciations of this fundamental problem. Recently, a non-iterative discrete approach, tensor voting, has been introduced to solve this problem and has been applied successfully to various applications. As an alternative, we propose a variational formulation of this problem in the continuous setting and derive an iterative algorithm which approximates its solutions. This method and tensor voting are somewhat the differential and integral form of one another. Although iterative methods are slower in general, the strength of the suggested method is that it can easily be applied when the ambient space is not Euclidean, which is important in many applications. The algorithm consists in solving a partial differential equation that performs a special anisotropic diffusion on an implicit representation of the known set of points. This results in connecting isolated neighbouring points. This approach is very simple, mathematically sound, robust and powerful since it handles in a homogeneous way manifolds of arbitrary dimension and topology, embedded in Euclidean or non-Euclidean spaces, with or without border. We shall present this approach and demonstrate both its benefits and shortcomings in two different contexts: (i) data visual analysis, (ii) skin detection in color images.

Using the Vector Distance Functions to Evolve Manifolds of Arbitrary Codimension

We present a novel method for representing and evolving objects of arbitrary dimension. The method, called the Vector Distance Function (VDF) method, uses the vector that connects any point in space to its closest point on the object. It can deal with smooth manifolds with and without boundaries and with shapes of different dimensions. It can be used to evolve such objects according to a variety of motions, including mean curvature. If discontinuous velocity fields are allowed the dimension of the objects can change. The evolution method that we propose guarantees that we stay in the class of VDFs and therefore that the intrinsic properties of the underlying shapes such as their dimension, curvatures can be read off easily from the VDF and its spatial derivatives at each time instant. The main disadvantage of the method is its redundancy: the size of the representation is always that of the ambient space even though the object we are representing may be of a much lower dimension. This disadvantage is also one of its strengths since it buys us flexibility.

The Vector Distance Functions

We present a novel method for representing and evolving objects of arbitrary dimension. The method, called the Vector Distance Function (VDF) method, uses the vector that connects any point in space to its closest point on the object. It can deal with smooth manifolds with and without boundaries and with shapes of different dimensions. It can be used to evolve such objects according to a variety of motions, including mean curvature. If discontinuous velocity fields are allowed the dimension of the objects can change. The evolution method that we propose guarantees that we stay in the class of VDFs and therefore that the intrinsic properties of the underlying shapes such as their dimension, curvatures can be read off easily from the VDF and its spatial derivatives at each time instant. The main disadvantage of the method is its redundancy: the size of the representation is always that of the ambient space even though the object we are representing may be of a much lower dimension. This disadvantage is also one of its strengths since it buys us flexibility.

Current mode read-out circuit for InGaAs photodiode applications

Infrared focal plane arrays have many military, industrial, medical, and scientific applications that require high-resolution and high-performance read-out electronics. In applications involving InGaAs sensor arrays, data read-out can be carried out by circuits implemented with 0.35@mm CMOS technology. In this paper we propose a dynamically regulated cascode current mirror for pixel read-out. From simulation results, we expect this circuit to achieve a better trade-off between silicon area, signal-to-noise ratio, and output dynamic range than the trade-off that is currently achieved by current mode CMOS read-out circuits.

Image-based object editing

We examine the problem of editing complex 3D objects. We convert the problem of editing a 3D object of arbitrary size and surface properties to a problem of editing a 2D image. We allow the user to specify edits in both geometry and surface properties from any view and at any resolution they find convenient, regardless of the interactive rendering capability of their computer. We use specially-constrained shape from shading algorithms to convert a shaded image specified by the user into a 3D geometry.

Computation of the complexity of vector quantizers by affine modeling

Several methods for evaluation of the complexity of data compression systems and for including complexity measures in the traditional rate-distortion analysis have been published in recent works. In this work, we indicate that the relationship between rate-distortion performance and complexity for some practical coding schemes-entropy-constrained vector quantization (ECVQ) and interpolative vector quantization (IVQ)-can be represented by affine models. For the same rate-distortion performance, the complexity of an interpolative vector quantizer is known to be significantly smaller than the complexity of a full-search entropy constrained vector quantizer, and this complexity difference is a suitable illustration for the rate-distortion-complexity framework. We use high-resolution theory arguments to derive the affine models for ECVQ and IVQ. The proposed affine complexity modeling successfully predicts the cost of vector quantizers designed from data sets that were not used to generate the models.

ISee: perceptual features for image library navigation

To develop more satisfying image navigation systems, we need tools to construct a semantic bridge between the user and the database. In this paper we present an image indexing scheme and a query language, which allow the user to introduce a cognitive dimension to the search. At an abstract level, this approach consists of: 1) learning the natural language that humans speak to communicate their semantic experience of images, 2) understand the relationships between this language and objective measurable image attributes, and then 3) develop the corresponding feature extraction schemes. In our previous work we have conducted a number of subjective experiments in which we asked human subjects to group images, and then explain verbally why they did so. The results of this study indicated that part of the abstraction involved in image interpretation is often driven by semantic categories, which can be broken into more tangible semantic entities, i.e. objective semantic indicators. By analyzing our experimental data, we identified some candidate semantic categories (i.e. portraits, people, crowds, cityscapes, landscapes, etc.), discovered their underlying semantic indicators (i.e. skin, sky, water, object, etc.), and derived important low-level image descriptors accounting for our perception of these indicators. In our recent work we have used these findings to develop a set of image features that match the way humans communicate image meaning, and a semantic-friendly query language for browsing and searching diverse collections of images. We have implemented our approach into an Internet search engine, ISee, and tested it on a large number of images. The results we obtained are very promising.

A CMOS imaging system featuring focal-plane compression based on DPCM and VQ

We recently introduced a new focal plane image compression algorithm that is implemented with 607 transistors inside every 4 × 4 pixel block of a CMOS imager, using conventional 0.35 μm integration technology. This work focuses on preliminary results concerning the overall MTF of an imaging system in which the CMOS imager features focal-plane data compression based on DPCM and VQ with an overall bit rate below 0.94 bpp. Using bar-target pattern inputs, it is shown that details up to 2 cycles/cm are preserved in the decoded images.