Helman Stern

Vision-based hand-gesture applications

Body posture and finger pointing are a natural modality for human-machine interaction, but first the system must know what its seeing.

Minimal Resources for Fixed and Variable Job Schedules

We treat the following problem: There are n jobs with given processing times and an interval for each jobs starting time. Each job must be processed, without interruption, on any one of an unlimited set of identical machines. A machine may process any job, but no more than one job at any point in time. We want to find the starting time of each job such that the number of machines required to process all jobs is minimal. In addition, the assignment of jobs to each machine must be found. If every job has a fixed starting time the interval is a point, the problem is well-known as a special case of Dilworths problem. We term it the fixed job schedule problem FSP. When the job starting times are variable, the problem is referred to as the variable job schedule problem VSP, for which no known exact solution procedure exists. We introduce the problems by reviewing previous solution methods to Dilworths problem. We offer an approximate solution procedure for solving VSP based on the entropy principle of informational smoothing. We then formulate VSP as a pure integer programming problem and provide an exact algorithm. This algorithm examines a sequence of feasibility capacitated transportation problems with job splitting elimination side constraints. Our computational experience demonstrates the utility of the entropy approach.

Routing electric meter readers

Electric utility companies employ a crew of workers who periodically visit and read the electric meters of each customer in their service area. Each reader is transported by auto from a central office to the first customer on his work list. At the end of his work shift time limit the reader is free to leave the area possibly returning home or to the central office by public bus. Taking a graph that corresponds to the city network of streets meter readers must traverse each street while moving from house to house. It is possible that dead heading may be required—back tracking over a street that has already been covered. A working tour is an open path whose reading time plus deadheading time does not exceed the work limit. The problem is to find the minimum number of working tours. Stating the problem in this manner gives us an optimization problem closely related to the M-Chinese postman problem—an edge oriented routing problem. After presenting some background on this type of problem a heuristic procedure is used to solve an example from the City of Beersheva. The solution provides a 40% reduction in the number of working tours. The paper ends with a discussion of the solution, and provides conditions under which the algorithm should have practical utility.

Adaptive color space switching for face tracking in multi-colored lighting environments

There are many studies that use color space models (CSM) for detection of faces in an image. Most researchers a priori select a given CSM, and proceed to use the selected model for color segmentation of the face by constructing a color distribution model (CDM). There is limited work on finding the overall best CSM. We develop a procedure to adaptively change the CSM throughout the processing of a video. We show that this works in environments where the face moves through multi-positioned light sources with varying types of illumination. A test of the procedure using the 2D color space models; RG, rg, HS, YQ and CbCr found that switching between the color spaces resulted in increased tracking performance. In addition, we have proposed a new performance measure for evaluating color-tracking algorithms, which include both accuracy and robustness of the tracking window. The methodology developed can be used to find the optimal CSM-CDM combination in adaptive color tracking systems.

Adaptive color space switching for tracking under varying illumination

Abstract Many studies use color space models (CSM) and color distribution models (CDM) for detection of faces in an image. We develop a procedure that adaptively switches CSMs throughout the processing of a video. We show that this works in environments with varying types of illumination. In addition, a new performance measure for evaluating tracking algorithms is proffered. Extensive testing of the procedure found that switching between the color spaces resulted in increased tracking performance when compared to using single CSMs throughout. The methodology developed can be used to find the optimal CSM–CDM combination in the design of adaptive color tracking systems. The adaptive color space switching algorithm has linear computational time complexity O( S ), at each iteration, where S is the picture size in pixels.

Sensors for Gesture Recognition Systems

A gesture recognition system (GRS) is comprised of a gesture, gesture-capture device (sensor), tracking algorithm (for motion capture), feature extraction, and classification algorithm. With the impending movement toward natural communication with mechanical and software systems, it is important to examine the first apparatus that separates the human communicator and the device being controlled. Although there are numerous reviews of GRSs, a comprehensive analysis of the integration of sensors into GRSs and their impact on system performance is lacking in the professional literature. Thus, we have undertaken this effort. Determination of the sensor stimulus, context of use, and sensor platform are major preliminary design issues in GRSs. Thus, these three components form the basic structure of our taxonomy. We emphasize the relationship between these critical components and the design of the GRS in terms of its architectural functions and computational requirements. In this treatise, we consider sensors that are capable of capturing dynamic and static arm and hand gestures. Although we discuss various sensor types, our main focus is on visual sensors as we expect these to become the sensor of choice in the foreseeable future. We delineate the challenges ahead for their increased effectiveness in this application domain. We note as a special challenge, the development of sensors that take over many of the functions the GRS designer struggles with today. We believe our contribution, in this first survey on sensors for GRSs, can give valuable insights into this important research and development topic, and encourage advanced research directions and new approaches.

Postman tour on a graph with precedence relation on arcs

Since the introduction of the Chinese Postman Problem (CPP), many variations on the same theme have been developed. In this paper we examine still another variation. The arcs of the graph are partitioned and a precedence relation defined, specifying the order in which the elements of the partition have to be traversed. We first examine the conditions for a feasible solution to the problem. Next, we specify the graph properties of the precedence partition that insure a polynomial complexity solution of O(N5), where N is the number of nodes in the original graph. When the precedence relation on sets of arcs is general, we prove that the problem of finding the minimum length of feasible postman tour is NP-complete.

Cluster labeling and parameter estimation for the automated setup of a hand-gesture recognition system

In this work, we address the issue of reconfigurability of a hand-gesture recognition system. The calibration or setup of the operational parameters of such a system is a time-consuming effort, usually performed by trial and error, and often causing system performance to suffer because of designer impatience. In this work, we suggest a methodology using a neighborhood-search algorithm for tuning system parameters. Thus, the design of hand-gesture recognition systems is transformed into an optimization problem. To test the methodology, we address the difficult problem of simultaneous calibration of the parameters of the image processing/fuzzy C-means (FCM) components of a hand-gesture recognition system. In addition, we proffer a method for supervising the FCM algorithm using linear programming and heuristic labeling. Resulting solutions exhibited fast convergence (in the order of ten iterations) to reach recognition accuracies within several percent of the optimal. Comparative performance testing using three gesture databases (BGU, American Sign Language and Gripsee), and a real-time implementation (Tele-Gest) are reported on.

Polygonal entropy: A convexity measure

Abstract The notion of polygonal entropy is introduced which captures the degree of nonconvexity of a polygon through a measure of polygonal visibility. We show that the entropy of a simple polygon is maximal if and only if it is convex. When ‘normalized’ the entropy measure is a number between e (a small positive nonzero number) and 1 providing a convenient index which represents the degree of irregularity (nonconvexity) of a simple polygon. A similar but less computationally burdensome measure is also proffered. Suggested uses of the measure are: the evaluation of computational complexity of vision algorithms, pattern recognition and architectural space planning.

Low and high-level visual feature-based apple detection from multi-modal images

Automated harvesting requires accurate detection and recognition of the fruit within a tree canopy in real-time in uncontrolled environments. However, occlusion, variable illumination, variable appearance and texture make this task a complex challenge. Our research discusses the development of a machine vision system, capable of recognizing occluded green apples within a tree canopy. This involves the detection of “green” apples within scenes of “green leaves”, shadow patterns, branches and other objects found in natural tree canopies. The system uses both thermal infra-red and color image modalities in order to achieve improved performance. Maximization of mutual information is used to find the optimal registration parameters between images from the two modalities. We use two approaches for apple detection based on low and high-level visual features. High-level features are global attributes captured by image processing operations, while low-level features are strong responses to primitive parts-based filters (such as Haar wavelets). These features are then applied separately to color and thermal infra-red images to detect apples from the background. These two approaches are compared and it is shown that the low-level feature-based approach is superior (74% recognition accuracy) over the high-level visual feature approach (53.16% recognition accuracy). Finally, a voting scheme is used to improve the detection results, which drops the false alarms with little effect on the recognition rate. The resulting classifiers acting independently can partially recognize the on-tree apples, however, when combined the recognition accuracy is increased.