James R. Vallino

Calibration-free augmented reality

Camera calibration and the acquisition of Euclidean 3D measurements have so far been considered necessary requirements for overlaying three-dimensional graphical objects with live video. We describe a new approach to video-based augmented reality that avoids both requirements: it does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environments objects. The only requirement is the ability to track across frames at least four fiducial points that are specified by the user during system initialization and whose world coordinates are unknown. Our approach is based on the following observation: given a set of four or more noncoplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by: tracking regions and color fiducial points at frame rate; and representing virtual objects in a non-Euclidean, affine frame of reference that allows their projection to be computed as a linear combination of the projection of the fiducial points. Experimental results on two augmented reality systems, one monitor-based and one head-mounted, demonstrate that the approach is readily implementable, imposes minimal computational and hardware requirements, and generates real-time and accurate video overlays even when the camera parameters vary dynamically.

Affine object representations for calibration-free augmented reality

We describe the design and implementation of a video based augmented reality system capable of overlaying three dimensional graphical objects on live video of dynamic environments. The key feature of the system is that it is completely uncalibrated: it does not use any metric information about the calibration parameters of the camera or the 3D locations and dimensions of the environments objects. The only requirement is the ability to track across frames at least four feature points that are specified by the user at system initialization time and whose world coordinates are unknown. Our approach is based on the following observation: given a set of four or more non coplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by: tracking lines and fiducial points at frame rate; and representing virtual objects in a non Euclidean, affine frame of reference that allows their projection to be computed as a linear combination of the projection of the fiducial points.

Haptics in augmented reality

An augmented reality system merges synthetic sensory information into a users perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most augmented reality systems the user views a real world augmented only with visual information and is not provided with a means to interact with the virtual objects. We describe an augmented reality system that, in addition to visual augmentation, merges synthetic haptic input into the users perception of the real environment. Our system uses a PHANToM/sup TM/ haptic interface device to generate the haptic sensory input in real-time. The system allows user interactions such as moving or lifting a virtual object, and demonstrates interactions between virtual and real objects. Methods to provide proper visual occlusion between real and virtual objects are also described.

Undergraduate software engineering

Addressing the needs of professional software development.

Design patterns- evolving from passive to active learning

Students in Rochester Institute of Technologys Software Engineeringprogram gain an appreciation for the importance of design in their second year when they work on a term-long team-based software project. Student comments oJim express an eagerness to be taught more about rhe design of larger soJiware systems. Our next course, Engineering of Software Subsystems, aims to achieve that outcome. This paper describes the evolution of this design course. The course was initially delivered as three one-hour lectures and one two-hour lab per week. Particularly in lectures, the students were not engaged to actively learn the material. The course has taken several evolutionary steps moving from its initial low level of active learning to where it now is mostly under the control of student teams participating as active learners. Dora from one offering of the course suggests improved course evaluation ratings and a noticeable increase in student appreciation for the textbook.

Thinking inside the box: a multi-disciplinary real-time and embedded systems course sequence

Small electronic products for the mass market are increasing in complexity with the incorporation of programmable components. The software in these devices has constraints that are markedly different from software designed for a general-purpose computer. Most computing curricula deal almost exclusively with developing software for that general-purpose class. Real-time and embedded systems have increased in complexity to the point that their development is no longer within the expertise of a single discipline. Developers now must be cognizant of software engineering design methodologies and underlying hardware constraints. RIT is addressing this by developing a three-course sequence of cross-disciplinary real-time and embedded systems courses. We are teaching these courses in a studio-lab environment teaming computer engineering and software engineering students. The courses will introduce students to programming both microcontrollers and more sophisticated targets, by using a commercial real-time operating system and development environment, modeling and performance engineering of these systems, and their interactions with physical systems

Cybersecurity Education: Bridging the Gap Between Hardware and Software Domains

With the continuous growth of cyberinfrastructure throughout modern society, the need for secure computing and communication is more important than ever before. As a result, there is also an increasing need for entry-level developers who are capable of designing and building practical solutions for systems with stringent security requirements. This calls for careful attention to algorithm choice and implementation method, as well as trade-offs between hardware and software implementations. This article describes motivation and efforts taken by three departments at Rochester Institute of Technology (Computer Engineering, Computer Science, and Software Engineering) that were focused on creating a multidisciplinary course that integrates the algorithmic, engineering, and practical aspects of security as exemplified by applied cryptography. In particular, the article presents the structure of this new course, topics covered, lab tools and results from the first two spring quarter offerings in 2011 and 2012.

Video Detection and Tracking of Tracer Particles in a Model Packed-Bed Reactor

A video detection and tracking system was devised to track and record the position versus time of tracer particles moving through a transparent medium. The system was applied to studying the flow-related transport phenomena at all points within packed-bed reactors since the previously proposed mathematical models for describing them have not been proven adequate thoughout the entire packed-bed. A modified TV camera was used to obtain a video presentation of the motion of a highly reflective particle in an otherwise transparent bed. A microcomputer was then used to process the video signal, using the vertical and horizontal sync pulses as timing signals. Output of the system was a punched paper tape with position versus time data for later analysis in a large computer.

Non-Euclidean Object Representations for Calibration-Free Video Overlay

We show that the overlay of 3D graphical objects onto live video taken by a mobile camera can be considerably simplified when the camera, the cameras environment and the graphical objects are represented in an affine frame of reference. The key feature of the approach is that it does not use any metric information about the calibration parameters of the camera, the position of the user interacting with the system, or the 3D locations and dimensions of the environments objects. The only requirement is the ability to track across frames at least four features (points or lines) that are specified by the user at system initialization time and whose world coordinates are unknown. Our approach is based on the following observation: Given a set of four or more non-coplanar 3D points, the projection of all points in the set can be computed as a linear combination of the projections of just four of the points. We exploit this observation by (1) tracking lines and feature points at frame rate, and (2) representing graphical objects in an affine frame of reference that allows the projection of virtual objects to be computed as a linear combination of the projection of the feature points.

What should students learn in their first (and often only) software engineering course

The questions that are proposed as the basis for academy panel sessions are important ones to ask. In the aggregate, they cover an enormous expanse of the software engineering landscape. The reality of undergraduate computing education is that the vast majority of students do not go through software engineering curricula where there is time to address the academy questions in some depth. Instead, they are in computer science or computer engineering programs, and receive their software engineering education in a single course. What students should really learn in this first, and often only, software engineering course is important because the majority of computing students will not see any other software engineering. The course designer will need to make judicious choices in selecting the material for this course because all of software engineering will not fit in just one course even if you try by using one of the classic software engineering tomes as the textbook. I do not know the right answer to the question I pose in the title of this position paper. I suspect that there is no one set of software engineering topics that should be included, but rather a range of topics to select from based on the purpose and perspective of the course. The answer to this question is important to everyone who has responsibility for providing the software engineering education for the next generation of computing students.