John D. Wason

Automated Multiprobe Microassembly Using Vision Feedback

This paper describes the algorithm development and experimental results of a vision-guided multiprobe microassembly system. The key focus is to develop the capabilities required for the construction of 3-D structures using only planar microfabricated parts. Instead of using grippers, multiple sharp-tipped probes are coordinated to manipulate parts by using vision feedback. This novel probe-based approach offers both stable part grasping and dexterous part manipulation. The light weight of the part and relatively slow motion means that only kinematics-based control is required. However, probe motions need to be carefully coordinated to ensure reliable and repeatable part grasping and manipulation. Machine vision with multiple cameras is used to guide the motion. No contact force sensor is used; instead, vision sensing of the probe bending is used for the grasp force control. By combining preplanned manipulation sequences and vision-based manipulation, repeatable spatial (in contrast with planar) manipulation and insertion of a submillimeter part have been demonstrated with an experimental testbed consisting of two actuated probes, a passive probe, an actuated die stage, and two cameras for vision feedback.

Robot Raconteur: A communication architecture and library for robotic and automation systems

Robot Raconteur is a new distributed communication architecture and library designed for robotic and automation systems with distributed resources, including data and program modules. The motivation for this architecture is based on the need to rapidly connect sensors and actuators distributed across a network together in a development environment, such as MATLAB, without time consuming development of data communication infrastructure. The architecture consists of interconnected nodes, communicating through message passing. Each node is typically a process running on a computer or embedded device, which may be a critical real-time, non-critical real-time, or event driven process. Robot Raconteur is organized as three hierarchical levels: channels that provide communication between nodes, message passing which routes messages between endpoints within the nodes, and an object-based client-service model that is built on top of message passing. The implementation of Robot Raconteur nodes so far consists of Microsoft C#, Microsoft C++, MATLAB, MATLAB/Simulink xPC Target, and the Arduino embedded processor. Implementation on four distributed control systems consisted of multiple sensors, actuators, and computation nodes are presented, including smart room (an instrumented room with distributed lighting control and sensor feedback), dual-arm robotic system, multi-probe microassembly station, and adaptive optical scanning microscope.

Vision guided multi-probe assembly of 3D microstructures

This paper describes the operator assisted automated assembly of a 3-legged spatial platform by using a vision guided multi-probe assembly process. This is the first step towards the ultimate goal of building a microscale active spatial platform. Two issues are highlighted in this paper: contact management and vision feedback. Using multiple probes for part grasping and manipulation has the advantage of robustness and versatility as compared to micro-grippers. However, the contacts between the probes and the part need to be carefully managed to ensure a grasp that is stable for part pick-up and yet manipulable to allow part motion in a controlled fashion. By using vision guidance, the probes can be coordinated to grasp the parts and lift them off the die securely and reliably. We show that the contacts act as point contacts with friction, so when a part is pressed against a stationary probe, the part rotates about the axis between the contacts, changing its orientation so it may be inserted into a slot in the substrate. We have demonstrated that the three legs can be assembled in a fully automated fashion via multiple-camera vision feedback. The platform is at present assembled via tele-operation. The assembled microstructure measures 450 µm×600 µm. We are now working on the full automation of the assembly onto a substrate populated with MEMS actuators.

Multi-Probe Micro-Assembly

This paper describes the algorithm development and experimental results of a multi-probe micro-assembly system. The experimental testbed consists of two actuated probes, an actuated die stage, and vision feedback. The kinematics relationships for the probes, die stage, and part manipulation are derived and used for calibration and kinematics-based planning and control. Particular attention has been focused on the effect of adhesion forces in probe-part and part-stage contacts in order to achieve grasp stability and robust part manipulation. By combining pre-planned manipulation sequences and vision based manipulation, repeatable spatial (in contrast to planar) manipulation and insertion of a sub-millimeter part has been demonstrated. The insertion process only requires the operator to identify two features to initialize the calibration, and the remaining tasks involving part pick-up, manipulation, and insertion are all performed autonomously.

Predictive-focus illumination for reducing photodamage in live-cell microscopy.

Due to photobleaching and phototoxicity induced by high‐intensity excitation light, the number of fluorescence images that can be obtained in live cells is always limited. This limitation becomes particularly prominent in multidimensional recordings when multiple Z‐planes are captured at every time point. Here we present a simple technique, termed predictive‐focus illumination (PFI), which helps to minimize cells’ exposure to light by decreasing the number of Z‐planes that need to be captured in live‐cell 3D time‐lapse recordings. PFI utilizes computer tracking to predict positions of objects of interest (OOIs) and restricts image acquisition to small dynamic Z‐regions centred on each OOI. Importantly, PFI does not require hardware modifications and it can be easily implemented on standard wide‐field and spinning‐disc confocal microscopes.

Dextrous manipulation of a micropart with multiple compliant probes through visual force feedback

In our recent work, we have demonstrated effectiveness of the concept of multi-probe microassembly for manipulating and inserting microscale, sub-millimeter, parts to create three-dimensional microstructures. However, the approach has been based on trial-and-error manual teaching of grasp points to ensure a stable grasp during motion. As a result, the part orientation is restricted (nearly aligned with the world reference frame and lying flat) to ensure successful grasping and manipulation. In this paper, we developed a kinematics based hybrid motion and force control based only on vision feedback. We first conduct a systematic analysis of the bending of the probes while they are in contact with the part, to estimate the grasp force based on the vision feedback of the probe configuration. A Jacobian based controller is then used for position manipulation while maintaining the desired squeeze force. Experimental results with two probes and two camera are included to demonstrate the effectiveness of the controller to move the part to specified position and orientation while maintaining sufficient squeeze force to prevent part slippage.

Experimental verification of formation control with distributed cameras

Formation control experiments are performed using two robots, each equipped with a camera. When both robots are fully informed of the reference velocity, a decentralized feedback control, which achieves the desired relative position and the reference velocity, is experimentally verified. Another adaptive design is implemented in the situation where only one robot has the full reference velocity information and the other robot estimates the reference velocity. The performances of the experiments are analyzed and compared with different feedback gains and adaptation gains. Typical experimental results show that the difference between individual camera measurements leads to steady state error in practice. As a preliminary investigation, this fact is analyzed theoretically with the measurement difference modeled as constant disturbance.

Robot Raconteur ® version 0.8: An updated communication system for robotics, automation, building control, and the Internet of Things

Robot Raconteur® (RR) is a powerful communication system for robotics, automation, building control, and the “Internet of Things”. It has been used extensively in research at several universities and has reached Version 0.8, the first “beta” version ready for commercial use. A paper describing an early experimental version was presented at CASE in 2011 [1]. This paper presents the new programmatic, transport, and security features available in the new library. These new features include a new C++ based core library, an “Augmented Object Oriented” data model, transport security using TLS, WebSocket support, HTML5/JavaScript Browser support, and ASP.NET web server support. The new C++ library provides bindings for Python, MATLAB, C#, and Java. The library can run on all major operating systems and device architectures. Several example systems are presented.

Three dimensional kinematics of the thoracolumbar spine as quantified by a novel in vivo/in vitro active/passive robotic simulator

Low back pain is one of the leading causes of missed work. Surgical intervention is an option to those who have failed conservative therapy. The current gold standard of surgical procedures is bony fusion. Long term follow up of fusion patients has identified adjacent level disease as a complication. New motion preserving technologies aim to eliminate harmful effects on adjacent levels but have not been confirmed with long term clinical trials. In vitro testing provides repeatability for relative comparison between treatments, but the clinical relevance remains questionable without in vivo loads and motions. We have developed a novel in vivo/in vitro method using active motion to characterize the motion of the thoracolumbar spine using a robotic simulator. Preliminary results indicate a reduction in motion at the operative and super-adjacent level while motion is increased at all other levels. This may be compensation for the reduced motions at the operative and adjacent level.

Model-based control of a high-temperature crystal growth process

This paper describes a modeling and control approach for the thermal aspects of a high-temperature semiconductor crystal growth process. From a thermal perspective, each crystal growth cycle is composed of three distinct phases, heat up, growth, and cool down, each with specific control challenges and objectives. This paper focuses on the heat up and growth phases. A simulation model is first developed based on the induction furnace geometry and known material properties. This model is calibrated using the experimental process data by minimizing the weighted error between the predicted and actual temperature measurements. The two critical temperatures for the process are the temperature of the source material and the temperature of the crystal seed. For the heat up phase, the input profile is generated to rapidly ramp up the source and crystal temperature while avoiding damaging temperature spikes. In the crystal growth phase, the objective is to maintain the source temperature above sublimation while keeping the crystal temperature sufficiently low to allow condensation. These temperatures cannot be directly measured. Instead, an observer-based controller achieves the temperature control objective. Simulation results with FEM-in-the-loop validation are presented.