Tobias Tiemerding

Automated Robotic Manipulation of Individual Colloidal Particles Using Vision-Based Control

Automated manipulation of micro- and nanosized objects using robotic setups constitutes a major challenge due to the force-scaling laws and the limited control possibilities on that scale. This paper presents a new developed approach for automated manipulation of individual colloidal particles using a dedicated dual-probe setup inside a scanning electron microscope. Based on tailored probe geometries, the setup allows for reliable pick-up and release sequences of individual particles. Applying image processing of the visual feedback provided by the microscope enables for direct and fast control of the complex manipulation routines and thus allows for fully automated alignment sequences. Experimental results reveal a high repeatability of the process with hitherto unrivaled precision. The advantages and limits of this technique are highlighted with respect to further application scenarios.

Automated Mechanical Characterization of 2-D Materials using SEM based Visual Servoing

This paper presents an automated handling approach of two-dimensional nanomaterials using a robotic setup inside a high-resolution scanning electron microscope. Applying image processing of the visual feedback provided by the electron microscope, a fully automated sequence is developed to align a robotic driven force sensor with sub micrometer accuracy and to conduct nanoindentation measurements on a periodically perforated substrate. As an example, this automated sequence is utilized to examine the mechanical properties of a few-layer graphene membrane. The results of the mechanical characterization are compared to Raman spectroscopy data. The paper discusses the advantages and restrictions of this technique and responds to further application scenarios.

Robotic dual probe setup for reliable pick and place processing on the nanoscale using haptic devices

This paper presents reproducible pick and place handling of particles within the sub micron range. The handling strategy is based on a robotic dual probe setup that is integrated into a high resolution scanning electron microscope. By purposeful utilization of the predominant adhesive forces on the nanoscale, this setup facilitates the assembly of overall complex arrangements of different nanoparticles using haptic devices. The paper discusses control issues of the setup as well as the advantages and restrictions of the proposed technique.

Integrating robotic software frameworks for convenient software component exchange in micro- and nanoscale applications

Software development is a little discussed but essential part of micro- and nanoscale robotic systems. Current robotic systems are often controlled by a monolithic software that is either provided by the hardware manufacturer or specially tailored software for a particular application. One of the major challenges is to provide the research community with means that enable easy integration of software produced by different partners. The provided solution should not be bound to any particular programming language or scheme. This paper describes the currently used software approaches in control of micro- and nanorobotic systems. Progress towards convenient software integration is showcased by integrating two robotic software frameworks OFFIS automation framework and ROS. The first is an example of micro- and nanoscale oriented robotic software frameworks and the latter is designed for the needs of conventional robotic applications. The proposed integration approach is validated by an exemplary use case of a pick-and-place operation.

Automated mechanical characterization of 2D materials using SEM based visual servoing

This paper presents an automated handling approach of two-dimensional nanomaterials using a robotic setup inside a high-resolution scanning electron microscope. Applying image processing of the visual feedback provided by the electron microscope, a fully automated sequence is developed to align a robotic driven force sensor with sub micrometer accuracy and to conduct nanoindentation measurements on a periodically perforated substrate. As an example, this automated sequence is utilized to examine the mechanical properties of a few-layer graphene membrane. The results of the mechanical characterization are compared to Raman spectroscopy data. The paper discusses the advantages and restrictions of this technique and responds to further application scenarios.

Closing the loop: High-speed visual servoing and control of a commercial nanostage inside the SEM

In micro- and nanorobotics, it is important to increase closed-loop performance to achieve high-throughput for industrial applications. By using dedicated line scans instead of scanning microscope image acquisition, bottlenecks such as limited update rate, long latency and unpredictable jitter can be overcome. Earlier experiments used the line-scan approach for visual servoing of a custom made mobile robot. In this paper, the line-scan approach is used to guide the closed loop positioning of a Physik Instrumente (PI) nanostage. Additionally to the linescan controller and the commercial PI-stage controller, an FPGA system that acts as additional position controller was developed. Several evaluation measurements show the performance of the implementation in terms of accuracy and performance for the nanorobotic stage.

Modularized SPM-controller based on an FPGA for combined AFM and SMM measurements

This paper presents the design and implementation of a hardware/software controller system for combined atomic force microscopy (AFM) and scanning microwave microscopy (SMM) measurements. The purpose of the system is to connect and control existing components (e. g. any AFM scanner and vector network analyzer (VNA)), thus enhancing their functionality. Therefore, the system is of highly modular design. The base board offers multiple slots with a predefined piggy-back interface for different I/O components like digital-to-analog converters (DAC). Instead of hardwiring the overall connections, an field programmable gate array (FPGA) is used. An evaluation shows the superiority of the system in comparison to the state-of-the-art, despite the early stage of development.

Integration of different hardware interfacing protocols into a robotic software framework

Research as well as industrial projects in the field of micro- and nanorobotics often incorporate complex experimental setups using different devices with varying and often complex interfaces. In order to provide an easier interface to these devices, this paper outlines the extension of a versatile software framework called OFFIS Automation Framework, for automation, image processing and controlling purposes in this field of application. Especially, features like the integration of a macro world robot operating system, an industrial level and real-time capable automation system and the support of a standard laboratory device interface are introduced in this work. Furthermore, applications are presented for each of these new features. Finally, the field of applications is summarized and a brief outlook is given on future developments of this software framework.

Advanced Methods for High-Speed Template Matching Targeting FPGAs

Template matching is an important image processing algorithm for object detection and tracking tasks especially because of its very high accuracy. Yet, one major disadvantage of this algorithm is its very high computational effort due to its high complexity which results in low update rates using conventional software-based systems. Additionally, these systems are often afflicted by latency and jitter. Pre-calculating small and robust templates allows for using FPGA-based template matching as an approach to solve these problems. Therefore, the paper outlines three successive software-based approaches to find these small and robust templates. The first approach contains the basic algorithm and is refined by the following two approaches using pre-processing. The resulting templates allow for using the programmable hardware of an FPGA to cache necessary image information and, more importantly, to derive the best matching of template and source image, which results in high update rates. This paper shows novel approaches to high-speed template matching on FPGAs. The validation of these approaches has shown that the resulting tracking quality and feasibility is highly dependent on the relative size of the template in regard to the object to track. The results show tracking uncertainties between one single pixel for low and hundreds of pixels for high resolution videos.

Automated robotic manipulation of individual sub-micro particles using a dual probe setup inside the scanning electron microscope

Micro- and nanosized objects aligned in specific spatial order are of great interest for applications in photonics and nanoelectronics. In particular, piezo-actuated robotic setups are promising tools to arrange and manipulate these objects individually. However, automated robotic processing on the sub-micron scale remains challenging due to the force scaling laws and the limited possibilities in terms of control. This paper presents the current progress on fully-automated pick-and-place routines of individual colloidal particles using a dedicated dual-probe setup inside a scanning electron microscope. Applying tailored probes in combination with image processing of the visual feedback provided by the microscope allow for complex automation sequences. The limits of the current technique are highlighted and the challenges for automated processing of progressively smaller particles are discussed.