Ahmad Pahlavan Tafti

Recent advances in 3D SEM surface reconstruction

The scanning electron microscope (SEM), as one of the most commonly used instruments in biology and material sciences, employs electrons instead of light to determine the surface properties of specimens. However, the SEM micrographs still remain 2D images. To effectively measure and visualize the surface attributes, we need to restore the 3D shape model from the SEM images. 3D surface reconstruction is a longstanding topic in microscopy vision as it offers quantitative and visual information for a variety of applications consisting medicine, pharmacology, chemistry, and mechanics. In this paper, we attempt to explain the expanding body of the work in this area, including a discussion of recent techniques and algorithms. With the present work, we also enhance the reliability, accuracy, and speed of 3D SEM surface reconstruction by designing and developing an optimized multi-view framework. We then consider several real-world experiments as well as synthetic data to examine the qualitative and quantitative attributes of our proposed framework. Furthermore, we present a taxonomy of 3D SEM surface reconstruction approaches and address several challenging issues as part of our future work.

3DSEM++: Adaptive and intelligent 3D SEM surface reconstruction.

Structural analysis of microscopic objects is a longstanding topic in several scientific disciplines, such as biological, mechanical, and materials sciences. The scanning electron microscope (SEM), as a promising imaging equipment has been around for decades to determine the surface properties (e.g., compositions or geometries) of specimens by achieving increased magnification, contrast, and resolution greater than one nanometer. Whereas SEM micrographs still remain two-dimensional (2D), many research and educational questions truly require knowledge and facts about their three-dimensional (3D) structures. 3D surface reconstruction from SEM images leads to remarkable understanding of microscopic surfaces, allowing informative and qualitative visualization of the samples being investigated. In this contribution, we integrate several computational technologies including machine learning, contrario methodology, and epipolar geometry to design and develop a novel and efficient method called 3DSEM++ for multi-view 3D SEM surface reconstruction in an adaptive and intelligent fashion. The experiments which have been performed on real and synthetic data assert the approach is able to reach a significant precision to both SEM extrinsic calibration and its 3D surface modeling.

SparkText: Biomedical Text Mining on Big Data Framework

Background Many new biomedical research articles are published every day, accumulating rich information, such as genetic variants, genes, diseases, and treatments. Rapid yet accurate text mining on large-scale scientific literature can discover novel knowledge to better understand human diseases and to improve the quality of disease diagnosis, prevention, and treatment. Results In this study, we designed and developed an efficient text mining framework called SparkText on a Big Data infrastructure, which is composed of Apache Spark data streaming and machine learning methods, combined with a Cassandra NoSQL database. To demonstrate its performance for classifying cancer types, we extracted information (e.g., breast, prostate, and lung cancers) from tens of thousands of articles downloaded from PubMed, and then employed Naïve Bayes, Support Vector Machine (SVM), and Logistic Regression to build prediction models to mine the articles. The accuracy of predicting a cancer type by SVM using the 29,437 full-text articles was 93.81%. While competing text-mining tools took more than 11 hours, SparkText mined the dataset in approximately 6 minutes. Conclusions This study demonstrates the potential for mining large-scale scientific articles on a Big Data infrastructure, with real-time update from new articles published daily. SparkText can be extended to other areas of biomedical research.

3DSEM: A 3D microscopy dataset

The Scanning Electron Microscope (SEM) as a 2D imaging instrument has been widely used in many scientific disciplines including biological, mechanical, and materials sciences to determine the surface attributes of microscopic objects. However the SEM micrographs still remain 2D images. To effectively measure and visualize the surface properties, we need to truly restore the 3D shape model from 2D SEM images. Having 3D surfaces would provide anatomic shape of micro-samples which allows for quantitative measurements and informative visualization of the specimens being investigated. The 3DSEM is a dataset for 3D microscopy vision which is freely available at [1] for any academic, educational, and research purposes. The dataset includes both 2D images and 3D reconstructed surfaces of several real microscopic samples.

A comparative study on the application of SIFT, SURF, BRIEF and ORB for 3D surface reconstruction of electron microscopy images

Image feature detector and descriptor algorithms have made a big advance in almost every area of computer vision applications including object localisation, object tracking, mobile robot mapping, watermarking, panorama stitching and 3D surface reconstruction by assisting the detection and description of feature points in a set of given images. In this paper, we evaluate the performance of four robust feature detection algorithms namely SIFT, SURF, BRIEF and ORB on multi-view 3D surface reconstruction of microscopic samples obtained by a scanning electron microscope (SEM), a widely used equipment in biological and materials sciences for determining the surface attributes of micro objects. To this end, we first develop an optimised multi-view framework for SEM extrinsic calibration and its 3D surface reconstruction. We design a Differential Evolutionary-based algorithm to solve the problem in a global optimisation platform. Several qualitative and quantitative comparisons such as reliability on SEM extrinsic calibration and validity on 3D visualisation performed on real microscopic objects as well as a synthetic model. The present evaluation is expected to provide better insights and consideration to determine which algorithm is well deserved for multi-view 3D SEM surface reconstruction.

SeLibCV: A Service Library for Computer Vision Researchers

Image feature detectors and descriptors have made a big advance in several computer vision applications including object recognition, image registration, remote sensing, panorama stitching, and 3D surface reconstruction. Most of these fundamental algorithms are complicated in code, and their implementations are available for only a few platforms. This operational restriction causes various difficulties to utilize them, and even more, it makes different challenges to establish novel experiments and develop new research ideas. SeLibCV is a Software as a Service (SaaS) library for computer vision researchers worldwide that facilitates Rapid Application Development (RAD), and provides application-to-application interaction by tiny services accessible through the Internet. Its functionality covers a wide range of computer vision algorithms including image processing, features extraction, motion detection, visualization, and 3D surface reconstruction. The present paper focuses on the SeLibCV’s routines specializing in local features detection, extraction, and matching algorithms which offer reusable and platform independent components, leading to reproducible research for computer vision scientists. SeLibCV is freely available at http://selibcv.org for any academic, educational, and research purposes.

Machine Learning-as-a-Service and Its Application to Medical Informatics

Machine learning as an advanced computational technology has been around for several years in discovering patterns from diverse biomedical data sources and providing excellent capabilities ranging from gene annotation to predictive phenotyping. However, machine learning strategies remain underused in small and medium-scale biomedical research labs where they have been collaboratively providing a reasonable amount of scientific knowledge. While most machine learning algorithms are complicated in code, theses labs and individual researchers could accomplish iterative data analysis using different machine learning techniques if they had access to highly available machine learning components and powerful computational infrastructures. In this contribution, we provide a comparison of several state-of-the-art Machine Learning-as-a-Service platforms along with their capabilities in medical informatics. In addition, we performed several analyses to examine the qualitative and quantitative attributes of two Machine Learning-as-a-Service environments namely “BigML” and “Algorithmia”.

Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation

Scanning Electron Microscope (SEM) as one of the major research and industrial equipment for imaging of micro-scale samples and surfaces has gained extensive attention from its emerge. However, the acquired micrographs still remain two-dimensional (2D). In the current work a novel and highly accurate approach is proposed to recover the hidden third-dimension by use of multi-view image acquisition of the microscopic samples combined with pre/post-processing steps including sparse feature-based stereo rectification, nonlocal-based optical flow estimation for dense matching and finally depth estimation. Employing the proposed approach, three-dimensional (3D) reconstructions of highly complex microscopic samples were achieved to facilitate the interpretation of topology and geometry of surface/shape attributes of the samples. As a byproduct of the proposed approach, high-definition 3D printed models of the samples can be generated as a tangible means of physical understanding. Extensive comparisons with the state-of-the-art reveal the strength and superiority of the proposed method in uncovering the details of the highly complex microscopic samples.

SD-SEM: sparse-dense correspondence for 3D reconstruction of microscopic samples

Scanning electron microscopy (SEM) imaging has been a principal component of many studies in biomedical, mechanical, and materials sciences since its emergence. Despite the high resolution of captured images, they remain two-dimensional (2D). In this work, a novel framework using sparse-dense correspondence is introduced and investigated for 3D reconstruction of stereo SEM images. SEM micrographs from microscopic samples are captured by tilting the specimen stage by a known angle. The pair of SEM micrographs is then rectified using sparse scale invariant feature transform (SIFT) features/descriptors and a contrario RANSAC for matching outlier removal to ensure a gross horizontal displacement between corresponding points. This is followed by dense correspondence estimation using dense SIFT descriptors and employing a factor graph representation of the energy minimization functional and loopy belief propagation (LBP) as means of optimization. Given the pixel-by-pixel correspondence and the tilt angle of the specimen stage during the acquisition of micrographs, depth can be recovered. Extensive tests reveal the strength of the proposed method for high-quality reconstruction of microscopic samples.

OCR as a Service: An Experimental Evaluation of Google Docs OCR, Tesseract, ABBYY FineReader, and Transym

Optical character recognition (OCR) as a classic machine learning challenge has been a longstanding topic in a variety of applications in healthcare, education, insurance, and legal industries to convert different types of electronic documents, such as scanned documents, digital images, and PDF files into fully editable and searchable text data. The rapid generation of digital images on a daily basis prioritizes OCR as an imperative and foundational tool for data analysis. With the help of OCR systems, we have been able to save a reasonable amount of effort in creating, processing, and saving electronic documents, adapting them to different purposes. A set of different OCR platforms are now available which, aside from lending theoretical contributions to other practical fields, have demonstrated successful applications in real-world problems. In this work, several qualitative and quantitative experimental evaluations have been performed using four well-know OCR services, including Google Docs OCR, Tesseract, ABBYY FineReader, and Transym. We analyze the accuracy and reliability of the OCR packages employing a dataset including 1227 images from 15 different categories. Furthermore, we review the state-of-the-art OCR applications in healtcare informatics. The present evaluation is expected to advance OCR research, providing new insights and consideration to the research area, and assist researchers to determine which service is ideal for optical character recognition in an accurate and efficient manner.