Tadahiro Uehara

Mobile Application Testing: A Tutorial

To cope with frequent upgrades of mobile devices and technologies, engineers need a reusable and cost-effective environment for testing mobile applications and an elastic infrastructure to support large-scale test automation.

Testing as a Service (TaaS) on Clouds

Cloud computing leads an opportunity in offering testing as a service (TaaS) for SaaS, clouds, and cloud-based applications. This brings new business opportunities, challenges, and demands in innovative service models, testing techniques, QoS standards, and requirements. This paper provides a comprehensive tutorial on testing as a service in a cloud environment. It answers the common questions raised by engineers and managers, and provides clear conceptual discussions about testing as a service (TaaS), including its scope, objectives, motivations and values, distinct features, required techniques, as well as testing environments. It not only presents a classification of different types of testing services in TaaS, but also offers a clear comparative view and perspectives between conventional software testing service and cloud-based testing as a service. In addition, it examines underlying issues, challenges, and emergent needs.

Mobile Testing-as-a-Service (MTaaS) -- Infrastructures, Issues, Solutions and Needs

With the rapid advance of mobile computing technology and wireless networking, there is a significant increase of mobile subscriptions. This drives a strong demand for development and validation of mobile APPs and SaaS applications on mobile web. This paper is written to offer informative and insightful discussion about mobile testing-as-a-service (MTaaS), including its basic concepts, motivations, distinct features and requirements, test environments, and different approaches. Moreover, it presents a test process in MTaaS and three different approaches. Furthermore, the paper proposes one mobile test cloud infrastructure for mobile TaaS, and discusses the required mobile test frameworks and environments. Finally, the paper addresses existing issues, challenges, and emergent needs.

A cloud-based TaaS infrastructure with tools for SaaS validation, performance and scalability evaluation

With the fast advancements in cloud computing and software- as-a-service (SaaS), testing and evaluation of cloud-based software and SaaS applications became an important task for engineers. Since most existing tools are not developed to support cloud-based software testing and SaaS evaluation, there is a strong demand for a new cloud-based testing infrastructure and evaluation environment for SaaS applications. This paper proposes a testing-as-service (TaaS) infrastructure and reports a cloud-based TaaS environment with tools (known as CTaaS) developed to meet the needs in SaaS testing, performance and scalability evaluation. The paper presents TaaS concepts and CTaaS, including their infrastructure, design and implementation. In addition, the paper demonstrates the application results of our previously proposed graphic models and metrics for SaaS performance and scalability evaluation. Moreover, the paper reports one case study for a selected SaaS (OrangeHRM) using the developed TaaS environment.

Test data generation for web application using a UML class diagram with OCL constraints

In this paper, we report on our current work toward efficient and effective verification of web application’s basic design. We use a UML class diagram with Object Constraint Language (OCL) to describe the application behaviors and data constraints. Then we generate test data from the formally represented specifications. We make the observation that key web application behaviors can be captured through table size constraints as well as data constraints like foreign key constraints. Based on the observation, we translate the OCL specification into the equivalent constraints using table size expressions. We present a scheme to generate test data from the translated constraints using a Satisfiability Modulo Theories solver. We employ two techniques to reduce constraints. The first is string handling and the other is decomposition of table structures. We also report on an experimental result of test data generation. The result indicates a potential that our scheme works well for real applications in reasonable times.

Compatibility Testing Service for Mobile Applications

As more and more mobile applications are developed, mobile app testing and quality assurance have become very important. Due to the diversity of mobile devices and platforms, compatibility testing for mobile apps has been identified as one urgent and challenging issue. There are two major reasons contributing to this issue. They are: a) the large number of mobile devices with diverse features and platforms which are upgraded frequently, b) a higher cost and complexity in mobile app compatibility testing. This paper proposes one optimized compatibility testing strategy using a statistical approach to reduce test costs, and improve engineers operation efficiency. The paper provides a solution to generate an optimized compatibility test sequence for mobile apps using the K-Means statistical algorithm. A compatibility testing service has been proposed for mobile apps. Moreover, two case study results are reported to demonstrate its potential application and effectiveness.

SaaS Testing on Clouds - Issues, Challenges and Needs

With the fast advance of cloud computing, developing software-as-a-service (SaaS) is becoming a hot topic in cloud computing and engineering. Due to new features of SaaS applications, SaaS testing becomes an interesting hot topic for both industry and research communities. Although there are a number of published papers discussing cloud testing, there is a lack of research papers addressing new issues, challenges, and needs in SaaS testing. This paper provides a tutorial to discuss SaaS testing, including its concepts, focuses and objective, test process, test environments, and requirements. Moreover, the paper discusses the special SaaS features, and examines the related issues and challenges and needs in SaaS testing.

FSX: fine-grained incremental unit test generation for C/C++ programs

Automated unit test generation bears the promise of significantly reducing test cost and hence improving software quality. However, the maintenance cost of the automatically generated tests presents a significant barrier to adoption of this technology. To address this challenge, we propose a novel technique for automated and fine-grained incremental generation of unit tests through minimal augmentation of an existing test suite. The technique uses iterative, incremental refinement of test-drivers and symbolic execution, guided by a diagnostics engine. The diagnostics engine works off a novel precise and efficient byte-level dynamic dependence analysis built using Reduced Ordered Binary Decision Diagrams (ROBDDs). We present a tool FSX implementing this technique and evaluate it under two practical use-cases of incremental unit test generation, on five revisions of the open-source software iPerf, as well as on 3 large subjects, comprising more than 60 thousand lines of code, from in-house commercial network products. The evaluation shows that FSX can generate high-quality unit tests on large industrial software while minimizing the maintenance cost of the overall test-suite.

Automated testing for Java programs using JPF-based test case generation

Program testing requires a series of tasks such as preparing drivers and stubs, creating test cases, and executing unit tests. To reduce manual effort of performing such tasks for testing Java programs, we developed a tool that fully integrates and automates all of these processes, by using JPF with extensions as a symbolic execution engine for automatically generating unit test cases. In this paper, we present this tool and its application to real projects to evaluate its efficacy. The evaluation results demonstrate that the tool performs well in terms of the test time reduction compared with manual test as it eliminates the total amount of manual effort, while largely preserving a high coverage of greater than 90 % as our expected borderline.

Enhancing Symbolic Execution to Test the Compatibility of Re-engineered Industrial Software

After a legacy system is re-engineered, it is important to perform compatibility testing so as to identify the difference and reduce the introduced bugs. We can first apply symbolic execution to obtain an exhaustive set of test cases, then use them to check the compatibility of the old system and the new old. However, the path explosion problem of symbolic execution makes it difficult to work on realistic non-trivial applications. We show in this paper how to enhance symbolic execution, e.g. with extra constraints, path cutting, variable grouping, and test case selection, to successfully test the compatibility of an SMTP library (used in embedded systems) with around 20K lines of code. Our experience indicates that these enhancements are essential to apply symbolic execution on realistic industrial applications.