Myriam Munezero

Introducing continuous experimentation in large software-intensive product and service organisations

Abstract Software development in highly dynamic environments imposes high risks to development organizations. One such risk is that the developed software may be of only little or no value to customers, wasting the invested development efforts. Continuous experimentation, as an experiment-driven development approach, may reduce such development risks by iteratively testing product and service assumptions that are critical to the success of the software. Although several experiment-driven development approaches are available, there is little guidance available on how to introduce continuous experimentation into an organization. This article presents a multiple-case study that aims at better understanding the process of introducing continuous experimentation into an organization with an already established development process. The results from the study show that companies are open to adopting such an approach and learning throughout the introduction process. Several benefits were obtained, such as reduced development efforts, deeper customer insights, and better support for development decisions. Challenges included complex stakeholder structures, difficulties in defining success criteria, and building experimentation skills. Our findings indicate that organizational factors may limit the benefits of experimentation. Moreover, introducing continuous experimentation requires fundamental changes in how companies operate, and a systematic introduction process can increase the chances of a successful start.

Transitioning Towards Continuous Experimentation in a Large Software Product and Service Development Organisation – A Case Study

Context: Companies need capabilities to evaluate the customer value of software-intensive products and services. One way of systematically acquiring data on customer value is running continuous experiments as part of the overall development process. Objective: This paper investigates the first steps of transitioning towards continuous experimentation in a large company, including the challenges faced. Method: We conduct a single-case study using participant observation, interviews, and qualitative analysis of the collected data. Results: Results show that continuous experimentation was well received by the practitioners and practising experimentation helped them to enhance understanding of their product value and user needs. Although the complexities of a large multi-stakeholder business-to-business (B2B) environment presented several challenges such as inaccessible users, it was possible to address impediments and integrate an experiment in an ongoing development project. Conclusion: Developing the capability for continuous experimentation in large organisations is a learning process which can be supported by a systematic introduction approach with the guidance of experts. We gained experience by introducing the approach on a small scale in a large organisation, and one of the major steps for future work is to understand how this can be scaled up to the whole development organisation.

Notifying and involving users in experimentation: ethical perceptions of software practitioners

Background: Experiment-driven development with the help of real usage data helps to build software products and services that are of high value to their users. As more software companies use experimentation in their development practises, ethical concerns are increasingly important. Objective: There is a need for understanding the ethical issues companies must take into account when practising experimentation as a development strategy. This paper examines how software development practitioners experience the need for notifying users when involving them in experimentation. Method: We conducted a survey within four software companies, inviting employees in different functional roles to indicate their attitudes and perceptions through a number of statements. Results: Employees working in different roles have different viewpoints on ethical issues. While managers are more conscious about company-customer relationships, UX designers appear more familiar with involving users. Developers think that details of experiments can be withheld from users if the results depend on it. Conclusion: Barriers to successfully conducting experiment-driven development are different for different roles. Clear and specific guidelines are needed for ethical aspects of experimentation.

Finding and expressing news from structured data

In the age of increasing floods of information, finding the news signals from the noise has become increasingly resource and time intensive for journalists. Generally, news media companies have the important role of filtering and explaining this flood of information to the public. However, with the increase in availability of data sources, human journalists are unable to catch and report on all the news. This limitation, coupled with the need for media companies to continuously provide value to news readers, calls for automated solutions, such as automatically generating news from data. In order to support the journalists and media companies, and to provide value to audiences, this work proposes approaches for automatically finding news or newsworthy events from structured data using statistical analysis. Utilizing a real natural language news generation system as a case study, we demonstrate the feasibility and benefits of automating those processes. In particular, the paper reveals that through automation of the news generation process, including the generation of textual news articles, a large amount of news can be expressed in digestible formats to audiences, at varying local levels, and in multiple languages. In addition, automation allows the audience to tailor or personalize the news they want to read. Results of this work thus support and broaden the news offering and experiences for both media companies and the public.

Supporting Management of Hybrid OSS Communities - A Stakeholder Analysis Approach

In Hybrid Open Source Software projects, independent and commercially oriented stakeholders collaborate using freely accessible tools and development processes. Here, contributors can enter and leave the community flexibly, which poses a challenge for community managers in ensuring the sustainability of the community. This short paper reports initial results from an industrial case study of the “Qt” Open Source Software project. We present a visual stakeholder analysis approach, building on data from the three systems that provide for the Qt project’s complete software development workflow. This overview, augmented with information about the stakeholders’ organizational affiliations, proved to help the project’s community manager in finding potential for encouraging contributors and to identify issues that can potentially be detrimental for the community.

Towards Data-Driven Generation of Visualizations for Automatically Generated News Articles

A feature news story is often accompanied by illustrations and visuals. These visualizations can be, e.g., timelines, line charts, pie charts, or images. In this article, we present a largely data-driven and domain-independent approach for generating visualizations to accompany automatically generated news articles. We demonstrate the feasibility of our approach by applying it to statistical data on crime in Finland. The practical implementation demonstrates how the automatically generated visualizations provide additional information and interactivity to the news articles. We further illustrate how the approach presented is easily transferable to different domains with structured numerical datasets.

An exploratory analysis of a hybrid OSS company's forum in search of sales leads

Background: Online forums are instruments through which information or problems are shared and discussed, including expressions of interests and intentions. Objective: In this paper, we present ongoing work aimed at analyzing the content of forum posts of a hybrid open source company that offers both free and commercial licenses, in order to help its community manager gain improved understanding of the forum discussions and sentiments and automatically discover new opportunities such as sales leads, i.e., people who are interested in buying a license. These leads can then be forwarded to the sales team for follow-up and can result in them potentially making a sale, thus increasing company revenue. Method: For the analysis of the forums, an untapped channel for sales leads by the company, text analysis techniques are utilized to identify potential sales leads and the discussion topics and sentiments in those leads. Results: Results of our preliminary work make a positive contribution in lessening the community managers work in understanding the sentiment and discussion topics in the hybrid open source forum community, as well as make it easier and faster to identify potential future customers. Conclusion: We believe that the results will positively contribute to improving the sales of licenses for the hybrid open source company.