Kelly Blincoe

The promises and perils of mining GitHub

With over 10 million git repositories, GitHub is becoming one of the most important source of software artifacts on the Internet. Researchers are starting to mine the information stored in GitHubs event logs, trying to understand how its users employ the site to collaborate on software. However, so far there have been no studies describing the quality and properties of the data available from GitHub. We document the results of an empirical study aimed at understanding the characteristics of the repositories in GitHub and how users take advantage of GitHubs main features---namely commits, pull requests, and issues. Our results indicate that, while GitHub is a rich source of data on software development, mining GitHub for research purposes should take various potential perils into consideration. We show, for example, that the majority of the projects are personal and inactive; that GitHub is also being used for free storage and as a Web hosting service; and that almost 40% of all pull requests do not appear as merged, even though they were. We provide a set of recommendations for software engineering researchers on how to approach the data in GitHub.

An in-depth study of the promises and perils of mining GitHub

With over 10 million git repositories, GitHub is becoming one of the most important sources of software artifacts on the Internet. Researchers mine the information stored in GitHub’s event logs to understand how its users employ the site to collaborate on software, but so far there have been no studies describing the quality and properties of the available GitHub data. We document the results of an empirical study aimed at understanding the characteristics of the repositories and users in GitHub; we see how users take advantage of GitHub’s main features and how their activity is tracked on GitHub and related datasets to point out misalignment between the real and mined data. Our results indicate that while GitHub is a rich source of data on software development, mining GitHub for research purposes should take various potential perils into consideration. For example, we show that the majority of the projects are personal and inactive, and that almost 40 % of all pull requests do not appear as merged even though they were. Also, approximately half of GitHub’s registered users do not have public activity, while the activity of GitHub users in repositories is not always easy to pinpoint. We use our identified perils to see if they can pose validity threats; we review selected papers from the MSR 2014 Mining Challenge and see if there are potential impacts to consider. We provide a set of recommendations for software engineering researchers on how to approach the data in GitHub.

ProxiScientia: toward real-time visualization of task and developer dependencies in collaborating software development teams

This paper introduces ProxiScientia, a visualization tool that provides awareness support to developers, as they engage in collaborative software development activities. ProxiScientia leverages streams of fine-grained events that are generated by team members as they interact with software artifacts in their development environments. The main goal of the tool is to make each developer aware of coordination needs and opportunities as they arise, by depicting ego-centered views of the developers and tasks that most closely impact their work, and showing how they change in real time. In this paper, we illustrate the conceptualization of ProxiScientia and discuss its initial evaluation.

Understanding "watchers" on GitHub

Users on GitHub can watch repositories to receive notifications about project activity. This introduces a new type of passive project membership. In this paper, we investigate the behavior of watchers and their contribution to the projects they watch. We find that a subset of project watchers begin contributing to the project and those contributors account for a significant percentage of contributors on the project. As contributors, watchers are more confident and contribute over a longer period of time in a more varied way than other contributors. This is likely attributable to the knowledge gained through project notifications.

Do all task dependencies require coordination? the role of task properties in identifying critical coordination needs in software projects

Several methods exist to detect the coordination needs within software teams. Evidence exists that developers’ awareness about coordination needs improves work performance. Distinguishing with certainty between critical and trivial coordination needs and identifying and prioritizing which specific tasks a pair of developers should coordinate about remains an open problem. We investigate what work dependencies should be considered when establishing coordination needs within a development team. We use our conceptualization of work dependencies named Proximity and leverage machine learning techniques to analyze what additional task properties are indicative of coordination needs. In a case study of the Mylyn project, we were able to identify from all potential coordination requirements a subset of 17% that are most critical. We define critical coordination requirements as those that can cause the most disruption to task duration when left unmanaged. These results imply that coordination awareness tools could be enhanced to make developers aware of only the coordination needs that can bring about the highest performance benefit.

Learning global agile software engineering using same-site and cross-site teams

We describe an experience in teaching global software engineering (GSE) using distributed Scrum augmented with industrial best practices. Our unique instructional technique had students work in both same-site and cross-site teams to contrast the two modes of working. The course was a collaboration between Aalto University, Finland and University of Victoria, Canada. Fifteen Canadian and eight Finnish students worked on a single large project, divided into four teams, working on interdependent user stories as negotiated with the industrial product owner located in Finland. Half way through the course, we changed the teams so each student worked in both a local and a distributed team. We studied student learning using a mixed-method approach including 14 post-course interviews, pre-course and Sprint questionnaires, observations, meeting recordings, and repository data from git and Flow dock, the primary communication tool. Our results show no significant differences between working in distributed vs. Non-distributed teams, suggesting that Scrum helps alleviate many GSE problems. Our post-course interviews and survey data allows us to explain this effect, we found that students over time learned to better self-select tasks with less inter-team dependencies, to communicate more, and to work better in teams.

Creating a model of the dynamics of socio-technical groups

Individuals participating in technologically mediated forms of organization often have difficulty recognizing when groups emerge, and how the groups they take part in evolve. This paper contributes an analytical framework that improves awareness of these virtual group dynamics through analysis of electronic trace data from tasks and interactions carried out by individuals in systems not explicitly designed for context adaptivity, user modeling or user personalization. We discuss two distinct cases to which we have applied our analytical framework. These two cases provide a useful contrast of two prevalent ways for analyzing social relations starting from electronic trace data: either artifact-mediated or direct person-to-person interactions. Our case study integrates electronic trace data analysis with analysis of other, triangulating data specific to each application. We show how our techniques fit in a general model of group informatics, which can serve to construct group context, and be leveraged by future tool development aimed at augmenting context adaptivity with group context and a social dimension. We describe our methods, data management strategies and technical architecture to support the analysis of individual user task context, increased awareness of group membership, and an integrated view of social, information and coordination contexts.

Continuous Clarification and Emergent Requirements Flows in Open-Commercial Software Ecosystems

Software engineering practice has shifted from the development of products in closed environments toward more open and collaborative efforts. Software development has become significantly interdependent with other systems (e.g. services, apps) and typically takes place within large ecosystems of networked communities of stakeholder organizations. Such software ecosystems promise increased innovation power and support for consumer-oriented software services at scale and are characterized by a certain openness of their information flows. While such openness supports project and reputation management, it also brings requirements engineering-related challenges within the ecosystem, such as managing dynamic, emergent contributions from the ecosystem stakeholders, as well as collecting their input while protecting their IP. In this paper, we report from a study of requirements communication and management practices within IBM®’s Collaborative Lifecycle Management® product development ecosystem. Our research used multiple methods for data collection, including interviews within several ecosystem actors, on-site participatory observation, and analysis of online project repositories. We chart and describe the flow of product requirements information through the ecosystem, how the open communication paradigm in software ecosystems provides opportunities for “just-in-time” RE—and which relies on emergent contributions from the ecosystem stakeholders—, as well as some of the challenges faced when traditional requirements engineering approaches are applied within such an ecosystem. More importantly, we discuss two tradeoffs brought about by the openness in software ecosystems: (1) allowing open, transparent communication while keeping intellectual property confidential within the ecosystem and (2) having the ability to act globally on a long-term strategy while empowering product teams to act locally to answer end users’ context-specific needs in a timely manner. A sufficient level of openness facilitates contributions of emergent stakeholders. The ability to include important emergent contributors early in requirements elicitation appears to be a crucial asset in software ecosystems.

A hybrid model for task completion effort estimation

Predicting time and effort of software task completion has been an active area of research for a long time. Previous studies have proposed predictive models based on either text data or metadata of software tasks to estimate either completion time or completion effort of software tasks, but there is a lack of focus in the literature on integrating all sets of attributes together to achieve better performing models. We first apply the previously proposed models on the datasets of two IBM commercial projects called RQM and RTC to find the best performing model in predicting task completion effort on each set of attributes. Then we propose an approach to create a hybrid model based on selected individual predictors to achieve more accurate and stable results in early prediction of task completion effort and to make sure the model is not bounded to some attributes and consequently is adoptable to a larger number of tasks. Categorizing task completion effort values into Low and High labels based on their measured median value, we show that our hybrid model provides 3-8% more accuracy in early prediction of task completion effort compared to the best individual predictors.

Implicit Coordination: A Case Study of the Rails OSS Project

Previous studies on coordination in OSS projects have studied explicit communication. Research has theorized on the existence of coordination without direct communication or implicit coordination in OSS projects, suggesting that it contributes to their success. However, due to the intangible nature of implicit coordination, no studies have confirmed these theories. We describe how implicit coordination can now be measured in modern collaborative development environments. Through a case study of a popular OSS GitHub-hosted project, we report on how and why features that support implicit coordination are used.