Hans Heerkens

Designing Requirements Engineering Research

Engineering sciences study different different topics than natural sciences, and utility is an essential factor in choosing engineering research problems. But despite these differences, research methods for the engineering sciences are no different than research methods for any other kind of science. At most there is a difference in emphasis. In the case of requirements engineering research-and more generally software engineering research-there is a confusion about the relative roles of research and about design and the methods appropriate for each of these activities. This paper analyzes these roles and provides a classification of research methods that can be used in any science-engineering or otherwise.

Assessing the importance of factors determining decision-making by actors involved in innovation processes

Innovations can be seen as chains of non-routine decisions. With each decision, the innovator has to assess how important the various decision attributes are. Because the decisions are non-routine, innovators cannot fall back on judgements of past importance. Most decision support methods elicit importance judgements but do not help innovators or other decision-makers with the mental processes leading to the judgment. The ‘importance assessment process’ can be divided into seven phases (such as (sub-)attribute processing and various forms of weighting). The phase ‘(sub)-attribute processing’ is the most important phase in terms of effort devoted to it, and the most obvious pitfalls that prevent valid importance assessments appear in this phase. This article describes some of these pitfalls. A few simple instruments may provide better-founded importance judgements that can be better communicated to other actors involved in innovation processes.

How to Write and Read a Scientific Evaluation Paper

Scientific evaluation papers investigate existing problem situations or validate proposed solutions with scientific means, such as by experiment or case study. There is a growing amount of literature about how to report about empirical research in software engineering, but there is still some confusion about the difference between a scientific evaluation paper and other kinds of research papers. This is related to lack of clarity about the relation between empirical research, engineering, and industrial practice. In this minitutorial we give a brief rundown on how to structure a scientific evaluation papers as a special kind of research paper, using experiment reports and case study reports as examples. We give checklists of items that a reader should be able to find in these papers, and sketch the dilemmas that writers and readers of these papers face when applying these checklists.

Technology transfer and catch-up; lessons from the commercial aircraft industry

This paper analyses the technology development and technology transfer strategies in the aircraft manufacturing industry for four industrially developing countries. It is concluded from four case studies that technology catch-up is extremely difficult due to aircraft technology characteristics. Based on this, several propositions are developed concerning technology transfer, catch-up and several aspects of learning.

Design Science, Engineering Science and Requirements Engineering

For several decades there has been a debate in the computing sciences about the relative roles of design and empirical research, and about the contribution of design and research methodology to the relevance of research results. In this minitutorial we review this debate and compare it with evidence about the relation between design and research in the history of science and technology. Our review shows that research and design are separate but concurrent activities, and that relevance of research results depends on problem setting rather than on rigorous methods. We argue that rigorous scientific methods separate design from research, and we give simple model for how to do this in a problem-driven way.

Assessing the structure of non-routine decision processes in airline operations control

Abstract Unfamiliar severe disruptions challenge Airline Operations Control professionals most, as their expertise is stretched to its limits. This study has elicited the structure of Airline Operations Control professionals’ decision process during unfamiliar disruptions by mapping three macrocognitive activities on the decision ladder: sensemaking, option evaluation and action planning. The relationship between this structure and decision quality was measured. A simulated task was staged, based on which think-aloud protocols were obtained. Results show that the general decision process structure resembles the structure of experts working under routine conditions, in terms of the general structure of the macrocognitive activities, and the rule-based approach used to identify options and actions. Surprisingly, high quality of decision outcomes was found to relate to the use of rule-based strategies. This implies that successful professionals are capable of dealing with unfamiliar problems by reframing them into familiar ones, rather than to engage in knowledge-based processing. Practitioner Summary: We examined the macrocognitive structure of Airline Operations Control professionals’ decision process during a simulated unfamiliar disruption in relation to decision quality. Results suggest that successful professionals are capable of dealing with unfamiliar problems by reframing them into familiar ones, rather than to engage in knowledge-based processing.

Importance assessment of decision attributes: A qualitative study comparing experts and laypersons

Purpose – This paper aims to investigate differences between experts and laypersons concerning the way they assess the importance of each of the various decision attributes (cost, risk, feasibility) taken into consideration during decision processes in an organizational setting. Design/methodology/approach – Nine project managers at building companies (experts), and 18 university students (laypersons) performed a think-aloud assignment aimed at assessing the importance of two attributes (safety and comfort) during an acquisition process of minibuses by a fictitious company. Findings – Experts use less effort for the assignment, but perform the same mental operations in comparison with laypersons. Experts work in less detail than laypersons. Both laypersons and experts disregard important aspects of normative decision theory; for instance, they appear not to check for completeness of their assessments. Practical implications – The authors propose that the main difference between experts and laypersons seems not to be the way in which they conduct importance assessments, but rather the fact that laypersons have to make “clean sheet” assessments, whereas experts can rely on their knowledge and experience to merely modify existing attribute weights. This relying on weights used in previous decisions may lead to sub-optimal choices in non-routine decision situations. Originality/value – In much decision research, the focus is on elicitation of weights and on factors that influence weights, not on the way weights come about. By explicitly addressing the thinking process before the weights are actually set, we gain insight in a stage of the decision process that is rarely addressed. Hence, we potentially create possibilities for improving the weighing process.

On a tool for analysing cognitive processes using exploratory think-aloud experiments

We develop a method that enables cognitive processes to be analysed quantitatively without having a conceptual framework in place at the start of the research. With current methods, the general structure of the cognitive process in question has to be known before quantitative analyses can be preformed. With the method, we studied the cognitive process of weighting the importance of attributes in a purchasing decision. Our approach consists of seven steps: determining the research method (in our case: the think-aloud method); designing an experiment; designing a data collection method; designing a tool for preliminary analysis; designing a tool for qualitative analysis; designing a tool for quantitative analysis; assessing external validity. In this contribution we also provide a checklist for putting our method into practice.

FOKKER, A CLASH OF CULTURE

What role do cultural play in the survival of companies that face existential problems? We try to provide some answers by looking at the decline of the Dutch aircraft manufacturer Fokker. In 1996 this company went bankrupt and among the causes of this event cultural factors rank high, at least at first sight. We show that differences in national culture could have played a role in one of the defining moments in Fokkers existence: the failed take-over by the German aerospace giant Deutsche Aerospace (DASA). But we also show that an over-ambitious management was responsible for the situation in which there were no other options to survive besides a take-over by DASA. An increasing gap between the ambitions of the management to be one of the worlds prime aerospace companies and the limited resources of what was essentially a second-league player put Fokker in such a bad financial condition that it surely had to become an unbearable burden for DASA. So culture played a role in Fokkers decline, but its influ...

On importance assessment and expertise in non-routine decisions: an exploratory study on the cognition of the weighting process of capital goods' attributes

How do actors involved in the acquisition of capital goods assess the importance of their attributes? What is the role of expertise? Numerous instruments exist for measuring the importance attached to attributes, but little is known about the importance assessment process that precedes these importance judgments. Expectations concerning the behaviour of actors facing non-routine importance assessment problems are tested, yielding some interesting results. Firstly, the behaviour of these actors is consistent with a newly developed phase model. Even with a non-routine problem, structuring the assessment problem takes less effort than the actual weighting. Surprisingly, weighting attributes in isolation gets much more emphasis than weighting them against each other, despite the latter being the essence of importance judgments. Despite the subjects being laymen, they showed high confidence in their work. Finally, predictions concerning the behaviour of experts are made, based on Van der Heijdens dimensions of expertise.