J S Busby

An assessment of post-project reviews

A detailed analysis of post-project reviews was conducted to assess whether such reviews should be conducted, and, if so, how they should be conducted. It was found that post-project reviews are effective in disseminating knowledge about good practices, correcting errors in individuals’ knowledge (especially their knowledge about other functions within the organization), and predicting how well alternative practices would have turned out. At the same time these reviews demonstrated several limitations. The diagnosis that took place in reviews tended to be shallow, remedies were planned only at a very superficial level, and explanations of events tended to be overly specific. Participants also made potentially misleading assumptions: for example, that unimportant problems implied unimportant causes. Nonetheless, post-project reviews are important learning mechanisms and their value seems to be underestimated by individuals who do not appreciate the need to disseminate insights throughout the organization.

Error and distributed cognition in design

Abstract In this study of the errors made in the design of complex process plant, 75 cases were analysed as failures of distributed cognition. Error arose in the interactions of different designers, of designers and design tools, designers and the formal organisation, and designers and the environment beyond the immediate organisation. The analysis attempted to describe the failures of each type of interaction, but there were some patterns of error that emerged in all types. For example, many errors arose from absent cues or inattention to cues. Many also arose from problems with norms (in the form of codes, standards and procedures)—including, ironically, norms that had been implemented in the light of earlier errors. One of the main practical implications of these findings is that designers could benefit from thinking about their tasks in terms of distributed cognition since this suggests several heuristics which, among practitioners, seem to be frequently neglected.

The neglect of feedback in engineering design organisations

Abstract Learning from experience, and attending to the consequences of ones work, are strong norms in design organisations. Yet, in this study of engineering design organisations, it was found that feedback to designers was often unreliable, delayed, negative and sometimes missing altogether. Moreover, designers failed to learn from the feedback that was available-developing plans that were at odds with past outcomes and repeating previous errors. These effects appeared to have a number of origins, among them the need to maintain a tractable environment by circumscribing transactions between designers and the world at large, and a basic over-estimation of the extent to which design activity is feed forward in nature.

The Problem with Design Reuse: An Investigation into Outcomes and Antecedents

An exploratory study of the difficulties of design reuse has been conducted in two design organisations. The study involved building simple causal networks to explain each of 171 cases of problematic reuse described in unstructured interviews with 50 designers and design managers. Most reuse problems concerned transfer that was inhibited in some way, although error and unexpected effort in the reuse process was also a significant outcome. Many of the causes lying behind these problems were local (such as unprecedented client requirements). But many were more general and structural. For instance, reuse was often precluded by differences in preference among designers, or among clients, and these differences appeared to arise from reinforcing feedback effects, which in turn arose from limitations of human inference-like insensitivity to sampling variability and confirmation bias. Reuse was also precluded by engineering problems, such as the combinatorial properties of constraints, which meant, in some instan...

Understanding the mismatch between the demands of risk assessment and practice of scientists — The case of Deca-BDE

This review describes how a mismatch between the knowledge produced by scientists and the evidence demanded by regulators has emerged, and how society has struggled to find definitive answers to questions of safety, for an important flame retardant chemical in current use - Deca-BDE. This has involved two key disciplines: analytical chemistry and toxicology. Within the chemistry, a lack of standardized methodologies among scientists has resulted in a persistent yet largely undeclared failure to replicate results within the discipline. Within the toxicology, the quest for innovative, curiosity-driven research by university scientists in preference to using validated standard protocols, designed to promote consistency within the risk assessment process, has prompted questions about the credibility and relevance of scientific findings. Yet scientific laboratories have compelling reasons to do things the way they do in the cause of producing new knowledge, pointing to a sustained gap between the aims and practices of research scientists and those of risk management. A more rigorous scientific process that treats different elements of input data as discrete pieces of evidence is needed to ensure that science rather than politics will always define chemical safety.

Failure to Mobilize in Reliability-Seeking Organizations: Two Cases from the UK Railway

There is a considerable line of research on organizations dealing with large scale, intrinsic hazards. We know a good deal, as a result, about both the causation of catastrophic failure and its avoidance. Past work has explained failure in terms of (for example) structural vulnerabilities and organizational degradation - and reliability in terms of collective mindfulness, rigorous enculturation and high levels of social redundancy. This paper presents a study, based on a qualitative analysis of two disastrous collisions on the UK railway, of organizations that are strongly reliability seeking yet ultimately experience catastrophic failure. It argues that these cases implicated an organizational incapacity to mobilize systemic reform. The possibility of the two failures had been well-known in the organizations before their occurrence, but this knowledge could not be converted into modification. A model is presented to explain how processes of systemic reform co-exist with a set of phenomena that tend to undermine them. It is these that need to be the principal focus of efforts at managing catastrophic failure risks. Copyright Blackwell Publishing Ltd 2006.

Risk Migration and Scientific Advance: The Case of Flame-Retardant Compounds

It is a common experience that attempts to mitigate a risk lead to new risks, and that risks formerly thought to be of one kind become another kind as technical knowledge evolves. This phenomenon of risk migration suggests that we should take processes over time, rather than specific risks or specific technologies, as a unit of analysis. Several of our existing models of the social management of risks-such as that of social risk amplification-are process models of a kind but are still oriented around the playing out of a particular event or issue. A case study of risk in a group of flame-retardant compounds was used as the basis of a grounded, exploratory analysis of migration processes, the phenomena that influence them, and their consequences. This illustrated how migration naturally occurs from risks that are understood, in which risk bearers have at least some agency, to risks that are not understood and not capable of being influenced by risk bearers. It illustrated how the simultaneous improvement in measuring technology, which detects potential toxins at increasingly small concentrations, combines with intuitive models that ignore concentration to produce conditions likely to generate anxiety. And it illustrated how pressure groups and commercial interests exploit this effect. It also showed how migration makes precautionary action problematic, and how more generally it tends to undermine a societys capacity to cope with risk.

An exploratory analysis of counterfeiting strategies: Towards counterfeit-resilient supply chains

Purpose – The purpose of this paper is to identify strategies employed by product counterfeiters in their exploitation of legitimate supply chains; to develop a theoretical understanding of counterfeiting and its impact on competitive resources; and, to propose counter-measures for increasing the resilience of supply chains to the counterfeiting threat. Design/methodology/approach – An inductive, qualitative analysis of secondary case data obtained from three sources. Findings – Initial searching and coding identified four sets of strategies: extraction strategies, for obtaining products or materials from the legitimate economy; production strategies, for manufacturing counterfeit goods; distribution strategies; and, infiltration strategies, for introducing counterfeits into the legitimate economy. Secondary, focused coding revealed that much of what the counterfeiting strategies set out to achieve involves the generation, suppression or exploitation of signals. A theoretical account of counterfeiting and...

The appropriate use of performance measurement in non‐production activity: The case of engineering design

Notes limitations to measuring the performance of design activity in particular, and non‐production activities in general. First, validity and reliability in specific measures are strongly negatively correlated, making it hard to achieve both. Second, outcome measures are jointly determined by engineering design and other activities to varying degrees, and this problem of shared outcomes is only partly reduced by measuring at higher levels of aggregation. Third, there is no definite stopping rule for engineering design activity, yet unambiguous outcome measures rely on the existence of such a rule. Fourth, outcomes attributable to engineering design can sometimes only be measured a long time after completion of the activity, making them ineffective for most managerial purposes. There are also considerable problems in properly accounting for environmental variables. However, the use of performance measures have some benefits, e.g. correcting wrong inferences among engineering managers. Results point to the appropriate use of performance measurement in engineering design for raising questions and detecting discrepancies in performance at aggregate levels. They suggest that using measurement is inappropriate for managerial control, for attributing results to engineers or the environment, and for concluding problem solving activities.

Practices in Design Concept Selection as Distributed Cognition

Abstract: A study was conducted of the practices that engineering designers had learned from experience to apply during the search for, and implementation of, new solution concepts. Each of 36 practices was analysed in terms of the goal it was directed to, and how it distributed cognition beyond the mental world of the designer. The results suggest that designers distribute cognition over their environments in a wide variety of ways which are not restricted, for instance, to using design tools. They also suggest that designers learn these practices in the context of specific experiences, probably by trial and error or social observation rather than means–end analysis. All but two of the practices involved distributed cognition of some kind, but there were no cases where this distribution involved ceding executive control to the environment.