Frank Rossi

Assessing sensory panelist performance using repeatability and reproducibility measures

John Mandel of the National Bureau of Standards defined repeatability and reproducibility measures to assess within laboratory and between laboratory measurement variability in inter-laboratory (round robin) analytical studies. This paper will demonstrate that these same measures can be applied to the problem of assessing sensory panelist performance. Repeatability and reproducibility will be defined in the context of both analytical inter-laboratory studies and sensory panels. Formulas for the measures will be given. Examples using these measures with sensory panel data and their interpretation will be discussed. Since a typical sensory panel evaluates a much larger number of attributes compared with the number of analytical measurements in an analytical round robin test, some practical considerations as to how to use these measures efficiently will be proposed. Finally, a graphical method of displaying sensory panel data will be demonstrated as an alternative to using these measures, while keeping the repeatability and reproducibility concepts intact.

Blending response surface methodology and principal components analysis to match a target product

Highly trained sensory panels have long been used to evaluate food products on perhaps dozens of attributes. Principal components analysis is one of a number of multivariate data analysis techniques commonly used in analyzing sensory panel data. More recently, response surface designs have been used to direct the creation of product prototypes so that the effects of ingredient levels and/or processing conditions can be modeled. This paper will discuss how the two methodologies have been used together in projects where the goal is to identify ingredient levels and/or processing conditions that best match a target products sensory profile. Some unique problems arise when analyzing and interpreting the results of response surface models when the number of responses is quite large. This paper will explain how some of these problems have been addressed through the detailed discussion of the development of a cost reduced product. Six ingredients were systematically varied in a response surface design to create 48 prototypes. The prototypes and the target product were then measured on 33 sensory attributes. Design selection, data collection, response surface modeling, rotated principal components analysis and the use of both desirability and distance functions to identify ingredient level combinations that meet the product development objectives will be discussed in detail using the data analyses from this project. Recommendations for next steps in the product development process will also be given.

Measurement Systems Analysis (MSA)

Maria discovered large differences in performance of all production lines across the two production shifts, with generally consistent production within a shift. First shift production is consistently higher in viscosity. But, as Lisa pointed out, it is not clear whether the differences in viscosities between the shifts are real differences or they are due to the measurements being taken by different lab technicians.

Parametric modeling of time intensity data collected on product prototypes generated from a fractional factorial experiment to quantify sources of texture variability

Abstract Trained panels have been used to evaluate the sensory properties of food products for a number of years. Time–intensity sensory methodologies have been developed to identify and quantify the temporal sensory properties of foods and beverages. The data collected is represented in a time dependent intensity curve. Over the years, several multivariate data analysis techniques have been proposed to characterize time–intensity curves. One specific technique, fitting a parametric model to individual respondent curves, has been recently proposed. The model parameters quantify meaningful characteristics of the time–intensity curves: up and down slopes, times at which the curves reach and begin descent from the peak height, and the peak height itself. The use of statistical experimental designs to direct the creation of product prototypes so that the effects of ingredient levels and/or processing condition changes can be statistically modeled has become prevalent in the food industry in the last decade. Use of these designs in projects for cost reduction, quality improvement and variation reduction has helped to make the product development process more scientific and efficient. A common application of these designs in the product development process has been with consumer acceptance measures as responses to determine optimal product formulations. This paper discusses how the combination of the two methodologies has been used to identify ingredient levels and/or processing conditions that most affect product texture variability (product texture in this case being a temporal phenomenon). The parametric model fitting process, assessment of respondent repeatability and reproducibility, and the statistical modeling of the time-intensity response curve parameters with respect to the statistical experimental design are described in detail. A discussion of how the resultant modeling directed future product development efforts demonstrates the utility of pairing these methodologies.

Chapter 12 – Response Surface (Optimization) Experimental Designs

Maria use a response surface design to determine factor level combinations that will deliver the desired viscosity.

Descriptive Statistics and Graphical Analysis

Maria is asked to compare the viscosities from two production lines and uses descriptive statistics and graphs to make a determination.

Chapter 9 – Process Capability

The issue of dispensation of product was put on hold due to potential out of specification viscosity levels passed, and the team focus has resumed delivering the best product on a consistent basis. Lisas recent feedback had been that there was a positive impact on the BBQ lines since the SPC system was implemented. She asks Maria to present the dynamic SPC monitoring system implemented earlier on the BBQ lines to the operations management team.

Mixture Experimental Design

Steve opens up the research and development leadership team meeting bragging about the power of the work that Maria has just finished. Design of experiments has presented a powerful approach that had not been used before. Marias work has helped the plant not only to deliver more consistent BBQ viscosity, but also the systematic approach has led to a better understanding of how factors such as steam pressure and line speed affect viscosity. Steve has in mind several improvement opportunities, and he hopes that by applying the design of experiments methodology to them, the results will be similarly positive.

Statistical Process Control (SPC)

Now that the team has determined a way to adjust the process to change viscosity by adjusting the steam pressure, Maria has been asked to determine a strategy for monitoring and controlling viscosity during production. She has seen that there is not much consistency in what is currently being done and that each line operator has their own practice for how often they monitor viscosity and how they adjust process settings based on what they see.

Regression and Correlation

Maria uses regression and correlation analyses to determine ingredient and production factors affecting viscosity.