Daniel Ferguson

Work in Progress - Evaluating the impact of reflective thinking on learning objectives in undergraduate multidisciplinary project teams

The interprofessional projects program (TPRO) is a project-based course through which all undergraduate students are expected to develop their multidisciplinary teamwork, communication, and project management skills and knowledge while learning how to resolve ethical issues. Evaluation of these learning objectives is done through various performance measures of the program including a teamwork survey, a self-assessment of the learning objectives, and scores at IPRO Projects Day. A continued pilot for assessment is reflections, a written assignment team members complete 3 times over the semester. The fourth semester of our reflections pilot at Illinois Institute of Technology (IIT) includes questions about project management, teamwork, communications, and ethical issues, which are the meta-learning objectives of the program. We have compared the 11 teams that used reflections and 24 teams that did not use reflections in the Fall 06 semester, and 11 teams that used and 28 that did not in the Spring 07 semester. Overall, we find little meaningful differences between the teams on the performance measures.

Measuring the Levels of Reflective Judgment in Interprofessional Projects

The interprofessional projects program (IPRO) at IIT is comprised primarily of undergraduate student project teams focused on solving a real world problem. Reflection in service-learning and applied settings has been found to increase student ability to analyze problems. In the spring of 2006 we implemented a reflections pilot to test the level of reflective judgment in the students involved in the IPRO program. Reflections were coded to assess the degree of reflective judgment. Our results from spring 2006 indicate students function at pre to quasi reflective level of reflective judgment. The data has provided information to improve our reflections within the IPRO program.

Work in progress - improving interrater agreement used to measure learning outcomes

The Inter-professional Project Program (IPRO) at IIT hosts a one-day evaluation event (IPRO Day) every semester, during which 70-120 judges (~50% new each semester) judge 35-45 teams averaging 12 students each. The judges, in groups of three to five, rate ten criteria per team on a 1-5 scale for either group oral presentations or exhibits. While the statistics of measuring interrater agreement are well documented it is not completely clear from previous research how best to improve such agreement except for assertions that rater training is beneficial. Over the past two years, we have instituted several interventions to improve interrater agreement on IPRO Day: using experienced judges to coordinate each judging group, rewriting judging forms to match the learning objectives, developing rubrics to guide judging, and conducting orientation sessions with the judges. This study evaluates the effectiveness of these interventions by comparing judging group agreement results from semesters prior to the change with the semesters following the change. These results having broad applicability to all judging review panels.

Work in progress - enhancing ethical awareness within undergraduate multidisciplinary teams by preparing Codes of Ethics

IIT, Lehigh, Michigan Tech, and Rice universities are piloting a strategy based on the book by Jones & Ferrill, The Seven Layers of Integrity (2006), to teach ethical awareness to engineers and scientists. Students at the four universities are expected to prepare a code of ethics for their own project or course problem context. These codes are evaluated and each code of ethics must have an over-arching principle as well as seven canons describing the standards of conduct to which the individual or professional working in the problem context shall be held. Each canon must be supported by descriptions of ethical pressures and risks. Each of these requirements is scored and the scores from these dasiacodepsila evaluations constitute evidence of the extent to which teams understand the principles of ethics and can translate that understanding into a code of ethics.

Work in progress - comparing the results of Reflective Thinking interventions at IIT and Uppsala University

A common problem in higher education is the question of whether or not students are obtaining the skills they need to solve problems in the real world. At the Illinois Institute of Technology (IIT), an attempt to address this problem has led to the concept of Reflective Judgment, defined by Patricia M. King and Karen Strohm Kitchener (1994) as the ability to make good decisions about ill-structured problems, or problems that do not have a single right answer. At IIT, a written measure of Reflective Thinking has been developed using questions included in Reflections assignments. At Uppsala University, a similar interest in the question of how students learn has led educators to experiment with the use of Reflections assignments in their courses as a means of stimulating deeper learning. Comparing IITpsilas scoring of student Reflections for Reflective Thinking to Uppsalapsilas interest in ill-structured problem solving and use of Reflections as an enhancement of the student learning process, it is clear that while these two universities have in common the use of reflections and the goal of helping students develop advanced problem solving skills, they are taking two fairly distinct approaches. What is not clear is whether one approach is more effective than the other approach at enhancing studentspsila problem solving skills.

Work in progress - continuous process improvement in attainment of learning objectives

The Interprofessional Projects Program (TPRO) at Illinois Institute of Technology, a private, midsize, Midwestern university, is a project-based experiential learning experience with the learning objectives of [1] strengthening multidisciplinary teamwork skills, [2] improving communication skills, [3] learning project management, and [4] recognizing and practicing ethical behavior. In the last four years we have developed an assessment system for the purposes of measuring achievement of our learning objectives. In this paper we will discuss how we measure learning objectives attainment and the data and conclusions that prompted us to redefine our learning objectives, and completely revise all our measurement instruments after three years of data collection and analysis. This paper is aimed at helping all undergraduate academic programs attain the ABET EC 2000 Criterion 3 Outcomes, the AACSB requirements for business programs and accrediting requirements of agencies such as the Higher Learning Commission of the North Central Association of Colleges and Schools by explaining the process of establishing and validating, learning objectives assessment systems.

Work in Progress: Assessing Student Acquisition of Knowledge of Learning Objectives for an Interprofessional Projects Program

The Interprofessional Projects Program (IPROreg) at our university provides a multi-disciplinary, project team based course required of all undergraduates which helps develop in these students various knowledge and skills deemed essential by ABET and future employers. This knowledge and skill, defined from our course learning objectives [LO], are competency in teamwork, communications, project management and ethical behavior, developed in the context of addressing a real world problem. Through this research, we are assessing whether or not students that participate in an IPRO course gain a more thorough understanding of the declarative knowledge supporting comprehension of the Learning Objectives by administering a knowledge test to each IPRO student both at the beginning and at the end of each semester. It is beneficial to evaluate methods to measure the knowledge acquisition related to specific skills and behavior as described by learning objectives and defined competencies related to those Learning Objectives