Betsy M. Aller

WeBAL: a web-based assessment library to enhance teaching and learning in engineering

A need exists for classroom-level assessment and evaluation rubrics or instruments to provide meaningful feedback to students on their learning and performance. A web-based assessment library (WeBAL) has been compiled, reviewed, and made available to support engineering faculty when assessing or evaluating students engineering communication or teamwork activities, two of the a-k skill areas required by the Accreditation Board for Engineering and Technology (ABET) for accreditation of engineering programs and curricula. These instruments are annotated to support selection and classroom usage. A simple, online, menu-based selection process guides faculty users to appropriate instruments to meet specific needs, and all instruments may be downloaded and adapted as desired by faculty.

Evaluating and Improving Student Opportunities in a First-year Learning Community: Lifelong Learning and Career Awareness

Faculty researchers at Western Michigan University (WMU) have developed a rubric to holistically evaluate first-year students written responses to co-curricular and lifelong learning activities. This rubric draws on Blooms learning taxonomy to measure incremental levels in investment, meaning, and use. The rubric lists basic recall of the activity in the lower end of the spectrum, while synthesis of the activitys message with ones future plans is part of the highest level. The goals of this assessment include: 1) Determining if career awareness and lifelong learning objectives have been met; 2) Using results of the assessment rubric to improve future learning community activities; 3) Using assessment of written assignments to provide guidelines to first-year students for participation in and response to the activities; and 4) Adding to an existing library of tools and rubrics used to assess student learning in professional areas, such as lifelong learning, communication, and career awareness (the WeBAL Website at WMUs College of Engineering and Applied Sciences)

Work in progress: improving the senior capstone design experience through shared perspectives and best practices

A recent effort to share expertise and best practices in teaching senior capstone design has led to a consortium of engineering faculty involved with the senior capstone experience. This group seeks to better understand the possible approaches to teaching design, while drawing on each others knowledge in specific skill areas. Common goals are to move toward multi-disciplinary design project opportunities and to reduce redundancy of teaching common topics. Toward that end, lectures on a variety of professional engineering topics are being developed and is shared across the College of Engineering, beginning fall 2004. Because the capstone design sequence is increasingly the site of ABET accreditation documentation, an additional focus of this group is to support and enhance assessment activities in capstone design by sharing assessment/evaluation rubrics and best practices.

Refining a rubric for evaluating lifelong learning and career awareness in a first-year learning community

A four-level rubric based on Blooms learning taxonomy was created in 2005 to evaluate student responses to co-curricular activities and determine if the activities career awareness and lifelong learning objectives were achieved. Students were not provided with specific guidelines for the written summaries. Holistic evaluation of summaries by three independent researchers showed that co-curricular activities can be effective in enhancing a first-year students appreciation for lifelong learning as a key part of a professional engineers career. Also in 2005, the three researchers identified key words/phrases associated with each of the four levels of the rubric, and there was generally strong agreement among the researchers in using the rubric in scoring the written summaries. This paper reports quantitative and qualitative results of further investigation on the inter- rater reliability of the evaluation rubric, and whether providing students with a guideline for the written summaries influences their performance. For this expanded research, a total of six evaluators and two groups of 32 students were involved. Results of chi-square tests for the differences between the two groups of evaluators and the two groups of student summaries are presented. This research will expand perception of how Blooms learning taxonomy could be used in lifelong learning assessment, and should be of value to anyone interested in assessing lifelong learning.

Assessment of Faculty Mentoring Strategies of Student Learning Communities at Western Michigan University College of Engineering and Applied Sciences

Learning communities involving 263 first-year students were created at Western Michigan University College of Engineering and Applied Sciences in Fall Semester 2005 to enhance student learning and improve first-year retention. Placing groups of students in the same 2-to-4 classes did achieve the programs goals of creating connection among students and forming student study groups. There do not appear to be any major differences in student satisfaction among learning communities with regular face-to-face contact with a faculty mentor and those without regular face-to-face contact However, it appears that it is important to students satisfaction with the first semester, and perhaps to their more promptly testing their fit in the field, to enroll them in an engineering course early in their academic career

An online assessment system to enhance teaching and learning in engineering

~ This paper describes the development and implementation of an Online Assessment System with intelligent Support (OASIS), which uses an online decision tree to guide faculty through a library of assessment instruments, ranging from simple checklists to detailed rubrics. The initial focus areas are written and oral communication and teamwork, with planned expansion into the other ABET a through k areas. These instruments can be used to evaluate and grade student performance, assess student learning, and/or provide students with assignnrent expectations and criteria. Each instrument is annotated with ifs purpose. application, skill serfs) addressed, and suggestionsfor usage. with additional comments, resources, ondfregucntly asked questions (FAQs). An overview ofthe initial development. testing, and implementation of the OASIS system, the evaluation ofthe system, and ifs potential for broader application to assessment and accreditation activities, are discussed. index T e r m Assessment, Engineering coninrunication. Evaluation. Online assessment system, Teamwork. PROJECT N E E D AND D ESCRlPTlON Meeting ABET’S Engineering Criteria 2000 (EC2000) and Technology Criteria 2000 (TCZK) may pose a challenge for engineering and technology faculty, who are experts in assessing student performance in their disciplinary subjects, but who may be less experienced in assessing some of the a through k outcomes described in Criterion 3111. Materials or instruments for assessing andlor evaluating student performance in these areas are often not readily available, and those instruments that do exist may not be clear in terms of strengths, weaknesses, and best application. ldcntifying relevant assessment instruments often requires much searching and sorting by busy engineering faculty. A need thus existed for compiling, critically evaluating, and disseminating instruments that are annotated as to their application and usefulness in teaching and learning situations. The OASIS project uses technology to serve the needs of engineering and technology faculty in the College of Engineering and Applied Sciences (CEAS), helping them to improve their teaching and assess student learning in the EC2000/TC2K performance areas, across the disciplines and in the 100to 400-level courses, beginning with the ABET criteria of communication and teamwork skills. The outcomes of this project are:

Work in Progress: Western Michigan University Partnership with K-12 Teachers to Improve STEM Education

This paper describes activities and assessment outcomes related to a 3-day workshop held in October 2005 for practicing and pre-service K-12 teachers, offered for the second time by the authors. Six teachers from five K-12 school districts partnered with university faculty and two pre-service teachers to learn about the engineering design process; engineering and technology careers in general; and expectations for K-12 students interested in pursuing STEM-related college studies. Working with university faculty during the 2005-06 academic year, each of the workshop participants designed and constructed a STEM-related module for their classroom, including equipment and lesson plans, which met State of Michigan educational standards. The modules were classroom tested and feedback was documented for further module refinement. Improvements implemented as part of this second workshop, including using a project design notebook and sharing knowledge gained with other K-12 colleagues, and preliminary classroom evaluation showing increased student engagement are presented

Work in progress: A pilot project to assess the added value of engineering and student affairs collaboration on student cognitive and affective development

This Work-In-Progress paper describes a pilot project at Western Michigan University to assess the added value of academic and student affairs collaboration on student cognitive and affective development. The academic performance in first-year science, technology, engineering and mathematics (STEM) courses as well as retention to STEM are used as measures of cognitive development, and the students experience in co-curricular activities as captured in their written summaries is used as a measure of their affective development. While there is no statistically significant difference in individual course performance, first-year Engineering House (EH) students have a statistically significant higher GPA than non-EH in fall 2010, and they have a statistically significant higher fall-to-spring retention to engineering and applied sciences than non-EH students.

Finding the Needle in a Haystack: Using Treemaps to Analyze Supplier Performance

Supplier evaluation is critical to managing an organizations supply base. Key to that evaluation is portraying the information graphically. Organizations with large supply bases find traditional methods are overwhelmed with large datasets. A tier 1 automotive suppliers quality system tracked eight performance measures for its many divisions and plants within the divisions. Each plant had a multitude of suppliers. Even when pivot tables, and pivot charts were used, the effective representation of that information wasnt accomplished through spreadsheets. Treemapping is a data visualization technology that makes efficient use of space and allows very large hierarchies to be displayed in their entirety. It sorts data using a user-defined hierarchy and displays data to meet the users needs and decision making style. This paper provides a case study on the use of treemaps as a supplier performance rating system. The system was presented to users of the Excel-based system and their perceptions and evaluations of the treemap system were analyzed. They found the treemap system provided them with better information, made their job easier, and allowed for a full representation of supplier performance. Users were also appreciative of the ability to drill down on details of specific suppliers.

Work in Progress - Development, Use, and Evaluation of Materials to Support STEM-related Activities in K-12 Classrooms

This paper focuses on the development, evaluation, and classroom testing of STEM-related teaching materials from a June 2004 workshop organized by Western Michigan University (WMU) Engineering Design Center for Service-Learning (EDCSL). The workshop introduced three practicing and two pre-service teachers to the engineering design process; provided them opportunities to develop STEM-related materials with assistance from WMU engineering and technology faculty; provided instruction in writing clear and useable STEM-related materials within the state-mandated curriculum guidelines; and, ultimately, provided relevant and appropriate materials for use in K-12 classrooms. Results from classroom testing and evaluation of materials developed during the workshop or other materials reformulated because of the workshop are presented