Daniel Rees Lewis

Computer supported novice group critique

Groups of novice critiquers can sometimes provide feedback of the same quality as a single expert. Unfortunately, we do not know how to create systems for novice group critique in design education. We tested whether 4 principles: write-first scripts, critique prompts, interactive critiquing & formative framing, allow us to create systems that combine the advantages of face-to-face and computer-mediated critique. We collected observations and 48 interviews with 12 undergraduate design students who used a computer supported group critique system over 5 critique sessions, analyzed using grounded theory. We found that: (a) the write-first script helped overcome initial learning costs; (b) the interactive critique features created a dual-channel critique that increased the number of critiquers, duration of critique and interactivity; and (c) the system produced a greater volume of useful critique and promoted reciprocity among critiquers. The study provides improved principles for developing computer supported novice group critique systems in design.

The logic of design research

ABSTRACT Since the first descriptions of design research (DR), there have been calls to better define it to increase its rigour. Yet five uncertainties remain: (1) the processes for conducting DR, (2) how DR differs from other forms of research, (3) how DR differs from design, (4) the products of DR, and (5) why DR can answer certain research questions more effectively than other methodologies. To resolve these uncertainties, we define educational design research as a meta-methodology conducted by education researchers to create practical interventions and theoretical design models through a design process of focusing, understanding, defining, conceiving, building, testing, and presenting, that recursively nests other research processes to iteratively search for empirical solutions to practical problems of human learning. By better articulating the logic of DR, researchers can more effectively craft, communicate, replicate, and teach DR as a useful and defensible research methodology.

Agile Research Studios: Orchestrating Communities of Practice to Advance Research Training

Undergraduate research experiences enhance learning and professional development, but providing effective and scalable research training is often limited by practical implementation and orchestration challenges. This paper introduces Agile Research Studios (ARS)--a socio-technical system that expands research training opportunities by supporting research communities of practice without increasing faculty mentoring resources. ARS integrates and advances professional best practices and organizational designs, principles for forming effective learning communities, and design of social technologies to overcome the orchestration challenge of one faculty researcher mentoring 20 or more students. We present the results of a two-year pilot of the Design, Technology, and Research (DTR) program, which used the ARS model to improve the quality of learning, produce research outcomes, and lower the barrier to participation while increasing the number of students who receive authentic research training.

Designing Crowdcritique Systems for Formative Feedback

Intelligent tutors based on expert systems often struggle to provide formative feedback on complex, ill-defined problems where answers are unknown. Hybrid crowdsourcing systems that combine the intelligence of multiple novices in face-to-face settings might provide an alternate approach for providing intelligent formative feedback. The purpose of this study was to develop empirically grounded design principles for crowdcritique systems that provide intelligent formative feedback on complex, ill-defined problems. In this design research project, we iteratively developed and tested a crowdcritique system through 3 studies of 43 novice problem solvers in 3 formal and informal learning environments. We collected observations, interviews, and surveys and used a grounded theory approach to develop and test socio-technical design principles for crowdcritique systems. The project found that to provide formative feedback on ill-defined problems, crowdcritique systems should provide a combination of technical features including: quick invite tools; formative framing; a public, near-synchronous social media interface; critique scaffolds; “like” system; community hashtags; analysis tools and “to do” lists; along with social practices including: prep/write-first/write-last script and critique training. Such a system creates a dual-channel conversation that increases the volume of quality critique by grounding comments, scaffolding and recording critique, and reducing production blocking. Such a design provides the benefits of both face-to-face critique and computer-support in both formal and informal learning environments while reducing the orchestration burden on instructors.