Mark Floryan

Recognizing dialogue content in student collaborative conversation

This paper describes efforts to both promote and recognize student dialogue in free-entry text discussion within an inquiry-learning environment First, we discuss collaborative tools that enable students to work together and how these tools can potentially focus student effort on subject matter We then show how our tutor uses an expert knowledge base to recognize (with 88% success rate) when students are discussing content relevant to the problem and to correctly link (with 70% success) that content with an actual topic Subsets of the data indicate that even better results are possible This research provides solid support for the concept of using a knowledge base to recognize content in free-entry text discussion The paper concludes by demonstrating how this content recognition can be used to support students engaged in problem-solving activities.

Rashi Game: Towards an Effective Educational 3D Gaming Experience

We present an educational 3D game called Rashi Game, which instructs students via exploration using the inquiry teaching method. We first describe the Rashi Intelligent Tutoring System in its original form, and then describe the details and features of Rashi Game, the 3D game that we have developed. In particular, we argue that inquiry-learning environments are particularly viable for educational 3D games. This is because of the similarities between some common game mechanics and the inquiry-learning paradigm. These include freedom to explore open-ended environments, interaction with environments, and realistic scenarios. We briefly summarizing results that have been obtained via pilot studies of the efficacy of Rashi Game, and remark on what directions future work in educational 3D games might take.

Authoring Expert Knowledge Bases for Intelligent Tutors through Crowdsourcing

We have developed a methodology for constructing domain-level expert knowledge bases automatically through crowdsourcing. This approach involves collecting and analyzing the work of numerous students within an intelligent tutor and using an intelligent algorithm to coalesce data to construct the domain model. This evolving expert knowledge base (EEKB) is then utilized to provide expert coaching and tutoring with future students. We can compare the knowledge created in human crafted expert knowledge bases (HEKB) with knowledge resulting from our knowledge acquisition algorithm to judge quality. We find that our EEKB models have qualities that rival that of the human crafted knowledge bases and can be generated in significantly less time. We have built four unique knowledge bases using this methodology. This paper provides a pithy high-level overview of our approach along with some findings.

Optimizing the Performance of Educational Web Services

We describe how web service architectures can provide better performance to applications by offering fine-grained services. We define web service granularity in terms of the amount of data that can be retrieved from a service in a single request on average. This is important because developers cannot predict if students will be using state of the art hardware. Thus, service-oriented architectures (SOA) with fine service granularity can minimize network communication and allow server machines to perform more work for applications. We present the Rashi Intelligent Tutoring System and describe how its architecture has been adapted into a web service with two competing application interfaces. We show how the interface that uses more fine-grained services leads to significant improvements in network message response time, message size, and response size, without a significant change in the number of requests.

Students that Benefit from Educational 3D Games

We describe an educational 3D game called Rashi Game. Rashi Game features a fully functional, and open-ended, 3D environment for students backed by a domain-independent inquiry-learning tutor. We present pilot work that directly compares Rashis classic 2D interface against the 3D game environment. We compare both the students work within the system, as well as their reported sense of presence. Specifically, we notice some interesting patterns in student behavior within the game, dependent on the students preference for games, and argue that there may be potential for modeling when and how to present a student with an educational game based on simple factors such as whether or not the student plays games regularly, or based on student affect (e.g. a lack of motivation).

Who Needs Help? Automating Student Assessment Within Exploratory Learning Environments

This article describes efforts to offer automated assessment of students within an exploratory learning environment. We present a regression model that estimates student assessments in an ill-defined medical diagnosis tutor called Rashi. We were pleased to find that basic features of a student’s solution predicted expert assessment well, particularly when detecting low-achieving students. We also discuss how expert knowledge bases might be leveraged to improve this process. We suggest that developers of exploratory learning environments can leverage this technique with relatively few extensions to a mature system. Finally, we describe the potential to utilize this information to direct teachers’ attention towards students in need of help.

Improving the Efficiency of Automatic Knowledge Generation through Games and Simulations

We have created a generalized algorithm for automatically constructing domain level knowledge bases from student input. This method has demonstrated greater efficiencies than when knowledge is hand crafted by subject matter experts (SMEs). This paper presents two related methods for improving automated knowledge acquisition by leveraging the properties of games and simulations. First, we discuss game mechanics that, when added to our intelligent tutor Rashi, lead to higher quantity and quality of student input. In a separate but related analysis, we present a novel game type called a knowledge refinement game (KRG) to improve the knowledge in an expert knowledge base. This game motivates SMEs to refine the generated knowledge base, especially for data in which the system has low confidence. Utilizing an anonymous agreement policy ensures the quality of SME responses and results show that small amounts of KRG activity leads to noticeable improvements in the quality of the knowledge base. We assert that these two results in unison provide evidence that gaming has a powerful potential role in improving artificial intelligence techniques for education.

A Design Framework for Experiential History Games

This article presents empirical studies of a serious game focusing on various aspects of the American Civil War. We developed and deployed four distinct modules of our game for use within a fifth grade classroom in Virginia, USA. Of the first three modules deployed, only one lead to statistically significant results from pre-test to post-test. We used qualitative information from these first three trials to develop a design framework for experiential serious games of this form. We then developed and tested a fourth module by applying this framework and found significant learning improvements with this fourth module. This paper presents our game, results of empirical studies within a fifth grade classroom, and our proposed design framework identifying key aspects of the learning environment. Our results provide support for our hypothesis that application of this framework leads to increased learning gains. While we do not suggest that our framework is complete or exhaustive, we believe that designers of similar educational games can benefit by employing the principles of this framework directly.