Graham Farrell

Capstone project: fair, just and accountable assessment

Fair, just and accountable assessment for the individual student in the capstone project requires establishing criteria beyond the final project output to meet the objectives of the capstone project experience. Current research is limited and requires solutions that are not subjective, enable formative and summative assessment with student input and are not onerous on supervisors and/or students. This paper draws on input from highly experienced academics supervising capstone projects in ICT, critically reviews recent publications pertaining to capstone project assessment and current capstone students opinions. The outcome of this input led to the development, trial and evaluation of a toolkit to assist supervisors in both formative and summative assessment of capstone projects.

Innovative Online Assessment Using Confidence Measurement

This research investigates and evaluates an innovative approach to a web based formative assessment system. This self assessment tool is based on the traditional Multiple Choice Questions with additional facilitation of registering the level of confidence of the students response to each of the options available. The students generated score is designed to reflect their understanding of the topic under review.A series of experiments involving 93 participating students and 8 instructors were conducted producing some interesting qualitative and quantitative data. Our results show that it appears that this tool has promise as a positive contributor to the instructor and student as they journey along their learning path. In addition this research recommends directions for further studies.

Capstone project online assessment tool without the paper work

The capstone project is the culmination of a students academic learning, providing an opportunity to demonstrate and further their knowledge and their organisational, practical, theoretical and communication skills with valuable guidance from industry partners. From the perspective of both the student and the academic, fair, just and accountable assessment is the ideal, however often difficult to achieve. Previous research by the authors presented a paper based toolkit to assist supervisors in both formative and summative assessment of capstone projects that does not focus entirely on the final product. Trials of the toolkit demonstrated that although it provided an excellent measure of individual student contribution and achievement, it required an inordinate amount of work by both parties to be efficient and effective. To alleviate the workload and provide individual and comparative instant feedback, an online version of the TCS System, has been developed and tested by students and staff. This paper reviews and discusses the structure and benefits of the online system and the results of the software testing in relation to educational values.

Courtroom evidence presentation technology: overcoming traditional barriers

The Supreme Court of Victoria is an heritage listed building steeped in tradition and reticent to physical change. The current capability of the courtrooms to exhibit advanced technology derived evidence is substantially inadequate. This paper discusses the negative environmental impact, the inadequacies of courtroom facilities and inconsistencies in comparison to the available evidence and the requirements of jurors and courts in relation to evidence presentation. It also discusses the issues, possibilities and limitations involved in the implementation of an IT solution to the presentation of evidence.

Designing an online self-assessment tool utilizing confidence measurement

The convergence of multi-disciplined technologies to address and support the rapid evolution of e business is critical to the success of many ventures. There is an increasing demand for the availability of tools to assist in the evaluation and processing of information as it flows between the bodies involved. A tool that could predict and forecast expected occurrences with a stated level of confidence is considered to be extremely valuable and well worth pursuing. The area of Web based learning is rapidly becoming a fundamental component of corporate training areas and educational Institutions, with an increasing reliance on the delivery of their programs via the Internet or Intranet. The success of such programs is dependent on the marrying of Internet technology, Information systems, Human Computer Interaction and Educational Strategies while catering for the individual student.This research investigates and evaluates the design of an innovative approach to a formative assessment that permits the participant to register their level of confidence. The system then provides enriched feedback and can predict the performance of the individual, reducing the number of failures. The system is based on the traditional Online Multiple Choice Questions format, but it facilitates a method of registering the level of confidence of the students response to each of the options available, generating a score that reflects their level of understanding of the topic being considered. The iterative process of design and the results of a pilot program are discussed, and directions for further studies suggested.

An analysis of courtroom evidence presentation technology requirements and current solutions

With the ever-increasing volumes of evidence being used in todays courts, evidence presentation is an area that may be best modernised in Australian courtrooms through the use of Information Technology to greatly increase efficiency and effectiveness. In this paper we have identified some of the key usability requirements for evidence presentation applications to run on tablet technology. This study analyses three current evidence presentation applications for the iPad for their individual usability qualities, and against the requirements established. The study identified deficiencies in the three applications tested, particularly in the area of document annotation. As a result we propose a number of design principles that will ideally improve the usability of evidence presentation applications for iPads and other tablet technologies. This study will be extended to include different tablet operating systems, and focus on the changes in the design principles that are dependent on available tablet work area size.

IMPROVING LEARNING THROUGH INTERACTIVE MULTIPLE CHOICE QUESTIONS WITH CONFIDENCE MEASUREMENT

Multiple Choice Questions (MCQs) offer benefits to both students and educators, however they have been criticized for not offering the ability to test partial knowledge, having limited ability to assess deeper levels of understanding and by nature reward guessing. This research initially discusses and analyses various alternative online MCQ assessment scoring regimes, specifically implemented to eliminate the negative aspects of MCQs. It then compares these previous initiatives to an innovative MCQ with Confidence Measurement (MCQCM), designed and implemented for tertiary level Computer Science students, with the objective of this meta-cognitive approach to assessment is to provide richer formative feedback as a means to improve student learning and enhance the educational experience

Trial by tablet: user evaluation of the digital courtroom

Increasing volumes of paper based evidence documentation and its preparation is creating a growing need for an IT solution in Australian courtrooms. This paper discusses the findings of a study where members of the Australian judiciary, including judges, barristers, lawyers, court administrators, law academics and members of the general public, were introduced to an electronic evidence presentation system in a formal court environment. The purpose of the study was to identify particular needs of the users in order to produce a series of design recommendations for future.

Online assessment: splitting the screen to be seen

The limited viewing space provided by online assessment tools makes it difficult for a student to view supportive graphics, scripts or diagrams where the viewing screen is smaller than the complete question. This paper reports on the user centred design and the application of a bifocal information visualisation technique for the display of graphics of an online multiple choice assessment tool. Traditional online tests require the students to scroll down the viewing page to study a questions supporting graphic or diagram and then scroll up again to insert their response. In many instances, this drastically increased the level of tension in a student, who would lose sight of the graphic when answering the question. Consequently, the increased cognitive load often resulted in errors. We provide a presentation technique that displays the question in readable text with a compressed, distorted vision of the supportive graphic/diagram. This enables students to focus on the graphic when considering their answer while not losing sight of the question itself. Furthermore, it allows the student to toggle freely between a focus on the graphic and the question. Our evaluation of this bifocal display demonstrates a notable decrease in errors being made during transcription of answers, and a decrease in the stress level for students during the test.

Spreading word: author frequency of app user reviews

App stores allow developers to publish new updates directly to users. Users evaluate and leave public reviews of their opinions and experiences for others to see. App ratings and reviews are a purchase determinant for users, and are free user-based usability tests. Existing literature offers approaches to extract information from or to summarise app reviews, but what can we say about the authors themselves? We analysed about 8.7 million iOS app reviews written by over 5.5 million unique authors. We found that 71.5% of authors only wrote one review. Only 13,224 instances of authors re-reviewing were observed, by 12,667 authors for 3,345 apps.