Philip C. Abrami

Directions for Research and Development on Electronic Portfolios.

This lead article for the special issue of the Canadian Journal of Learning and Technology explores directions for research and development on electronic portfolios, which are digital containers capable of storing visual and auditory content; software for which may also be designed to support a variety of pedagogical processes and assessment purposes. The paper is organized around several key questions: What are the types and characteristics of electronic portfolios? What are the outcomes and processes that electronic portfolios support for their creators? What are the contexts in which EPs are most effective and worthwhile? Who are electronic portfolio users/viewers and how do we provide appropriate professional development to encourage correct adoption and widespread and sustained use? What do we know and need to know about technical and administrative issues? What is evidence of electronic portfolio success? How do we move forward with funding and infrastructure?

Improving literacy and metacognition with electronic portfolios: Teaching and learning with ePEARL

Can an electronic portfolio that is both a multimedia container for student work and a tool to support key learning processes have a positive impact on the literacy practices and self-regulated learning skills of students? This article presents the findings of a yearlong study conducted in three Canadian provinces during the 2007-2008 school year initially involving 32 teachers and 388 students. Due to varying levels of implementation our final data set included 14 teachers and 296 students. Using a non-equivalent pre-test/post-test design, we found that grade 4-6 students who were in classrooms where the teacher provided regular and appropriate use of the electronic portfolio tool ePEARL (i.e., medium-high implementation condition, n=7 classrooms and 121 students), compared to control students (n=7 classrooms and 175 students) who did not use ePEARL, showed significant improvements (p<.05) in their writing skills on a standardized literacy measure (i.e., the constructed response subtest of the Canadian Achievement Test-4th ed.) and certain metacognitive skills measured via student self-report. The results of this study indicate that teaching with ePEARL has positive impacts on students literacy and self-regulated learning skills when the tool is used regularly and integrated into classroom instruction.

A Review of e-Learning in Canada: A Rough Sketch of the Evidence, Gaps and Promising Directions

This review provides a rough sketch of the evidence, gaps and promising directions in e-learning from 2000 onwards, with a particular focus on Canada. We searched a wide range of sources and document types to ensure that we represented, comprehensively, the arguments surrounding e-learning. Overall, there were 2,042 entries in our database, of which we reviewed 1,146, including all the Canadian primary research and all scholarly reviews of the literature. In total, there were 726 documents included in our review: 235 – general public opinion; 131 – trade/practitioners’ opinion; 88 – policy documents; 120 – reviews; and 152 – primary empirical research. The Argument Catalogue codebook included the following eleven classes of variables: 1) Document Source; 2) Areas/Themes of e-learning; 3) Value/Impact; 4) Type of evidence; 5) Research design; 6) Area of applicability; 7) Pedagogical implementation factors; 8) A-priori attitudes; 9) Types of learners; 10) Context; and 11) Technology Factors. n nWe examined the data from a number of perspectives, including their quality as evidence. In the primary research literature, we examined the kinds of research designs that were used. We found that over half of the studies conducted in Canada are qualitative in nature, while the rest are split in half between surveys and quantitative studies (correlational and experimental). When we looked at the nature of the research designs, we found that 51% are qualitative case studies and 15.8% are experimental or quasi-experimental studies. It seems that studies that can help us understand “what works” in e-learning settings are underrepresented in the Canadian research literature. n nThe documents were coded to provide data on outcomes of e-learning (we also refer to them as “impacts” of e-learning). Outcomes/impacts are the perceived or measured benefits of e-learning, whereas predictors are the conditions or features of e-learning that can potentially affect the outcomes/impacts. The impacts were coded on a positive to negative scale and included: 1) achievement; 2) motivation/satisfaction; 3) interactivity/ communication; 4) meeting social demands; 5) retention/attrition; 6) learning flexibility; and 7) cost. Based on an analysis of the correlations among these impacts, we subsequently collapsed them (all but cost) into a single impact scale ranging from –1 to +1. We found, generally, that the perception of impact or actual measured impact varies across the types of documents. They appear to be lower in general opinion documents, practitioner documents and policy making reports than in scholarly reviews and primary research. While this represents an expression of hope for positive impact, on the one hand, it possibly represents reality, on the other. Where there were sufficient documents to examine and code, impact was high across each of the CCL Theme Areas. Health and Learning was the highest, with a mean of 0.80 and Elementary/Secondary was the lowest, with a mean of 0.77. However, there was no significant difference between these means. n nThe impact of e-learning and technology use was highest in distance education, where its presence is required (Mean = 0.80) and lowest in face-to-face instructional settings (Mean = 0.60) where its presence is not required. Network-based technologies (e.g., Internet, Web-based, CMC) produced a higher impact score (Mean = 0.72) than straight technology integration in educational settings (Mean = 0.66), although this difference was considered negligible. Interestingly, among the Pedagogical Uses of Technology, student applications (i.e., students using technology) and communication applications (both Mean = 0.78) had a higher impact score than instructional or informative uses (Mean = 0.63). This result suggests that the student manipulation of technology in achieving educational goals is preferable to teacher manipulation of technology. n nIn terms of predictor variables (professional training, course design, infrastructure/ logistics, type of learners [general population, special needs, gifted], gender issues and ethnicity/race/religion/aboriginal, location, school setting, context of technology use, type of tool used and pedagogical function of technology) we found the following: professional development was underrepresented compared to issues of course design and infrastructure/ logistics; most attention is devoted to general population students, with little representation of special needs, the gifted students, issues of gender or ethnic/race/religious/aboriginal status; the greatest attention is paid to technology use in distance education and the least attention paid to the newly emerging area of hybrid/blended learning; the most attention is paid to networked technologies such as the Internet, the WWW and CMC and the least paid to virtual reality and simulations. Using technology for instruction and using technology for communication are the two highest categories of pedagogical use. n nIn the final stage, the primary e-learning studies from the Canadian context that could be summarized quantitatively were identified. We examined 152 studies and found a total of 7 that were truly experimental (i.e., random assignment with treatment and control groups) and 10 that were quasi-experimental (i.e., not randomized but possessing a pretest and a posttest). For these studies we extracted 29 effect sizes or standardized mean differences, which were included in the composite measure. n nThe mean effect size was +0.117, a small positive effect. Approximately 54% of the e-learning participants performed at or above the mean of the control participants (50 th percentile), an advantage of 4%. However, the heterogeneity analysis was significant, indicating that the effect sizes were widely dispersed. It is clearly not the case that e-learning is always the superior condition for educational impact. n nOverall, we know that research in e-learning has not been a Canadian priority; the culture of educational technology research, as distinct from development, has not taken on great import. In addition, there appears to have been a disproportionate emphasis on qualitative research in the Canadian e-learning research culture. We noted that there are gaps in areas of research related to early childhood education and adult education. Finally, we believe that more emphasis must be placed on implementing longitudinal research, whether qualitative or quantitative (preferably a mixture of the two), and that all development efforts be accompanied by strong evaluation components that focus on learning impact. It is a shame to attempt innovation and not be able to tell why it works or doesn’t work. In this sense, the finest laboratories for e-learning research are the institutions in which it is being applied. n nImplications for K-12 Practitioners n nWhen implemented appropriately, technology tools are beneficial to students’ learning, and may facilitate the development of higher order thinking skills. nStudent manipulation of technology in achieving the goals of education is preferable to teacher manipulation of technology. nTeachers need to be aware of differences between instructional design for e-learning as compared to traditional face-to-face situations. nImmediate, extensive, and sustained support should be offered to teachers in order to make the best out of e-learning. nImplications for Post-Secondary n nSome educators suggest that e-learning has the potential to transform learning, but there is limited empirical research to assess the benefits. nPost-secondary education would benefit from a Pan-Canadian plan to assess the impact of e-learning initiatives. nIt is important that instructional design match the goals and potential of e-learning. nResearch is needed to determine the feasibility and effectiveness of such things as learning objects and multimedia applications. nProperly implemented computer mediated communication can enrich the learning environment; help reduce low motivation and feelings of isolation in distance learners. nE-learning appears to be more effective in distance education, where technology use is required than in face-to-face instructional settings. nImplications for Policy Makers n nEffective and efficient implementation of e-learning technologies represents new, and difficult, challenges to practitioners, researchers, and policymakers. nThe term e-learning has been used to describe many different applications of technology, which may be implemented in a wide variety of ways (some of which are much more beneficial than others). nSchool administrators must balance the needs of all stakeholders, and the cost-benefit ratios of technology tools, in deciding not only which technologies to use, but also when and how to implement new technologies. nTraditional methods of instructional design and school administration must be adjusted to deal with the demands of distance education and other contexts of technology use. nProfessional education, development, and training for educators must ensure that teachers will be equipped to make optimal pedagogical use of new methods.

An Electronic Portfolio to Support Learning

n this paper, we provide a description of a CSLP research project that looked at portfolio use within a middle school, the web-based e-portfolio software we have developed within the context of the Quebec educational system, our plans for further development of the tool, and our research plans related to the use of portfolios to support learning. Our aim is to combine research evidence on portfolio use with practical feedback from the field in an attempt to develop easy-to-use, powerful software designed to support active self-regulated student learning in schools.

Technology Infusion in Success for All: Reading Outcomes for First Graders

This article evaluates 2 technology applications for teaching beginning reading. One, embedded multimedia, involves brief phonics and vocabulary videos threaded through teachers lessons. The other, computer-assisted tutoring, helps tutors with planning, instruction, and assessment. An experiment in 2 high-poverty, high-minority Success for All schools compared 159 first-grade students randomly assigned to technology or nontechnology conditions in a year-long study. Across all students, significant differences favored the technology condition on Woodcock Letter-Word Identification and Word Attack and GORT Fluency and Total scales (median ES = +0.28). Tutored first graders who received both technology enhancements scored significantly higher on the GORT, Woodcock Letter-Word and Word Attack, Fluency, Comprehension, and Total scales (median ES = +0.53). Nontutored students who experienced just the embedded multimedia scored significantly higher than nontutored control students on Woodcock Letter-Word Identification and GORT Total scores, and marginally higher on GORT Fluency (median ES = +0.27). Results suggested that video and computer technology embedded in instruction may accelerate childrens learning.

Research on Distance Education: In Defense of Field Experiments

This article extends the issues and arguments raised in Bernard, Abrami, Lou, and Borokhovski (Distance Education, 25(2), 175–198, 2004) regarding the design of quantitative, particularly experimental research in distance education. A single experimental, study from the distance education literature is examined from six different perspectives to show the differences between preexperiments, true experiments, and quasi‐experiments in terms of their impact on interpretability and generalizability (i.e., internal and external validity). Arguments for and against experimentation are discussed and the article ends with a description of meta‐analysis, the quantitative synthesis of experimental research, and its potential for providing answers to questions that no single study can adequately address.

Encouraging self-regulated learning through electronic portfolios

At the Centre for the Study of Learning and Performance (CSLP) at Concordia University in Montreal, Quebec, we have developed the Electronic Portfolio Encouraging Active Reflective Learning Software (ePEARL) to promote student self-regulation and enhance student core competencies. This paper summarizes the literature on electronic portfolios (EPs), describes ePEARL, and documents our research findings to date including analyses of teacher and student reactions. Participants in this study were 62 school teachers, mostly from elementary schools, and their students (approximately 1200) from seven urban and rural English school boards across Quebec. Student and teacher post-test questionnaire responses suggested that the use of portfolios, and the learning processes they support, were positively viewed and learned well enough to be emerging skills among students. Contrariwise, teachers commented that teaching SRL strategies was new and thus required a change in teaching strategies, strategies that they were not yet accustomed to. Focus groups also revealed the challenges of using portfolios to teach children to self-regulate. And finally, the analysis of student portfolios evidenced only small amounts of student work or high levels of student self-regulation. Resumexa0: Au Centre d’etudes sur l’apprentissage et la performance (CEAP) de l’Universite Concordia a Montreal, Quebec, nous avons concu le logiciel de portfolio electronique reflexif pour l’apprentissage des eleves (PERLE) afin d’encourager l’apprentissage autoregule chez les eleves et d’accroitre leurs competences de base. Cet article presente un resume de la documentation sur les portfolios electroniques, une description de PERLE, ainsi que nos resultats de recherche documentes a ce jour, y compris des analyses des reponses des enseignants et des eleves. Les participants a cette etude se composaient de 62 enseignants, la plupart dans des ecoles primaires, et de leurs eleves (environ 1200) provenant de sept commissions scolaires anglophones urbaines et rurales du Quebec. Les reponses des eleves et des enseignants au posttest suggerent que les portfolios et les processus d’apprentissage qu’ils soutiennent ont ete percus de maniere positive et qu’ils ont ete suffisamment assimiles pour se traduire par de nouvelles competences chez les eleves. En revanche, les enseignants ont mentionne qu’enseigner les strategies d’apprentissage autoregule etait nouveau et que cela exigeait de modifier leurs strategies d’enseignement pour en adopter d’autres auxquelles ils n’etaient pas encore habitues. Les groupes de discussion ont egalement fait ressortir les defis lies a l’utilisation des portfolios dans le but d’apprendre l’autoregulation aux enfants. Enfin, l’analyse des portfolios des eleves a revele que seulement une petite portion des travaux d’eleves demontrait des niveaux eleves d’autoregulation.

Electronic portfolios in the classroom: factors impacting teachers’ integration of new technologies and new pedagogies

This article presents the findings of a study on the use of an electronic portfolio (EP) in 16 elementary classrooms across Canada. Using a mixed-methods approach, data were collected to understand how teachers used EPs in their classrooms, to what extent they integrated the EP into their practice, and the factors influencing their use. Using expectancy theory, findings indicate that low implementers experienced significant technical obstacles and/or were reluctant to change their established practices, whereas high implementers reported feeling supported by their administration, experiencing growth in their teaching practice, and using more pedagogical practices that support self-regulated learning as a result of the scaffolding provided by the software.

Computer-Assisted Tutoring in Success for All: Reading Outcomes for First Graders

Abstract This article presents a randomized experiment evaluating a computer-assisted tutoring program. The software program, Alphies Alley, provides reading tutors with assessment and planning tools and performance support. It provides students with animated presentations and engaging activities. In a yearlong study involving 25 schools using the Success for All reading program, 412 low-achieving first graders were randomly assigned to be tutored daily for 20 min with or without Alphies Alley. Tutors were also randomly assigned. On individually administered reading measures, controlling for pretests, there were no significant differences overall, but among students with tutors rated as “fully implementing,” those who experienced Alphies Alley scored significantly better on three of four measures. These results suggest that if well implemented, technology that enhances the performance of tutors has promise in improving the reading performance of at-risk children.

Are We Using Technology for Learning

This study investigated the role that computer technology plays in transforming the learning process in higher education. Specifically, we looked at the relationship between computer-technology use, active learning, and perceived course effectiveness. The sample consisted of 1966 students in 81 graduate and undergraduate classes at a large, urban university. The survey categories were: 1) learner preferences; 2) course structure; 3) active learning; 4) time on task; 5) learning with technology; 6) perceived effectiveness of computer use; 7) context of computer use; and 8) overall perceived course effectiveness. Results suggest that there is a relationship between computer technology, active learning, and perceived course effectiveness. Students who use computer technology a lot appear to benefit the most from active learning.