Leveraging Technology to Improve Learning Outcomes

Researcher: Karthik Muralidharan (UC San Diego)
Graduate Student Researcher: Sarojini Hirshleifer
Location: India

Learning outcomes and teaching quality are low in most developing countries. While inequalities in school access have been reduced, inequalities in learning outcomes appear to be increasing over time, with students from weaker socio-economic backgrounds falling significantly behind and dropping out at higher rates. Young people are also likely to under-invest in education since the costs of studying are immediate while the payoff is distant and uncertain.

One of the most promising approaches to address both the challenges of instructional quality and of student engagement has been to increase the use of information and communication technology (ICT) in instruction. Governments in a number of developing countries are investing considerable resources to deploy ICT in schools, in some cases providing tablets or laptops for every student—despite the existing evidence that providing hardware alone has no impact on test scores. Thus, ICT in education has not as of yet lived up to its promise as a twenty-first century learning tool.

To address that failure, the Nalanda Project in India has developed and deployed a technology-based learning platform using KA Lite software and a low-cost hardware configuration. KA Lite, developed by the Foundation for Learning Equality, makes high-quality Khan Academy content available to schools that lack access to the Internet. Students use tablets to practice what they have learned through interactive, mastery-based exercises.

This research project had two objectives: i) to conduct a pilot study of the Nalanda Project; and ii) to conduct an experimental evaluation of interventions that aimed to increase student motivation by leveraging the instant feedback component of KA Lite, and providing incentives to students based on both test scores (outputs/results) and practice exercises completed (inputs/effort). This phase of the project was implemented in 45 4th through 6th grade classrooms (across 18 schools) in Mumbai and Pune, India between October 2014 and April 2015.

The Nalanda Project was developed by our implementing partner, the Motivation for Excellence Foundation (MFE), which provided the hardware and developed relationships with the two school partners (Akanksha Foundation and Teach For India).

Results

Despite their growing importance as a policy tool, incentives have not consistently performed as expected in a variety of contexts, particularly for complex tasks. Daily cognitive effort is a key input into many such tasks, but some incentives are designed to instead reward relatively distant outputs. Thus, there are likely to be a number of constraints that make it challenging for people to optimize their production functions to earn incentives: i) there is growing evidence that people may not recognize key inputs into their production functions; ii) people, especially young people, may not have the patience to exert effort now in order to earn rewards later; and iii) some people may not have the self-control to exert effort in every period.

This classroom-level randomized experiment implemented an incentive designed to overcome these constraints by leveraging a technology-based learning platform (KA Lite) to reward students for day-to-day classroom effort-based inputs (exercise module performance throughout the month). The technology is an essential component of making this type of effort-based incentive possible. Without KA Lite (or similar software), it would be difficult and costly to monitor and provide incentives for effort. The impact of the input incentive is measured against an incentive for test performance at the end of the month (an output) as well as a control group that does not receive any incentives. The results indicate that rewarding effort-based inputs has substantial and significant effects on learning outcomes relative to the control as well as relative to rewarding outputs.

Some implementation challenges arose as the result of the Nalanda project being a very new and novel program with complex technology, curricular, and pedagogical components. For example, it took time to ensure the multiple hardware components were compatible, and that the software upgrades developed for the research were stable. In addition, syncing the classroom data was necessary for successful implementation. In order to support that, classrooms were provided with a 3G hotspot, but this approach faced technical challenges. Finally, integrating technology into teaching at scale involves carefully mapping technology-based content to the local curriculum and training teachers to integrate that content into their standard pedagogy.

MFE is further developing the Nalanda model, with the goal of creating a model that is sufficiently stable that it can be evaluated at scale.