The Knowns and Unknowns in mLearning

  • Ustad Mobile
  • November 18, 2013
The Knowns and Unknowns in mLearning

It is the question that will not die and doesn’t evolve as it should. Can technology improve learning outcomes? We have known for decades that certain things can help in the learning process: computer aided instruction, sesame street, reminders, feedback since way before the proliferation of mobile phones. The right questions are how?, how can we measure that?, and most important: how much does that cost?

A recently published meta analysis of 76 randomized control trials (RCTs) on interventions for improving education in developing countries found the largest effects came from computers and technology.

Known Knowns

Before lamenting the state of evidence on the role of technology in improving education; I wish to lament the state of evidence on improving education in developing countries more generally. That is not without reason; these are difficult to reach places. In some countries in remote areas we don’t even know if the school building physically exists to the extent we need GPS enabled smartphones, drones or satellite imagery just to be sure that the school is actually physically there. Knowing or measuring what students are learning, assuming they actually physically come to the building, is a new challenge. So: A known known is that we often don’t know what students and teachers know or don’t know there if we don’t so much as know that classes are running there.

What does mobile learning really include? Well using apps is a form of computer aided instructions. There are serious games, so game based learning research is hardly out the window. Reminding learners about something via SMS is a form of reminder nevertheless. Even we have research from Matthew Kam’s MILLEE project that shows this research still holds true when applied to using games for learning on mobile phones: there were statistically significant improvements in learning outcomes.

Patrick McEwan just published meta analysis of 76 RCT’s locatable on education improvements in developing countries (that is up from a total of 9 being available in 2003) – even the dramatic rise in RCTs means there is a maximum peak rate of 6 or 7 RCT experiments in developing countries published per year. The number one program type that achieved statistically significant improvements in student learning: “computers or technology”. Just ahead of teacher training. RCTs are not cheap exercises; insisting on RCTs before more basic experimenting creates a no innovation chicken and egg loop.

Evans figure 2

[Figure 2 taken from David Evan's (World Bank) blogpost on Patrick McEwan's meta-analysis blog]

We know we know that students can know more faster with technology enhanced learning and we know this is applicable to small portable computers (also known as mobile phones).

Known unknowns

Key recommendation of the report: report cost data. 56% of these interventions had no data on cost; that clearly is a problem. We still don’t have sufficient comparable cost data on what works to use for decision making on opportunity costs. Right now it looks like computers or technology are relatively expensive: so let’s rework the question:

The question is not “does it work?” The question is: how much does any of this cost and then how can we do that cheaper and sustainably?

We don’t know how to build capacity to create mobile learning applications. That’s what we’re working on with Ustad Mobile. I know if you look in the app store you won’t find any way to make your own materials for low cost devices without learning how to code BREW or Java 2 Micro Edition (and re-invent the wheels of audio video slides, quizzes and mini games). Given that we know cost is a problem spending time and therefor money re-inventing this is a problem.

We know that we don’t know enough about the cost of technology for people to learn to know stuff.

Mike Trucano of the World Bank has a very good answer that addresses this: The best technology is the one you already have, know how to use, and can afford . I’d say we really have to do better with the devices we have: there’s a whole lot that needs more than 160 characters in an SMS message to teach. But these $30+ Java enabled feature phone of which there are 3 billion out there now can do audio, video and small apps. Ustad Mobile will enable all of those to provide a full computer aided learning experience.

Unknown unknowns…

Ehhh… hmm… Where’s my hoverboard already?

This post was republished from Ustad Mobile as part of NetHope's effort to facilitate collaborative learning and community knowledge-sharing. Please click here to read the article in its original form. We are always looking for relevant and thought-provoking ICT-related posts to republish. We value your suggestions; if you'd like to recommend a post, please email our Editor-in-Chief Paige Dearing at


Our website uses Cookies to enhance your experience and to measure, understand, and improve our website. By continuing to use this website you agree to the use of Cookies. If you do not agree, you can decide to disable Cookies in your web browser. Learn More About Cookies
Got it!