Mobile phones and education

Below are notes from the eLearning Africa session titled Mobile phones offering a lifeline to learners.

M4Girls project

Mindset logo Kirston Greenop, research manager at Mindset Network, presented on the current M4Girls (mobile/math for girls) project in South Africa (SA), which aims to support maths learning using a technology that is highly pervasive but not allowed in classrooms. The audience concurred that in most countries mobile phones are not allowed in classrooms and definitely not in exams.

Some project stats:

Nokia 6300 phones used (entry level phone)
43 mini videos (2-3 mins long) loaded onto phones (taken from existing Mindset digital content). Interesting finding: the girls wanted more videos. After watching 3 minute video they asked: “Where is more?” This counters the original assumption that only short mobile movies will retain user attention.
3 “mobisode” (mobile episode) animations.
2 games:
- An overt maths problem solving game
- An implicit business development simulation game
All curriculum aligned
Grade 10 girls: 20 got phones, 20 in control group who didn’t get phones
6 month project

Initial results of the project:

The games are of a good quality, comparable to or better than anything else on the market in SA.
Exceptionally high usage of the games by the learners.
Problem solving is collaborative. When stuck with a problem, the kids asked each other and their siblings.
“Teacher in my pocket”.

Challenges to watch:

Power dynamics between learners and teachers. Teachers want to mediate the usage of the phones and they can’t do that when the learners take the phones home.
Don’t constrain the phone. Ideally the girls would have prepaid minutes to go online, chat, research, etc.
Youth are a very discerning market. Don’t make it too educational, they will dump it.

Future plans:

To extend the content to teachers and parents, to involve them.
To speed up research and increase game features, develop levels, etc.

Full, in-depth results of the project will be available in November 2008.

Adapting tertiary education learning environments to mobile devices

Project at a Nigerian university where many students work full-time and thus need a learning experience that does not rely on classroom time. A learning repository was developed (7 undergraduate and 2 post-graduate courses) that can be accessed via a desktop PC or a mobile device (phone, PDA, etc.) The content is stored in XML and style sheets are applied when serving to different devices. Most students in Nigeria have mobile phones with internet access capabilities, thus can access the “learning on the move” service.

Limitations found in the project:

Power and memory limitations of mobile phones.
Internet connection costs.
User interface problems: small display, confusing layout.
Need to remember the importance of human face-to-face learning.

Uganda MarketInfonet

Ignatz Heinz presented on a Ugandan concept project for context-based basic skills training — literacy and numeracy — amongst rural farmers and fisher folk. In Uganda there is a high level of illiteracy, especially in the agricultural sector that makes up 80% of the work force. MakertInfonet is an internet-based knowledge management and SMS-based communication tool to provide access to market information and sound agricultural practices. The project wants to embed the learning of literacy and numeracy in everyday life contexts and so have chosen to capitalise on this existing tool that is used by many people, everyday, for their livelihood. Right now they don’t know how this project will play out, but the concept is certainly interesting. Links: INFONET-Biovision and Avallain.

Effective evaluation of ICT for education in Africa

Today I attended a pre-conference workshop on the effective evaluation of ICT for education in Africa, hosted by David Hollow, a doctoral researcher at the University of London (Royal Holloway). Below are my notes.

In groups we discussed the following questions related to monitoring and evaluation (M&E):

What is the most significant challenge facing effective evaluation of ICT for education programmes?
Why do you think this?
How do you think the challenge can be addressed?

Some suggestions for what must change in M&E:

Important to give teeth to M&E, in other words tie it to funding, staff promotion, performance appraisals of project stakeholders, etc.
Funders, donors and administrative organs must drive effective M&E. It must be required and included in the project — not summatively at the end but throughout the project.
Multidisciplinary approaches must be taken in M&E, especially within universities.
Develop skills of monitors and evaluators.
When running a project across a diverse range of schools, it can be problematic to apply one evaluation approach. Schools that are well-resourced will respond differently to poor schools, but we tend to apply the same measurements for impact.

Presentation 1: John Traxler, of the University of Wolverhampton, presented on some of the legal and ethical challenges of M&E:

Informed consent:

Obtaining consent can be difficult, e.g. through parental permission.
Participant risk, e.g. perhaps by giving them expensive mobile phones in a poor community. Or asking potentially embarrassing questions in a focus group discussion.
Participant withdrawal: How does this affect the project results?
Financial or in-kind compensation: What? How much?

Confidentiality:

Data may be used by other organisations not initially part of the project.

Power, class, difference:

Evaluation often works across differentials in power and class.

Presentation 2: Til Schoenherr, inWEnt, presented on Capacity building in elearning: unintended outcomes. inWEnt runs a number of elearning capacity building projects. Below are some of the unintended outcomes that he put down to elearning:

In alumni network of trainees, cultural and religious diversity has not resulted in any racism or discrimination.
Change of teaching/learning patterns, e.g. Teachers coach rather than instruct.

Presentation 3: Bjorn Everts, Education Manager, Eduvision, presented on Ethiopia XO-5000, and spoke about the benefits of conducting M&E throughout the project life cycle. In the Ethiopia XO-5000 project 5,000 XO laptops were introduced into five Ethiopian schools. Eduvision developed the software for the laptops. They conducted a 3-stage evaluation — before, during and after — to assess the feasibility and impact of introducing “innovative learning” in Ethiopian schools. Followed a multi-method approach – quantitative and qualitative methods. Some findings:

A challenge was that there was no initial consensus among project partners about the aims of the M&E, or even what M&E is.

It is important to create feedback loops throughout the project and to constantly revise your plan. As part of initially refining the plan it is useful to take the M&E plan to each project stakeholder and talk it through with them. It could prevent the making of mistakes.
Remember to keep focussing on the aims and objectives.

Important lessons learned:

Spend as much time as possible to build local capacity for feedback, input and self-reflection. The teachers were best suited to monitor the project.
Conduct as many of the methods in the local language as possible.
Ensure all parties understand M&E before proceeding. Don’t assume everyone knows what M&E is. Explain everything in layman’s terms.
Very important to emphasise that you don’t want canned feedback. The most valuable feedback is honest feedback.
Be flexible with multiple partners.

Two of the more obvious points are:

Ethics and buy-in: comprehensively inform your participants what the M&E process is about and how important their participation is.
Document the indirect effects of M&E and communicate this to managers.

Presentation 4: Prof. Tim Unwin. Findings of a recent one-day workshop on M&E processes of ICT in education projects in the Middle East.

You can’t change education over night. So, what are the short term and long term indicators for M&E? What are we looking to change and therefore measure?
Dissemination of M&E findings is very important. But we need to overcome the fear of sharing negative results. The real issue is not always the findings, but the process.
Tim asked how we encourage a culture of M&E in the whole project team? It was suggested to include M&E in the training of the team, and to keep pushing the message that M&E is about identifying gaps in projects or organisations and that the end result is not to lay blame but to improve projects or organisations.
An interesting question was: Should we have pilot projects at all, or start at scale? Challenges and problems only emerge when we go to scale. Why not go big from the start?

Presentation 5: A Canadian academic presented the case study of a palliative care training project in Canada. She noted that in the input –> process –> output framework of projects, inputs are individual and organisational, especially around goals of the project. For example, what does the organisation want (e.g. x number of bums on seats in a training intervention) and what do the beneficiaries of the project want out of it (to learn how to provide palliative care)? Defining these different stakeholder goals upfront, and making sure that all stakeholders are aware of the full set of goals, is very important for effective M&E.

Final comments:

M&E of ICT for education projects is hugely challenging. Because of this, effective M&E is often marginalised or trivialised.
Every situation has a unique context. There is no one-size-fits-all approach to M&E – but that doesn’t mean that there aren’t good practices and lessons to learn from other efforts.
One of the participants asked for practical advice on how to conduct M&E by non-experts, with no budget. An “M&E for Dummies”.
Think about how best to communicate the findings to the stakeholders.
There aren’t that many experts in this space. Have the courage to try your own approaches or the approaches of others even if they are not formal experts in the field.
Important to have a framework: plan, process, indicators, partnerships, etc.
David Hollow listed the four Ps of M&E:
- Stakeholder participation
- Partnership
- Plurality of methods
- Focus on process (and not outputs)

For me a key affirmation that came out of the final discussion was that we should not be afraid to follow M&E methods that might not be well known, but that are good fits for our projects. There are not many true experts in the this field, and with constant budget and time constraints, we often have to conduct M&E ourselves.

eLearning Africa 2008 kicks-off

I’m in Accra, Ghana, for the third annual eLearning Africa conference. Approximately 1,500 delegates from 83 countries — mostly in Africa — are gathered to discuss, share and promote various approaches to ICT in Education. The conference covers primary, secondary and tertiary education, and includes aspects such as teacher training, infrastructure, content and open educational resources.

Myself and Karien Bezuidenhout outside the Accra International Conference Centre where eLA was held. “Akwaaba” means welcome.

Western Cape Education Budget 2008/09

Western Cape Education Minister Cameron Dugmore presented the province’s Education Budget for 2008/09 today. Of interest is the following:

ECD: Funding for Early Childhood Development (ECD) including Grade R increases by 37.6% to R226.792 million in order to expand enrolment and improve quality. This includes R91,372 million to ensure universal enrolment of all 5-year-olds by 2010.
Literacy and numeracy: There will be ongoing intensive literacy and numeracy support in schools where results are below par, with 100 officials and 450 Learning Support Teachers deployed since 2007. There is continued deployment of 510 Teaching Assistants (TAs) in the Foundation Phase. There is a strong Family Literacy pilot where our TAs are working with 280 families. We will launch a major Family Literacy radio campaign next month to reach parents in poor communities.
Mother-tongue education: The Project schools, which wrote their Grade 6 WCED tests through the medium of isiXhosa have multiplied their literacy scores by between three-fold and five-fold, and shown signs of progress in almost all Numeracy topics too. Since this was the first year of inception of the plan, such progress is remarkably encouraging.

It is encouraging that ECD is being invested in. (See my post about the National Education Budget for why.) Usually these speeches mostly focus on what’s going right, as opposed to what’s going wrong, so it’s hard to tell whether a 37.6% increase in ECD funding is enough. Still, it can’t be a bad thing.

It’s also encouraging to see mother-tongue education being researched in the province.

Grand Theft Morals, or ultimate role model?

Grand Theft Morals, or ultimate role model? is a piece written for Tech Leader, the M&G’s new offering in the Thought Leader stable. In the article I highlight some of the learning outcomes associated with digital games. Then I hold up Grand Theft Auto IV against that light, and see if what we know about digital games-based learning still shines through. In case you don’t know, GTA IV is the most popular game of all time, and is also extremely violent.

National Education Budget 2008/09

Today I attended the tabling of the National Education Budget 2008/09 in parliament. Below are my excerpts from Education Minister Naledi Pandor’s full speech:

A total of R123 billion has been allocated to the education sector (both national and provinces).
The recent report of the Committee on School Retention indicated that South Africa (SA) has achieved universal access to primary schooling and near universal access to schooling up to the age of fifteen.
A key intervention, in line with our theme of changing lives and communities, was the implementation of a second-chance programme for learners who failed matric in 2007. The overwhelming learner response to the programme revealed a hungry thirst for education among children we tend to cast off as failures at grade 12. Over 400,000 full-time and part-time candidates are writing exams as we speak – a number very close to our total pool for 2007.
The response to the Kha ri Gude Mass Literacy Campaign has been overwhelming enthusiasm. Gauteng already has 32,000 learning, North West 42,000, Eastern Cape 100,000, and Limpopo 47,000. These are the provinces with the largest numbers of illiterate people. Current enrolments suggest we have exceeded our target of 300,000 enrolled.
In addition to providing adults with the skills of reading, writing, and numeracy (up to ABET level 1), a successful campaign will also mean that South Africa will meet the commitment made at Dakar in 2002 to reduce illiteracy by at least 50%.
During 2008 a key focus will be on a recruitment campaign to attract young people into foundation phase teaching, particularly students keen and able to teach in the various African languages.
Data on un- and under-qualified teachers in the system will be collected. The outcome of this will be the production of a five-year plan for a focused systemic approach to teacher upgrading to be implemented from 2009.

The Education Deputy Minister, Enver Surty, explained that today’s global knowledge economy demands competence in using ICTs. Therefore all teachers and learners must be ICT literate.

During the parliamentary session, other MPs responded to the budget. Of interest were the following points:

The goal of the teacher upgrading program is to have no unqualified teachers by 2013.
A professional teacher development points system will be implemented from 2009. For the first time ever learner performance will be considered during teacher performance reviews. The message from a number of speakers was clear: teachers who are absent from school, or who turn up drunk, or who do no work must be brought to book! Teachers need to be at school, in class and teaching.
In addition to the call for teacher accountability there was overall support for the re-opening of teacher training colleges and for teacher salaries to be increased.
There was a general call for the School Feeding Scheme (aka the School Nutrition Program) to also include high schools. Currently only primary schools are supported.
Apparently SA is a low spender on early childhood development (ECD). This is problematic because there is a proven correlation between enrolment into ECD programs and primary school completion.

(As a final note, being in parliament is quite an experience. It’s a bit of a circus in there, organised chaos. Cellphones are constantly ringing, people come and go, half-baked heckling greets some speakers and one MP even tripped on the carpet as she was walking to the toilet.)

How to stem the Matthew Effect in education

How to stem the Matthew Effect in education is a piece I wrote about a presentation given by Dr Luis Crouch at the Integrated Education Programme (IEP) conference (in February).

The “Matthew Effect” (a term coined by Keith Stanovich, a psychologist who has done extensive research on reading) denotes processes whereby inequality is created or maintained. In literacy terms, learners at the end of grade one who can read well begin a pattern of outperforming those learners who cannot read well. With time, the gap widens. Learners who score poorly in literacy from the beginning will go on to fall behind in all other subject areas. The same applies to numeracy.

Dr Crouch shared his thoughts on how to stem this trend in education.

Notes from the Integrated Education Program (IEP) Conference

A little late, but below are my notes from the Integrated Education Program (IEP) Conference in Pretoria (6-7 February 2008).

Overall notes

The Integrated Education Program (2004-08) aimed to improve the quality of primary education by supporting programmes in teacher education, as well as school management and governance, in selected districts in the Eastern Cape, KwaZulu-Natal, Limpopo and the Northern Cape. The program was funded by USAID (approx. R155m) and implemented in partnership with the South African Department of Education. Overall goal: to improve learner performance in literacy and numeracy (in gr 3), and mathematics and science (in gr 6) in participating schools (the poorest of the poor in SA).

As a result of the program the average improvement in learner performance was 5%. This shows just how difficult it is to raise the average grade.

The director general in the Department of Education, Duncan Hindle, said that we need to:

Shift resources from the FET to GET phases, especially Foundation Phase.
Raise expectations amongst parents and society at large of the high levels of teaching that they should expect for their children, of what the service levels should be. He visited a school where at 11am on a Friday all of the teachers had left for the weekend. When he asked the parents if they were concerned about this, they didn’t realise that this was not normal practice.

General comments from speakers and the audience:

A culture of assessment has not taken root amongst teachers. Not enough regular assessments, e.g. testing, and recording of results.
Language of instruction (LOI): Despite the language policy, there is confusion in the some schools as to whether they should teach in the local language or in English. Some of the schools said that the LOI for gr 3 was isiXhosa, but when the learners were given survey instruments in isiXhosa they could barely understand the documents. The learners managed to answer the English instruments. There is a need to clarify the implementation of the LOI policy.
Teachers have to be held accountable for performance.
At the same time, they must be given solid training and support materials.
It is crucial to develop literacy skills in the Foundation Phase for cognitive development. This requires reading and extended or complex writing, which is more than a paragraph, ideally a page of writing.

Statistical analysis in SA Education

Dr Luis Crouch. See How to stem the Matthew Effect in education. Additional notes below.

Suggestion for improving education in SA:

Make schools divulge their results. Universal external assessment might help.
Ensure that schools understand what their expected grade achievement is.
Popularise good teachers: those that arrive on time, that cover the curriculum, etc.
He gave an example of a school in the EC where the teachers vote democratically on what they’ll teach that week. Can’t run a school system like that.

Mid-90s, almost no public statistical analysis by Apartheid government. Since then, explosion of stats, but how much have we learned? Further, most academics, NGOs and education departments are slow to take up and act on the sound research findings that have been produced in post-94 SA, .

Learner drop-out is something to consider. But a much bigger problem is whether those in school are actually learning anything. The average child in SA learns less than 97% of European kids.

Effect of socioeconomic status (SES): Within the group of low parental wealth there is a big discrepency in reading scores. The poor get highly variable levels of education.

To run a good school, need:

Principal who is pedagogical leader and good manager.
Teachers that follow curriculum.
Learners that are disciplined.
Accountability.

IEP learner achievement results

Ms. Carla Pereira, JET Education Services

Findings
Gr 3: Literacy
Overall improvement in project schools over control groups (using very basic descriptive statistics), though not a huge improvement.

Gr 6: Science
Sometimes in project schools and control schools the grades actually dropped over the course of the studies.

Performance by skills
She confirmed that literacy intervention has an impact on mathematics and science skills that have narrative components. If a learner can’t read, he/she can’t understand maths and science concepts that require reading.

IEP impact study

Eric Schollar provided a qualitative review of the impact of the IEP and also lessons learned about the actual implementation of the program, which are relevant to any large-scale educational intervention.

(Gain refers to the difference between the control and project groups for the pre-, mid- and post-tests.) The percentages given below represent the gain made by the one group relative to the other, not the actual test results. The summary of the IEP impact is as follows (in terms of gain):

Literacy (gr 3): +4%
Numeracy (gr 3): +11.3%
Mathematics (gr 6): -0.1%
Natural Sciences (gr 6): +2.7%

Gain for gr 6 is not significant, but is very strongly concentrated at gr 3 level. Overall, the average gain (gr 3 & 6) is +4.5%. Schools in KZN made the most impressive gains.

Distribution of impact
Of the test groups (127 schools), the distribution of impact (i.e. greater than 4%) of the IEP was as follows:

Positive impact: 45.7%
No impact: 26.8%
Negative impact: 27.6%

So for a program such as IEP, which cost R155m, more than half of participating schools were either unchanged or left in a worse position by the program. It is hard to understand how this happens, but apparently this distribution is about average for large-scale educational interventions. The reason is that some schools are so dysfunctional that more resources provided (e.g. through the IEP) make no positive difference whatsoever. External organisations simply cannot impose managerial authority on those schools. Basically, only the school can save itself!

Analysis of the impact
Schools that made the highest mean gains did so as follows:

Literacy: +35%
Numeracy: +24%
Mathematics: +12%
Natural Sciences: +18%

In other words, in some schools the project group improved the literacy scores by 35% relative to the control group.
Schools with the lowest mean gains:

Literacy: -18%
Numeracy: -9%
Mathematics: -10%
Natural Sciences: -25%

Why was impact concentrated in KZN?
Only in KZN were all scores increased at post-test. Why? The following reasons were offered, all of which are relevant to any educational intervention:

In KZN the learner workbooks were supplied to all schools. Assessment resource banks (ARB) were provided in all participating provinces but in KZN they were accompanied by common assessments (supported by the DoE).
In KZN information from the external evaluation was regularly supplied and performance targets set for schools.
In KZN detailed monitoring instruments were used.

Overall insights
The overall insights and lessons learned for educational interventions are as follows:

Teachers are very strongly in favour of the classroom-level support they have enjoyed through the IEP. Many of them are still uncertain of the practical application of the planning and methodological principles of the OBE curriculum in classrooms. Independent service provider (ISP) field workers are popular partly because they can demonstrate these aspects in real situations. (This insight dispels the commonly held notion that teachers don’t want outsiders coming into their classrooms and helping them to do their jobs.)
Mr Schollar felt that one of the main causes of learner performance improvements is the provision of a syllabus supported by learner workbooks, together with common assessment tools.
Systemic, rather than localised, assessment is vital for the SA educational system. Mr Schollar believes that one of the most important steps taken by the DoE since 1994 is to begin to implement across-the-board assessments. In some instances, local assessments pass learners to the next grade while those same learners fail national assessments.
Because many teachers are “confused about OBE”, IEP training in this regard is one of the most highly valued elements of the program. Quality of outcome was mainly due to the materials supporting classroom teaching.
Teachers very strongly favour the provision of project materials that provide guidance to classroom planning, activity and assessment. Mr Firoz Patel, Deputy Director-General, DoE, who presented later, even proposed that the support materials provide minute-by-minute guidance on how to implement the NCS in the context of the OBE philosophy.

Closing remarks

Dr Nick Taylor, CEO, Jet Education Services

Teachers perceive OBE to be something completely different to their previous understanding of teaching. So they feel that they can’t teach OBE until they’ve been trained and developed to do so. But OBE is not something magical. They are still meant to be imparting knowledge. Need to move beyond dependency culture on training.
Secondly, need to follow text books It’s all in there, just follow the book.
Because of poor teachers, learners are being socialised into a mentality of low expectations, low grades and low performance. The learners don’t know that they should be pushing themselves harder, that they can achieve more.

The need for critical research skills

We often hear educators complain that when their learners use the internet to find information for projects they simply copy and paste from the first reference that comes up on Google.

Vicki Davis, a teacher in the US, posted an interesting blog entry about this:

If students take the “first thing they come to” to determine their opinion, then we are sorely at the mercy of Google’s algorithms and the determination of webmasters who desire to be heard. Understanding how to search, how to validate sources, and even how to use deep web resources is an essential part of being literate.

Citing multiple sources is also essential for the critical researcher. The blog post describes what needs to be done in schools to teach these skills, as well as the barriers to curriculum change that make this a challenge. Some interesting comments from other teachers are also posted.

I support her argument: that “the ability to form one’s opinion and validate sources is the key” for digitally enabled youth. These qualities form part of communication and analytical skills in the 21st century.

YouTube’s OK for scholars, non-profits and the queen

In YouTube’s OK for scholars, non-profits and the queen (Thought Leader) I give examples of academia, international organisations and monarchs using YouTube to share content and engage their constituents.

For these serious groups to use a site hitherto largely reserved for pop culture is interesting. It signifies a shift towards richer visual media, and is part of a broader shift towards a more participatory culture.