By Colin Bangay, Senior Education Adviser for the UK’s Department for International Development (DFID) in India. The views expressed do not necessarily reflect the UK government’s official position or policies.
Citizen-led learning assessments have been one of the most internationally influential educational initiatives of the decade. However, what of impact in their home countries?
This blog is written on ASER India’s tenth birthday, prompting us to celebrate its success but also look to the future. ASER in India has been ground-breaking, inspiring participatory learning assessments across the globe: Uwezo in Kenya, Tanzania, and Uganda, Bèekunko in Mali, Jàngandoo in Senegal, and ASER Pakistan. The findings of these assessments are widely cited and underpin important commentary on learning in the EFA Global Monitoring Report. Collectively, this movement has been critical in shifting attention away from the exclusive focus on access, brought about by the shape of MDG 2, to one on learning embedded in the post 2015 sustainable development goals and the learning metrics task force.
But what of ASER’s impact on affecting reform efforts? A 2014 RCT impact assessment of ‘Uwezo’ in Kenya concluded the programme had no discernible impact on either private or collective action. This finding echoes a comprehensive survey of community led initiatives by The Global Partnership for Social Accountability, which warns that information alone is not enough to affect change. ASER India’s 2006-14 review of learning trends also tells a disappointing story – at best of learning stagnation – giving rise to two questions – Why might that be? What might be done about it?
On the ‘why’, my hunch would be that India’s disappointing ‘learning trajectory’ emanates from technical, socio-economic and political issues – including:
- Not long enough? Seeing positive change in national learning data in a country with 259 million school students will take time, especially when many new school entrants are first generation learners from the most disadvantaged communities.
- Not surprised? The ‘Uwezo’ study revealed parents were not surprised by findings – after eight year of ‘bad news’ some might argue that ASER India may have lost its shock value.
- Community not empowered and equipped? There’s a big difference between ‘knowing’ and ‘acting’ (as all dieters are aware). For progress you need ‘motivation’ and ‘mechanisms’ to make change happen.
- Government not engaged? Like it or not, in most countries the government is the main provider of education. A universal truth for improvement anywhere is a community with ‘voice’ and a government with ‘ears’ to listen and ‘teeth’ to act.
So what might be done?
- Move beyond bad news … bad news fatigue undermines motivation, displaces engagement with solutions, and can alienate. While media attention focuses on the ‘raw numbers’, much less is made of ASER’s broader policy relevant contributions spanning primary to secondary and teacher to out of school variables.
- Triangulate – ASER’s 2011 report cleverly referenced a range of learning assessments concluding that “broadly all indicate that learning is poor in Indian Schools”. This, approach made it harder for critics to dismiss ASER’s findings or allege that they were overly promoting offerings of its parent NGO, Pratham. There’s lots of compelling data out there, including the government’s own National Achievement Surveys. These broadly corroborate ASER’s findings. Further, they provide enriching insight on where learning challenges are by subject, and, are increasingly looking at the ‘spread of performance’ by gender, rural-urban and social group (there’s lots hiding underneath performance averages in India!).
- Seek government buy in: In 2011, the ASER India report was launched by the then Minister of Education. This year, there was no high level ministerial engagement. ASER figures do feature in influential government publications, e.g. the 2015 Ministry of Finance Economic Survey and ASER/Pratham is a partner with some states. Securing government engagement isn’t always easy – but it is critically important. Perhaps there is learning to be gained from ASER Pakistan and ‘Uwezo’ in E. Africa where government involvement appears more active.
- Strengthen Comparability: ASER is a monumental annual multi-lingual undertaking. Add to these changes in test forms and test protocols over the years and margins of error are inevitable. Advances in psychometrics mean there is always scope for improvement in comparing across time and geography. ASER has begun a technical dialogue on this issue with recommendations captured in the last chapters of a recent British Council Report. For the future, ASER must move beyond sampling only rural schools as India is projected to be 50% urban by 2030.
- Go beyond benchmarking: Consider complementing ASER with smaller sample assessments which get granular about where the learning impediments lie. Build a virtuous cycle of assessment to learning action – and pay particular attention to strengthening the link between diagnosis and action.
Click to enlarge
- Reach out – use ASER India to engage others: India’s learning challenge is massive, but in comparison to say environment, it is one around which most everyone is prepared to unite. New ‘crowd sourcing’ type approaches such as Ideo offer exciting ways to harness India’s drive and creativity. And there’s already plenty out there – from enlightened philanthropy such as Central Square Foundation through teacher led approaches such as STIR.
As we shift to the new Sustainable Development agenda, we all need to recognise that delivering learning in classrooms is a far bigger challenge than building them. It is an exciting endeavour, with big prizes when we get it right, but as DFID’s 2013 learning framework shows it is a multi-dimensional challenge and complex challenges aren’t prone to quick fixes!
India’s size makes for a massive learning challenge. And yet the country is blessed with an incredible capacity for innovation. Success lies with the energy, drive and creativity of its dedicated officials, citizens and NGOs. There’s plenty of room for collaboration and ASER is well placed to play a catalytic role. Knowing the ASER team – they’re not ones to rest on their laurels – lots more to come in the next ten years!
The failure of the various assessments to improve outcomes is based on a fallacy: that teachers will somehow adjust instruction once they see test results. The staff who make this implicit assumption were sent to good private schools, where teachers could do this – on a small scale. And most of donor staff went to good schools!
Practically no assessments in poor countries have been known to influence instruction. The “citizen” assessments are no exception. But donors are willing to spend billions to improve learning, and the various consultant companies sell them hope. It’s simpler to test than to figure out how to teach. So assessment the myth goes on.
If the donors, consultants, etc. want to get serious about teaching the poor of entire countries,
teachers must know exactly what to do and must be supervised while doing it. To figure out what to do, memory research must be put to use. Also the methods must be simple enough for less educated teachers to carry out.
Without the help of cognitive science in the classrooms of the poor, the donors’ money will continue to be wasted. The only hope is a miracle.
Both countries with good results in international races like those of the bottom of the list have in common A COMPLETE IGNORANCE of brain activity in the apprehensive process. This makes this laden with INTUITION AND LITTLE SCIENCE educational activity, forcing the results at the expense of the human condition. Well then, in general, GOOD RESULTS IS SYNONYM OF HIGH LOAD EMOTIONAL NEGATIVE for most of the people on the sidelines of the economic condition. This situation is possible to reverse if and only if teaching in great part the dominant intuition, which is made of the need TO INVEST IN THE SCIENCE THAT’S RESPONSETO THE PROBLEM. The mother learning is the learning of the reading, a topic with a high failure rate, adults of all countries with higher studies suffer from the problem of READING WITHOUT A FULL UNDERSTANDING until the last moment of his life. THE SOLUTION TO THIS PROBLEM IS THE SOLUTION TO THE EDUCATION PROBLEM OF THE PLANET. For Spanish language MI MANUELITO (see youtube video) is the solution fully proven in the Peru in a red zone of Callao with children with LEAD problems. Those children after the experience found its high learning potential and many of them became outstanding in their respective schools, to the point that, by 2015, many of them are COLLEGE STUDENTS. POSSIBLE EDUCATIONAL QUALITY AND A GOOD LIFE IF THERE IS SCIENCE IN THE PROCESS and above all, THE EXPERIENCE IS MADE REPLICABLE avoiding the current divorce between ASSESSMENT AND COMMUNITY.
Tanto países con buenos resultados en las pruebas internacionales como aquellos del fondo de la lista tienen en común UNA COMPLETA IGNORANCIA de la actividad cerebral en el proceso aprehensivo. Esto hace que la actividad educativa este cargado de INTUICIÓN Y POCA CIENCIA, forzando los resultados en desmedro de la condición humana. Así entonces, por lo general, BUENOS RESULTADOS ES SINÓNIMO DE ALTA CARGA EMOCIONAL NEGATIVA para la mayoría de la gente al margen de la condición económica. Esa situación es posible revertir sí y sólo si la actividad docente dejara en gran parte la intuición preponderante, para lo cual se hace de necesidad INVERTIR EN LA CIENCIA QUE DE RESPUESTA A LA PROBLEMÁTICA. El aprendizaje madre es el aprendizaje de la lectura, un tópico con un alto índice de fracaso, adultos de todos los países con estudios superiores padecen el problema de LEER SIN UNA PLENA COMPRENSIÓN hasta el último momento de sus días.LA SOLUCIÓN A ESA PROBLEMÁTICA ES LA SOLUCIÓN AL PROBLEMA EDUCATIVO DEL PLANETA. Para lengua española MI MANUELITO (ver video en youtube) es la solución plenamente comprobada en el Perú en una Zona roja del Callao con niños y niñas con problemas de PLOMO. Esos niños luego de la experiencia descubrieron su alto potencial de aprendizaje y muchos de ellos se volvieron sobresalientes en sus respectivas escuelas, al punto que al 2015 varios de ellos son ESTUDIANTES UNIVERSITARIOS. ES POSIBLE CALIDAD EDUCATIVA Y UN BUEN VIVIR SI HAY CIENCIA EN EL PROCESO y por sobretodo, LA EXPERIENCIA SE HACE REPLICABLE evitando el divorcio actual entre EVALUACIÓN Y COMUNIDAD.
With reference to this article by Mr. Colin Bangay, sharing here an experiment by NGO Door Step School being run for the last four years in primary schools of Pimpri Chinchwad (PCMC) and Pune Municipal Corporation (PMC) schools (in Maharashtra, India).
Background
Door Step School (http://www.doorstepschool.org/) is an NGO registered in 1989 in Mumbai. It has been working in the field of education in Mumbai since then and in Pune since 1993. All its efforts are entirely geared to creating awareness and opportunities for education; facilitating access to mainstream schools, and imparting education to the most marginalized children in our society. The target group is children between three and fourteen years. Multiple inter-related programs (http://www.doorstepschool.org/pune/projects/) have been developed and implemented to tackle the main problems faced by our public schools viz. non-enrolment, wastage and stagnation.
One such program ‘Project Grow with Books’ was initiated in 1999 in only 10 Pune Municipal Corporation (PMC) schools. What started as an ‘extra reading’ initiative for improving reading skills of children in government schools, its metamorphosed version continues to run today in 240 schools in PMC and PCMC area. The program is run on school premises, during school hours with full cooperation of teachers, principals and school authorities. It is scheduled in the school time-table and uses the time allotted for additional reading which is 70 minutes per week per class of about 40 students. (http://csrforpmcschools.org/partner-detail/The-Society-for-Door-Step-School/2.aspx)
‘Book Fairies’ are grass-root level functionaries who conduct reading class sessions. A team of two book fairies conducts a session. Every session is planned in such a way that every child gets to read a book by himself or herself and also gets to read aloud 5 to 10 lines from that book under the supervision of a ‘Book Fairy’. This takes about 35 minutes of a session time. The remaining thirty five are divided in activities like story-telling, singing action songs, playing word games etc. About 10 minutes of the session time goes in distribution and collection of books. After the 70 minute session is over, children are issued books to take home; they exchange the books in the next session. As a part of this program we take reading ability tests of all children twice a year. The main purpose of these tests is to select books appropriate to reading levels of children.
‘First Steps Forward’ – Our New Initiative
The test results that we got initially were not very different from ASER reports. Year after year we used to find that almost 30% of the 1st Graders were promoted to 2nd Grade even though they did not know all the letters of the Devanagari script – the basic blocks for reading Marathi language. We were not very happy about the situation and decided to act upon it. We went through the steps viz. assess, analyze, and diagnose every year but did not take any remedial action because, as I see it now, our diagnosis was not right. We kept looking at schools and teachers and concluded that poor quality of education or learning is solely because of them.
However, we forgot to take into account the role parents play in children’s education. The number of hours and the efforts educated parents in India, (whose children attend good reputed private schools), invest in their children’s education is something to be experienced to believe! The children who attend government schools have to cover the same syllabus, without the kind of inputs and exposure the private school children get from their parents. Also, in addition to regular support and inputs, private school children have a head start because all of them have pre-primary education and some of them even have experience of play-school.
Once we examined the problem through this perspective, and accepted it as reality, we started remedial measures. We called this initiative “First Steps Forward”, meaning that 1st Graders who lag behind will step forward and come up to the expected level of 1st Grade before they are promoted to 2nd Grade. Since our main focus is on language and that too, on reading skills, we said that all children in 1st Grade should be able to read a book prescribed for them by the School Board, by the end of the academic year. In terms of learning, it means that all children should be able to read all letters, ‘matras’ (letter + vowel sounds), and composite letter words, without difficulty. We already had Reading Ability Tests designed to measure all these aspects. Since we accepted this diagnosis, we designed a program to substitute parents’ role for the same children.
Before sharing how we achieved this goal, we would like to show here how much we could achieve and how we are progressing as we gain more and more experience of running this program.
Results of the First Steps Forward Initiative –
http://1.bp.blogspot.com/-s4DRFh_UxiA/VZ5gRNAij_I/AAAAAAAABwo/qCxd5NyAN9w/s640/dss_fsf_table1.jpg
The results, as we see them, show three points clearly:
1. The remedial measures we have taken are showing results;
2. We are improving by the year; and
3. We have a long way to go to achieve our goal!
The third point struck us hard in the first year itself. We were aiming to achieve hundred percent but could only cover 50 percent. So we analyzed the data vis-a-vis the children’s attendance and we realized that attendance percentage of a child is an additional factor in diagnosis. Private schools insist on 100 percent attendance. The rules are so strict that no parents dare keep their children at home unless it is an absolute must. In government schools, minimum 80 percent attendance is required. In other words, the goal itself is set at a lower level. In addition, there is no compulsion on parents to explain the reasons for absence.
When we looked at the performance of children with 80 percent and above attendance, we found that up to 85 percent of the children could achieve the expected level. (It must also be noted that few children still remain at the ‘letters incomplete’ level. We still have to look at the reasons for their failure to achieve even the lowest level.)
We started our initiative in 2011. The 1st Grade children whom we covered in 2011, completed 4 years of schooling this year. We looked at their performance as well and the picture is as we see below:
http://3.bp.blogspot.com/-Whttqzt_HeM/VZ5jysItNII/AAAAAAAABw0/g7nfvII_zf0/s400/dss_fsf_table2.jpg
Note: This data is only from 41 schools where we started this initiative in July 2011. The number of children is less because here we only took those children who continued in the same school from Grade 1 to Grade 4.
Details of the initiative ‘First Steps Forward’
1. Our specially trained ‘Book Fairies’ work on this initiative with full support and involvement of class teachers.
2. Sessions are for 35 to 45 minutes every day during school hours and in the classroom itself.
3. Yearly planning is done along with every class teacher. Aim is to reinforce what the teacher teaches in the class. This is further broken down in six-monthly, monthly, weekly, and daily planning. School holidays and other contingencies are taken into account while planning. There is a general chart of action.
4. Children vary in their learning capacities as well as learning levels, hence every child’s progress is assessed on a weekly basis while planning for that week. Based on a child’s progress, different activities are planned for each child; and groups are formed according to learning levels.
5. To give reading practice of the letters, ‘matras’ etc. learnt during a week, a variety of teaching aids (teaching material, activities, games, songs, stories, etc.) are used. In fact, we have identified as many as 12 different ways of teaching each letter through play and other creative activities.
6. Ongoing and individualized assessment; analysis, and immediate corrective steps is the Mantra we follow.
7. In addition to the above measures which are school-based, we involve parents, older siblings, and if required, neighbors, in the entire process. We talk about the Right to Education and bring awareness in the community. We discuss with parents what we are trying to achieve. We show them the books, the content, and explain what we expect from them and how they can facilitate the process even if they are not literate. We provide them with charts, pictures, etc. to be put on their walls. These are based on our weekly planning. Elder siblings are also involved in the same manner. Throughout the year, we are in contact with parents.
8. We plan to work on improving attendance more rigorously from this year. Initially when we started the program, the percentage of children who had 80 percent or more attendance was around 40 percent of total enrolment, but slowly and steadily now it has gone up to 52 percent.
9. It is important to note here that last year, the local government education department requested us to organize training for 103 First Grade teachers from municipal schools where we are working. The training was mainly for making and using simple but engaging teaching aids from easily available reusable material. All these aids are regularly used by our Book Fairies in the classroom. (http://blog.doorstepschool.org/2014/12/parivartan-team-extends-training-to-103.html)
We know this is just the start. ‘First Steps Forward’ is a small initiative in the much larger picture of ensuring learning in school. But we see a ray of hope – enhanced awareness in the communities, partnered with more learning opportunities, innovative teaching approaches, and regular remedial action – this initiative will definitely lead to better learning outcomes.
(http://blog.doorstepschool.org/2015/07/first-steps-forward-initiative-for.html)
It is great to see finally some results, particularly linked to methods that have been tried.
But you can get even better results by maximizing the time that students actually practice reading. Only practice (initially with small units of print) will increase reading speed. The research on learning curves is very clear on this prediction.
As presented above, you are splitting the time between reading and language. Language is great, but is developed much more easily because we have parts in the brain dedicated to it. By contrast, reading involves neuronal recycling and is harder. Please spend the time on reading.
Presenting the letters and matras in ‘creative’ and fun ways must be ok, but you don’t know what is unusual or creative enough to cause ‘flashbulb’ memories of the letters for students. The variable for long-term potentiation is repeated and extensive exposures, maybe 3 times in 10-minute intervals.
Teach the conjoint consonants systematically. I.e. the letters that are split vs. those that are reduplicated. You can leave them until grade 2. Until then, you can write everything using halant. Indians don’t use halant often, but for beginners it works. e.g. dr = d halant r rather than a form children have not learned.
Cognitive science gives pretty good advice on teaching the low-level variables that students need. We have put them to use, and they work.
Dear Ms. Helen Abadzi,
Thank you for the comment and positive, concrete suggestions. We will definitely try them out and document the results.
However, since we are not educationists, certain terms and concepts are new to us and hence difficult to understand. But we will try to understand them and use them in practice and will share results with you and others.
We will be greatly obliged if you can suggest some reading material for improving our understanding of methods you have suggested.
Thanks,
Rajani Paranjpe
Founder-President, Door Step School
Thank you very much for your reply. The smallest units of print are basically letters. It’s best to start with those and string them together.
I am a cognitive psychologist and I have extensively reviewed the research with an eye towards low-income populations. Please see a free e-course at http://www.udemy.com/reading-essentials
Get in touch with me at habadzi@gmail.com
There are indeed some quick fixes. And in some respects one size does fit all. These pertain to the basic neurocognitive variables that are shared by all who are homo sapiens.