You are currently viewing our desktop site, do you want to download our app instead?
Moneyweb Android App Moneyweb iOS App Moneyweb Mobile Web App

New evidence supports the belief that South Africa’s education is not all bad

If samples of schools are not nationally representative, that could result in national averages which are not comparable over time.
Image: Shutterstock

In the past few years, there has been uncertainty around whether the learning outcomes of South Africa’s schools are improving from their historically low levels.

There are three testing programmes South Africa can draw from to gauge trends, and all are international: the Trends in International Mathematics and Science Study (TIMSS), focusing on Grade 9; the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SEACMEQ), focusing on Grade 6; and the Progress in International Reading Literacy Study (PIRLS), focusing on Grade 4.

In 2012, TIMSS results pointed to substantial improvements in lower secondary maths and science since 2002. A few years later, SEACMEQ revealed improvements in the upper primary level.

Then in late 2017 came the news from PIRLS that reading in lower primary had remained static between 2011 and 2016. Was the schooling system not starting to pick up after all? The evidence was inconsistent and unclear.

In late 2019, I was asked by South Africa’s Department of Basic Education, the national authority for schools, to examine the raw data from the literacy study, which are publicly available, to verify the flat no-change trend.

My findings were surprising. The raw data appeared not to have been properly analysed in arriving at the conclusion that there was no progress. In fact, the progress was remarkably large.

It’s important to note, however, that even after improvements, South Africa still underperforms relative to most other middle income countries. What’s encouraging is that there’s a move in the right direction.

Inconsistencies

Testing done by programmes such as these involves selecting a nationally representative sample of around 300 schools in one year, and another such sample in a later year. Students in these schools write tests which repeat identical and highly confidential questions across different years.

This is what makes results comparable in ways that would never be possible in an examination system. It would be impossible to keep past examination papers secret.

If samples of schools are not nationally representative, that could result in national averages which are not comparable over time.

Was this perhaps driving a no-change trend in the Grade 4 literacy results?

Sampling problems had been found to produce inaccurate trends in a few cases outside South Africa.

I found there was nothing wrong with the sampling in the literacy study. But what was completely unexpected was to find that the 2011 to 2016 trend in classical scores didn’t correspond to the flat trend appearing in the study’s official reports. Classical scores are what teachers would be familiar with: 15 correct out of 30, meaning a 50% score.

These testing programmes use a complex statistical approach which results in another kind of score, called an item response theory score. As seen in the graph below, officially South Africa’s item response theory score in the study moved from 323 to 320.

The classical scores, in contrast, pointed to a large increase over the 2011 to 2016 period, from around 32% correct to 42% correct. Of 43 countries with a 2011 to 2016 trend, South Africa’s trend was the third-steepest increase, after those of Morocco and Oman. If correctly calculated, the item response theory scores should have pointed to a similar gain.

The data. Provided by the author

The inconsistency came about when the 2011 national score was converted from one scale to another by Boston College, the institution responsible for processing the data. This conversion affected only South Africa, as only South Africa took part in an easier test in 2011 and also participated in the study in 2016.

In early 2020, all references to the no-change finding were removed from the international report in response to the findings described here. Unfortunately, given that the erroneous trend was published in 2017, it remains replicated in many places.

The correction has important policy implications for South Africa. It removes the uncertainty. All three programmes now point in the same direction, which is upward. This offers hope in a context where South Africa’s underperformance in the international programmes is widely known and lamented.

What changed?

What lies behind the improvement? It’s a mix of education and non-education factors. Urbanisation has improved the access of young people to resources which facilitate learning, from electricity in the home to public libraries. In the schooling system, there’s evidence that access to textbooks has improved. Curriculum reforms have made it clearer what teachers should do.

Not acknowledging that there are improvements raises the risk of policy change, where perhaps policy stability is necessary. On the other hand, South Africa’s historical levels of performance have been so low that it would have been relatively easy to shift scores in the right direction. For further improvement to be assured, it can’t just be business as usual.

Research indicates that better teaching of reading in the early grades, a prerequisite for virtually everything else in education, is possible and necessary.

Understanding the limits.

In a recent report on the attainability of the Sustainable Development Goals in the area of educational quality, I looked at what historical trends across the world suggest are the fastest possible rates of improvement in the international testing systems.

South Africa is in fact making progress not too far from the “speed limit”. An inconvenient truth about schooling systems is that when they progress, they are more like tortoises than hares. This has implications for testing programmes which monitor systemic progress. They need to be sufficiently rigorous and fine-tuned to pick up even relatively small gains.

During the past 20 years, the quality of South African schooling progressed from a level well below that of Botswana, to almost the level of Botswana. But Botswana too is an underperformer relative to income per capita.

Both countries need to improve the quality of teaching and learning much further. A proper basic schooling for all citizens is a human right. And the evidence is clear that quality schooling also bodes well for socio-economic development in the long run.The Conversation

Martin Gustafsson, Education economist, Stellenbosch University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

COMMENTS   10

Sort by:
  • Oldest first
  • Newest first
  • Top voted

You must be signed in to comment.

SIGN IN SIGN UP

FAKE NEWS !! When my son went to a “gubment” school in grade one It took me 8 months to convince his mom that this is a 3rd world “gubment” school. We took him out put him in private school AND HE WAS ALREADY 1 YEAR BEHIND!! Nice try

Not fake news, your son might be an ADD case.

My kid did entrance exams for 2 private schools after coming from a… gubment (?) school. He sat the exams with a stack of other private school kids. My kid got the highest entrance marks. When we asked, he said the test was easy and no different to his current grade and he wasn’t sure why other kids were battling. But those schools were begging us to send him. Granted, he did sports and culture, but his marks is what they were most impressed with. He also wasn’t top of his school, top 25%, but not top.
So we looked into the IEB. There’s no difference. Umalusi see it in exactly the same light. So do foreign varsitys.
We never sent my son to those private schools. We kept him in district, sending him to my alma mater where they write NSC. That school has about 30% of the kids go study abroad without writing SATs, A levels, ATARs etc. The school has relationships and are known to many overseas varsitys (they visit annually to poach the kids). The rest choose to stay in local varsitys, mostly UCT, Stellenbosch, Wits, Tuks, UKZN (where we stay) and UFS, mostly cause the parents can’t afford to send their kids, want them to be closer to home or take up positions in family businesses (because of unemployment and all). Sure, some grades might not be high enough, but then they’re battling to get into varsitys here too. Just like IEB kids.

Maybe your son just takes after you and isn’t smart.

Thank you Martin for an excellent positive article and thank you Moneyweb for publishing one.

I hunt every day for at least one positive article that can show SA Inc is not purely declining.

What is your opinion of Nicola Harris and Click Foundation in terms of the ability to super charge the rate of improvement of the system?

Seeing you’re anxious to find some, here’s more positive news for you: Most people in South Africa won’t get murdered today. There, feeling better now?

That’s really awesome and Amazing thinking Gina! If you read the Sunday times today on Stadio. Chris van der Merwe is stepping down to focus on the private sector training medical doctors in South Africa. This is a game changer and will be a huge boost to SA healthcare!

OMG ….you get your facts and “warm fuzzy feelings” from the Sunday Times?

You must be so ingrained ignorant

The underlying parameter for analysis of an educational system is the measure of the average intelligence of the students.

You cannot compare schools, or nations, or educational systems, without FIRST normalizing the data for this difference in IQ (and also adjust for other material differences, like income inequality for example).

There are huge differences in average IQ across almost any cohort you choose.

Within schools themselves (when I was at school the classes were clearly separated according to academic prowess). And between schools from different socio-economic areas.

And even between nations.

The issue to dwell on, is not that there ARE differences (these are both undeniable AND significant), but what are the TRUE drivers of academic excellence?

It certainly is not just “money”, or “resources”.

Still waiting for curriculum to reflect full transformation , new subject ” Unions and subversive study’s – Grade 8 to Grade 12 “

End of comments.

LATEST CURRENCIES  

USD / ZAR
GBP / ZAR
EUR / ZAR

Podcasts

INSIDER SUBSCRIPTIONS APP VIDEOS RADIO / PODCASTS SHOP OFFERS WEBINARS NEWSLETTERS TRENDING PORTFOLIO TOOL CPD HUB

Follow us:

Search Articles:Advanced Search
Click a Company: