Demystifying data in the social sector

Every good data analyst is aware of the “Garbage in = Garbage out” formula.  What this means is, the analysis is only as good as the data. One cannot expect to get actionable insights from data when there is a data quality issue, data integrity issue & data that does not capture the outcome indicators. 

In the social sector, there is no dearth of data. Every activity could be a step to possible interventions, policy ideas and partnerships. Every day spent on the ground is an experiment in the making, leading the organization to pave new paths to solve a complex problem. When this is the case, how can the organization know if they are onto something great? Through evidence. Always. By recording each activity, by doing that activity over a repeated period of time and capturing that activity in both a quantitative and qualitative manner. This is too basic, isn’t it? Yes, it is. But “capturing the activity” is not something everyone gets right. In fact, how you capture the activity is what most non-profits get wrong.

I’ve had the opportunity to ponder over some of the challenges with data in the social sector since I spend ~40% of my day working as a data specialist at a non-profit organization. And here’s what I have learned - Non-profits can become more "data-smart".

1)      Understanding data in an abstract sense: The term “Data is the new oil” is tossed around so often that people are pressured to believe it. Most people don’t relate to it in their everyday lives, thus they do not find it relevant. Those who believe it is important, do so, due to the donor reporting norms in the organization.

2)      Data is an after-thought: In most organizations, proposals are your appetizers, programs are your main-course, data & M&E is your dessert. For best results, proposals, programs, Data and M&E should be baked together. Data should inform the proposals, data should inform the program design. M&E should not be an after-thought.

3)      Quick solutions over deliberate decision-making: Changes on the ground may affect the program strategy. The decisions for changes in program strategy should not occur in isolation. Temporary data solutions to cater to the immediate program needs can harm the long term data strategies. Elaborate planning on how the data captured due to the change can integrate into the larger program data can save time & effort later.

4)      Assuming systems in pilots will last forever: New programs have different data scoping needs compared to a well-established programs. Pilots are highly volatile & require agile solutions. Organizations should learn from evolving programs and set up a more sustainable data system in time. In short, sustainable data solutions can only be built once the programs become stable.

5)      What data architecture?: Most data collection happens via google forms, SurveyMonkey which leaves data dispersed in unmanageable places & structures. Dashboards are built on the scattered pile of data and more data is added to the sheets, until one day, the delicately balanced data system collapses. A robust data architecture comes with planned data engineering. Identifying data sources, stringing the data together & building a structure that allows for visualization. Visualization should be thought of only after data engineering is well-handled.

Data architecture is the first step for refined impact measurement. Data strategy baked into the organizational strategy is known to most organizations in the private industry, but it is yet to become a norm in the non-profit world. With most activities now moving online, due to the pandemic, we have more data at our disposal now. Using this opportunity to make the shift towards becoming more "data smart" should be the way forward.

Self-Reflection handbook 101

1 whole year has passed by without me taking a break to check into myself. One. Full. Year.

It’s not like I haven’t had opportunities to do so, it doesn’t mean I haven’t reflected at all. I have had few short conversations about things I care about with people who matter to me. But the question I ask myself today as I type this out is, “Has that been enough?” The answer is no. A big NO in fact. 2019 has marked a monumental change in my life.

It’s the year I moved back to India, started my career in the social sector, got engaged to the love of my life, set my wedding date, realized what it means to leave a piece of my heart in a different continent, shifted my perspectives on a ton of issues, got my first climate change panic attack, realized my parents are getting old, saw myself fit and stand out in a career I was proud to have, understood what it meant to do a balancing act with the parents and in-laws, freaked out about getting married, saw my first North Indian wedding, had my first north Indian trip, discovered my love for all things art, cotton clothes and many many other small & big experiences both.

But as I sat down to write this, the blank screen intimidated me. I felt absolute loss for words and all the experiences that had to somehow magically transform themselves into beautiful words seemed to shy into their shells, disapproving the lack of colour that the whiteness of the screen exhibited; the colour that my eyes somehow seemed to get adjusted to. After a lot of deliberation and debate with myself, I gathered these questions I could ask to push myself to think about the past year constructively.

1)      What has become the 3 most important things to me in the past year? (My priorities)

2)      What are the 3 main perspectives I’ve developed in the past year?

3)      What are the 2-3 things have I changed my mind about in the past year and why?

4)      Have I become more empathetic & compassionate compared to the last year? If yes, which incident triggered it? If not, would I like to work towards it, this year?

5)      Have I become more courageous than I was, last year? If yes, what led to it? If no, is that something I’d like to work towards?

6)      Have I identified what my strengths are, the past year?

7)      Do I know what my weaknesses look like the past year?

8)      What are my most effective go-to self-care tips?

9)      What are the successful mantras I’ve identified this year to sustain my relationships with all my loved ones this year?

10)   What is it (are things) about my job that makes me want to go to work everyday?

11)   What is it (are things) about my job that I wish to change?

12)   If I could describe my ideal job description, what would that be. (Create a checklist & check your current job against it)

13)   Has my goal about where I want to be in 3 years changed over the past year? If yes, how & why. If no, is it because I haven’t thought about it or is It because I’m too sure of it?

14)   Have I read enough over the past year? What are some of the reformative / formative areas I’d like to read more about?

15)   What aspects of physical care have changed in the past one year?

These 15 questions are intentionally vague, allowing me to paint a broad stroke of every thought that occurs to me when I read the question – isn’t that the point of reflection anyway?

I write this piece on Day 7 of self-isolation amidst the corona outbreak in India. The situation is obviously grim and I have too much time on my hands now anyway. I hope to use this template of self-reflection every now & then to keep in touch with myself and everything important to me. 

The hope is, as a person spending a lot of time with myself indoors for the next couple of weeks to come, I end up writing more than I did the past one year by using these questions as my prompts. I don’t have to start on a blank screen staring at me anymore, I have these 15 questions nudging me to indulge in my thoughts, nurture them and hopefully grow. If we all don’t bring our best versions out at a time of crisis like this, then as a human race in totality, what have we been preparing for, anyway?

“CURE” the “Doori”?

Last week, I watched Gully Boy. The song “Doori” strongly resonated with me. Was it the filming, the acting or the lyrics, I’ll never know. I had a “woke” moment at the time and was left pondering over the concept of “Doori. “Doori” means distance but in the song means “inequality” and is touchingly portrayed.

Now, let me warn you, this piece doesn’t tell you something you haven’t read/heard about in media in the past. But it simply tries to reinforce some key points that somehow seems to escape the decision makers.

So, let me take you through my train of thought after the song stopped playing

1)      Should a country like India worry about inequality? – Hot topic in the global media is “Inequality”. Questions on whether the rich must be taxed more and how the marginal tax rates should be increased are the never-ending ongoing debates which I’ve lost track of. But does India need to think about it right now? NOT AT ALL. We don’t have the luxury or the time to do that. Get rich and get rich fast – should be the focus of the state. Increase the size of the pie by making opportunities available.

2)      How to get rich fast? – By learning a lesson or 2 from China. Chinese economic reforms brought the poverty down from 88% 4 decades ago to 6% in 2 years ago. This massive downfall in poverty levels was achieved due to many experimental policies and by primarily opening the country’s economy globally. I’m clearly oversimplifying here and I’m obviously not qualified enough to handle a question of this magnitude. However, I know we have a template. A template we can learn from. A template tried and tested. A template, that is China.

3)      CURE the problem: If there’s one thing I’ve learned after being a part of corporate, reading books on effective communication – it is to come up with quirky, easy to remember acronyms!

So, I came up with one – CURE: Collect, Understand, Revise, Execute

Collect Quality data: Due to the lack of good quality data, India struggles to get a handle on how serious certain issues are. For ex: From fudging reports by NSSO (allegedly) to outright denying it, the current government is failing to do enough about the job crisis at hand. The demographic dividend is now a ticking bomb and no more an asset that we posed it to be, globally. The inconsistency in the stories, datasets of NSSO and EPFO tell us, is all the more a reason why we need more serious discussions and actions on collection of more data. NSSO data is collected once every 5 years, Census data – once every 10 years, the labour bureau’s employment surveys last happened in 2015-16. CMIE (Center for Monitoring Indian Economy) happens every year. But higher frequency, better quality data means better analysis, right?

Understanding the gaps: Again – oversimplifying this, but analyzing data from various sources will lead to a better understanding about the variables at play that create or amplify the problem. This will help us distinguish what needs attention from what could be allowed to take a back seat.

Revise the laws: Labour laws in India have been a roadblock in creating more feasible employment opportunities over the years. Lack of medium size enterprises increase the pressure on large size enterprises to employ people in high numbers that causes unintended imbalances. Though this sector has been liberalized, there is scope for a lot more and there are detailed analyses carried out on this subject which are all linked towards the end of this article that I have read in the past to fairly understand how deep rooted this issue really is.

Evaluate and Execute schemes: There have been a lot of discussions on “Minimum income guarantee” and how all the “poor” would receive money from the government every month to support them for basic food necessities. But I believe the root of the problem is not addressed with schemes like this. Governments currently and in the past have spent abysmal amount of money on subsidies and loan waivers when the root cause for increasing job crisis is rapid urbanization and asymmetry of information between the demand in skills and the supply of jobs. People employed in agriculture are also quickly moving away, seeking jobs in cities. Integration of this population in cities and their transition to non-agricultural jobs should be made smooth – this must be the aim of any scheme that government of today plans to introduce.

I’m not sure if “Doori” will go away any time soon, but we have seen examples backed with enough evidence that it’s possible to lift people out of poverty. We only need to pay attention and believe in incremental measures to “CURE” this problem.

References:

https://www.washingtonpost.com/world/2019/02/01/indias-job-crisis-is-worse-than-people-thought-its-government-tried-squelch-data/?utm_term=.7fd55731ea75

https://www.businessinsider.com/mark-zuckerberg-read-this-book-to-understand-china-2017-6#the-restructuring-of-chinas-oil-sector-shows-that-the-countrys-economic-ascent-wasnt-entirely-smooth-4

https://www.abc.net.au/news/2018-12-01/40-years-of-reform-that-transformed-china-into-a-superpower/10573468

https://www.livemint.com/Industry/PpHdVFJRlhRamo50DZRIiJ/Six-ways-to-turn-around-the-India-job-crisis.html

How do you define a “good” teacher?

Measuring teacher’s effectiveness has been one of the most challenging, yet important issue that governments, private and public schools across the world is trying to tackle.

The Karnataka State’s department of primary and secondary education has prepared personalized report cards for teachers in government and aided schools based on the SSLC (Secondary School Leaving Certificate or the 10^th exams) that was conducted in May 2018. This report card is based on how the class fared in board exams and would be given a score between 0 to 10, with 10 being the highest. They would also be categorized into 4 slots: Very good, good, average & below average. This evaluation measure also considers the number of students taught during the year, the pass percentage & the average marks of the class. Now this opens the room for a whole new debate – is this the right evaluation technique? Is correlating the class of students’ performance in the final board exam to the teachers’ effectiveness the right approach?

Graduating from “Access to Education” to “Access to Quality Education”

Without digressing into how the Indian Education system has been in shambles since 1947 and how the bad policies are throttling the system year on year, we know that the general narrative has now shifted to access to good education and not simply education anymore. Indian families that send all their children to private schools spend over 4 percent of their total expenditure more on education than a similar household which sends all its children on public school. It’s a 60% increase above the average education expenditure and this is financed by a reduction of expenditure on food, healthcare and entertainment among others.[1] The rhetoric around education claims that it is the best human capital investment individuals and government can make. Thus, it becomes important to know what the return on education is, for the individuals to make better informed choices, increasing the individual freedom and empowering them. Most families believe that Private Schools have a better return on investment (quality of education) compared to government schools.[2]

Teacher’s role in a student’s success

Studies have shown that teachers matter more to student achievement than any other aspect of schooling.[3] While external factors like the family circumstances or personal characteristics have higher impact on the student achievement, most policies focus on improving teacher’s effectiveness since it’s easier to control. Measuring teacher’s impact has been a bone of contention for several years across education specialists, since measuring “impact” has been a tricky business. Karnataka government’s plan to release a “Teacher Report Card” is yet another attempt to jump into this business, but this method is utterly flawed. Why? Well,

a)       Students start out at different levels, so growth in their scores over a period is a better basis for judging teacher effectiveness.

b)      Studies have shown that external factors like health, neighborhood, family have between 4 to 8 times more influence on student’s scores.

c)       If a student had more than one teacher in a single subject that year, it would be hard to zero in on the teacher’s contribution to the student’s learning.

d)      Important skills can be underrepresented in or excluded in board exams. Final board exams do not fully assess the student’s grasp of higher order skills, interest in subject, academic curiosity which are essential components that decide the fate of the student’s academic path.

e)      Unintended consequence: This would not reflect the actual ground impact and could demoralize the teachers, creating more harm than good.

There are several other techniques such as Value-Added Model, Student Growth Percentile techniques used to measure the impact. They’re highly data driven methods and though not perfect, they pave way to better understanding the role of the teacher in the student’s learning outcome.

Value Added Modeling:

VAMs capture the teacher’s effectiveness over a period, thus holding the teacher accountable for consistent performance. This means focusing not on how students test at a single point in time but rather on how much improvement they make from one testing period to the next.

The model works as follows:

If a teacher A teaches grade 6 math, statisticians obtain grade 4 & 5 test scores of all her students, as well as background information. Using this historical data, prediction of grade 6 math scores will be made for each of the student. If student A, scores 5 points more than the predicted score and student B, scores 2 points below the predicted score, Teacher A’s value-added estimate will be average of the difference between the actual & predicted score of the students.

Since VAMs consider the previous test scores and the background of the student, one teacher’s value-added estimate can be compared to another and be ranked accordingly. (teachers teaching the same subject, same grade).This method is largely used by several states in the US and is also actively being adopted by states during recent times.[4]

Conclusion:

Teaching is a complex activity and cannot have just a single method used to measure the effectiveness. Using multiple measures to assess the many aspects of teaching effectiveness as completely as possible and providing teachers with detailed information to help them understand their strengths and weaknesses would help in building knowledge about best practices across the teaching community. Karnataka state department’s decision at building a teacher’s report card to build a culture of sharing best practices among teachers is a welcome move, but the means to develop such a report card must be rethought and carefully analyzed.

---

[1] http://www.newindianexpress.com/nation/2018/aug/25/families-cut-food-entertainment-expenses-to-send-kids-to-private-schools-research-1862728.html

[2] http://2010.economicsofeducation.com/user/pdfsesiones/042.pdf

[3] https://www.rand.org/education/projects/measuring-teacher-effectiveness/teachers-matter.html

[4]https://www.rand.org/education-and-labor/projects/measuring-teacher-effectiveness/value-added-modeling.html 

Is this the kind of planet we want to leave behind?

Latest UN study says we have 12 years to prevent the world from an environmental catastrophe

11 months ago, Om, my nephew was born. Almost every morning since then, I spend at least a good ten minutes staring at his pictures & videos, where he innocently tries to explore his surroundings and learn more about the little world around him. The past 11 months have swooshed by and I can’t get enough of him and the rate at which he’s growing. But the past couple of days have been disconcerting. A few days ago, I woke up in the morning to read an article about the latest UN IPCC (Intergovernmental Panel on climate Change) report. Since then, I can’t stop beating myself about the sort of planet you and I, together, are leaving behind for Om and his other friends. I cannot stop obsessing about the impending danger this brings to everyone and everything we love. I cannot seem to wrap my head around the scale of this problem. So, this post is my attempt to understanding what this means for India.

IPCC report & its takeaways

Intergovernmental Panel on climate Change, a UN committee says the Paris Climate agreement, made in 2015 – to reduce greenhouse emissions, maintaining the global warming from rising more than 1.5 degrees Celsius is simply not enough. It says, we would require a 40-50% of reductions in emissions by 2030. This may seem impossible, but the report leaves us with hope. It talks at length about ways in which we can cap this situation through use of technology.

India: The biggest loser

Social cost of carbon (SCC) is a policy tool which is being used to attach a monetary value to the societal damages caused by the emission of every ton of carbon. It captures the loss of productivity due to extreme heat, loss of agricultural produce and other outcomes which impact the lives of people negatively. Though an imperfect tool, it helps us in gauging the costs and benefits and the way we need to adopt to fight climate change.

Research shows that India loses the most in the world due to climate change – up to $86 for every ton of carbon emission. This means that India loses $210 billion annually at current emission rates. US is behind, losing $48 per ton. With so much to lose, we cannot sit on our hands waiting for the disaster to strike. Another interesting find is that, how climate change results in negative social cost of carbon for countries like Russia, i.e they gain as the world gets hotter. Some countries pay a higher economic cost than the rest. But one ton of carbon emission contributes equally to climate change regardless of where it comes from. Thus, as countries come together to set regulations to check the damages caused, they should all be equally responsible to share the load, without just focusing on the SCC of their own.

 

Collective conscience: The only way to make the world a better place to live

India emits 6% of the global greenhouse gases and will bear almost 20% of the global economic burden because of climate change. Most wealthy countries are almost balanced in terms of the high SCC and its greenhouse gas emissions. This sort of injustice can only be resolved by ensuring India does its best in cutting its carbon footprint through many ways which have been elaborately discussed by scientists in the past (also beyond the scope of this article). It’s also time we sit up and start pushing our neighbours and other wealthy countries to bring down their emissions, which is what Paris climate agreement attempts to do. Though some of the wealthy nations have very little incentive to do so economically, morally, all the countries owe this to our future generations. 

It is immoral to leave a place worse off than the one we grew up in. It is nearly impossible for ordinary people like us to envisage the sort of changes a 2 degree increase in climate can do, because we take so much for granted. We are so used to the status quo, we are ignorant and in denial about the impact it can make to our lives, in less than 15 years! Though we all talk about how hot the world is getting, there is nearly nothing we are doing on a day-to-day basis to change our practices. This is exactly what scares me. If we do not change our lifestyles and adopt the idea of being environmentally more conscious, how do we even hope this problem to get any smaller? I hope to see a day where everyone around us lead their everyday lives trying to make their surrounding a cleaner, healthier place to live in. Till then, I guess this will continue to trouble me.

References:

https://www.vox.com/2018/9/26/17897614/climate-change-social-cost-carbon

https://insideclimatenews.org/news/24092018/climate-change-economic-damage-america-social-cost-carbon-china-india-russia

https://19january2017snapshot.epa.gov/climatechange/social-cost-carbon_.html

"Parks & Recreation" shows us how to fix our education system

How Leslie Knope’s ideas of management can change the way our Indian higher education system operates

A hearty shout out to all the Parks & Rec fans out there! I’m sure everyone who has watched the awesome TV show, would agree with me on this -beneath all the silly, sometimes-mindless comedy, it is a show that leaves you thinking about how to lead a good life filled with passion, goals and just pure zest. Every time I watch it, I’m left thinking about it with an entirely new perspective. This time, I figured there were so many simple, yet powerful messages a few episodes sent across and they strongly resonated with me.

So, this post is my attempt at breaking down the different elements of what makes this show so fantastic & how we can think about how implementing some of these principles into our education system. Why education, you ask? Well, because it needs some heavy-lifting now more than ever & it’s high time we address it. 

“Never half-ass two things, whole-ass one thing”

 Ron Swanson gives Leslie this piece of advice when she finds herself juggling between various assignments, her work with the Parks & Rec department & running for city council. 

Lesson for the education system: #1

The Central government & the State government are always locking horns about funding, quality, access of education. They are hardly ever on the same page about what the priority is.

Education, a subject of national importance has been a concurrent subject since 1950. Higher education is a subject of shared responsibility of the State and the Centre. But they have always exercised these responsibilities in different areas. While the Central government paid attention to quality higher education, the State governments have stressed upon expansion of access rather than on quality. Producing less expensive and low-quality higher education institutes to increase access have been proven to be more politically rewarding for the State governments. Due to budgetary constraints, coupled with popular pressures, states push for expansion of higher education institutions. The asymmetry of the federal system has led to a constant friction between the Centre and the State about the utilization of funds and misplaced priorities. While both quantity & quality are of equal importance for a fast-growing education sector like ours, it is of paramount importance for both the entities to work to prioritize, plan & execute.

Meaningful partnership. Always.

Almost every single episode in the show talks about team effort, identifying strengths & simply the joy that comes out of working together.

Lesson for the education system: #2:

While the Central government complains that states are not doing enough to implement the national programmes such as universal elementary education or allocation of 6% of GDP to education, the state governments complain about how they are strong-handed into implementing uniform schemes, restricting the need for tailoring to the local needs. Although this has brought down the inter-state disparities with respect to per capita expenditure and enrollment ratio, the inequalities remain high. Our constitution strengthens the role of the Central government, despite Education being a concurrent subject. In contrast, the US constitution declares Education to be a state subject. Even in the absence of any constitutional obligation for this purpose, the federal government takes definitive interest in making scholarships & fellowships available in Higher education & for improvement of teacher education. The federal aid is given without any federal control. This is the meaningful partnership which needs to be advocated in case of Indian Higher education system too.

Keeping the community in mind

The show shows us the sort of positive impact people in the local government can make if it is empowered and given the right resources to operate. It shows us how enforcing useful programs/policies such as soda tax, sex education was staunchly opposed, hence proving the need & preferences of the community plays a vital role in creating any program.

Lesson for the education system: #3

A subject such as higher education is highly community-oriented. Research in the US have shown that, policies related to education need to be stitched based on 4 distinct factors – Educational load, ability, effort & achievement. Educational load - denotes the number of children to be educated per 1000 of population. Ability – like the name states, is the ability of the community to support education, measured by their taxable capacity. Effort – It is the percentage of its taxable capacity spent on education. Achievement – it is measured by the standard of education. Thus, the variations are high between villages, cities & states. Thus, blanket education schemes across the country will not yield the intended results. Ideally, the State government must use these metrics to create its own education packages for its districts. The funding provided by the Central government should be carefully calibrated on these metrics, to equalize education across the country. The Central government should only nudge the States to think along these lines by providing achievable targets & incentives to reach the targets. This bottom-up approach empowers the local government & the State government to make their own decisions & navigate towards their end goal at their own pace.

Conclusion

30 million students, 40,760 colleges and 757 universities. Let those numbers sink in for a while. That’s how large the Indian Higher education system is – the third largest in the world. The student population was much lower than 200,000 in 1947. In 6 decades, it has grown more than 150 times. “Explosive growth”, you say? Despite that, only 23.6% of the Indian 18-23 age-group have access to education or choose to pursue higher education. We’re on the path to equalizing education, but we still have a long journey ahead of us. Like Leslie Knope, it is quintessential to never lose sight of the goal and work relentlessly towards it. Parks & Rec gives you a lot more than laughter, it reminds you of your priorities and inspires you to do the right thing! Ought to take a lesson or two from it!

References:
http://www.arvindguptatoys.com/arvindgupta/JPNaik_01.pdf
http://www.nuepa.org/New/download/Publications/Occasional_Paper-50_Prof.JBGTilak.pdf

International Student Assessment, 2021 : Our moment of truth!

2021 is set to mark an end to the decade-long boycott of the Program of International Student Assessment. Why is this more than just another exam taken by students and why should we spend more time and resources preparing ourselves for this?

16,000 students from 400 schools across Himachal Pradesh and Tamil Nadu appeared for PISA (Programme for International  Student Assessment) in 2009. The results? – Not encouraging at all. India was placed 72^nd out of 74 countries that participated. China also a first timer, topped the assessment in math and science with Shanghai schools. The outcome of these results? – India pulled out of the testing blaming the “out of context” questions. But after 9 years, India has finally decided to sign up for PISA testing, hoping to score better than the last time after requesting OECD to conduct PISA like assessment tests across Kendriya Vidyalayas and Navodaya Vidyalayas.

While appearing for this international student assessment is a welcome move, it is important to take this more seriously than we did the last time. There have been quite a few discussions about how the 2009 participation was just a litmus test and how India would have fared better if Shanghai equivalent of India such as Delhi, Mumbai, Bangalore, etc had taken part in PISA. But it is also true that the Indian education system is broken and it’s better late than never to come to terms with that. Complaining about how the test isn’t tailored keeping India’s culture in mind is just cowardice pretending to be logic.

If disturbing revelations from ASER, Educational initiatives year on year about the learning outcomes of students across the country have not been enough for the government and NCERT to sit up and take notice, perhaps an international assessment like PISA could tell us what we already know and bring the changes we are all hoping for a long long time.

What is PISA? What does it measure?

Programme for International  Student Assessment was introduced by Organisation for Economic Cooperation Development’s Center for Educational Research and Innovation in 2000. It mainly focuses on 15-year old students’ reading, mathematics and science literacy every three years. It also includes an optional section for financial literacy and collaborative problem solving. It is used to assess the functional skills that students have acquired at the end of compulsory schooling. It is a good measure to assess to what extent students have attained skills essential for full participation in the society. PISA’s goal has been to evaluate performance of systems, how experts can measure the end-goals of education system and to generate better processes to collect new data across systems.

The results of such international assessments is vital to India’s growth, undoubtedly. This is why:

We have always believed that Indian students are good at math. Even President Obama had mentioned how the American schools were under threat from India. But TN/HP math results of PISA showed us that the best of the states’ students were 24 points behind average American 15 year old. The best performers in TN/HP were 100 points the average child in Singapore and 83 points behind the average Korean student.[1]

Well, I agree TN & HP alone isn’t a fair sample here, but the story the data tells us here is staggering. The major takeway from this “litmus test” should have been how we can use this assessment to our benefit and not shy away from participation in the future – which India failed to do.

PISA – The gold standard

PISA has been used as the gold standard for evaluation of the performance of education system. Its high credibility has led to many important education reforms in the West. PISA results in addition to the public pressure and the media’s attention to the results has prompted countries like Germany, Denmark, Japan to open up the stage to policy reforms in education. Germany after PISA 2000, pushed for intense debate over the then education scenario which led to significant reform measures including coming up with national standards and laying down a better support structure for the disadvantaged students and immigrants. Similarly Japan brought changes to its national assessment practices and important curriculum changes to catch up with the growing world.[2]

The flip side – What to look out for, after entering the PISA playfield

Participating in PISA means no more place to hide. Our education system will be out in the open to be judged and measured against the other OECD nations that sign up for this. Educational outcomes are complicated to be tracked over time. The country report card that PISA produces can put the policy makers under intense pressure to improve the score, incentivizing them to focus on the optics rather than actual, systemic change. Thus, PISA outcomes must only be used as a measure than quantifies complex educational outcomes into simplified metrics that can give us the general sense of trends and the gaps in learning. Using the PISA scores to reduce the percentage of students who are below proficiency Level 2(baseline proficiency level), brainstorming methods to reach 500 PISA points (OECD average) would be the path to take. It’s important not to get carried away by the rank, like we’re doing with the World bank’s ease of doing business global ranking.

The perfect window of opportunity

An external shock such as PISA outcomes can be exactly what India’s struggling educational sector needs now. With this external shock will come the perfect window of opportunity to gather enough people to make noise about this and push for the reforms for a better learning environment and methods. With this, we will have a better system in place to trade best educational practices with other countries. PISA results bring hope, since over the years it has shown that poor performance in school does not automatically follow from a disadvantaged economic background. It also shows that it is possible to achieve strong educational performance over short periods of time. Thus, it is critical that India makes a policy shift from educational inputs to learning outcomes to better prepare the young. PISA would nudge us in the right direction, break us out of our cocoon and put us in our place. We might not like what we see, but that’s the only way to learn.    

---

[1] https://ajayshahblog.blogspot.com/2012/01/first-pisa-results-for-india-end-of.html?_sm_au_=iVVs6N21PPfZ7j4Q

[2] http://simonbreakspear.com/wp-content/uploads/2015/09/Breakspear-PISA-Paper.pdf