‘SUTRA modellers’ claim of India reaching herd immunity last September was wrong’
The month of April 2021 saw India totally unprepared for the surge in the numbers of novel coronavirus disease (COVID-19) patients. The Union Ministry of Science and Technology’s mathematical model, SUTRA, failed to predict the second wave and grossly underestimated the caseload that the government should have been prepared to deal with.
What went wrong? Gautam Menon, professor with the Departments of Physics and Biology, Ashoka University, Haryana talks to Vibha Varshney and deconstructs the disaster. Edited excerpts:
Vibha Varshney: What has led to this unprecedented second wave of COVID-19 cases in India?
Gautam Menon: The sudden increase in cases in India occurred due to two reasons. First, the appearance of new virus variants that are more infectious. Second, the country began to open up in December and after.
Schools reopened in some parts of the country; crowding in public spaces began to become significant again; political activity began and public transport in high density areas, such as in Mumbai, resumed.
While all these together led to an increase in the number of cases, this sharp rise would likely not have happened in the absence of the variants.
It is impossible to predict something that arises due to such random events. However, we could have been warned by the experience of countries like Brazil and the UK where variants changed the course of the disease dramatically.
VV: We have the Union Ministry of Science and Technology’s mathematical model, SUTRA, to help predict the course of the disease. Why did this not help?
GM: It is impossible to predict a wave of infections in advance, since this really is a random event. However, once it was realised that the wave had begun and the number of cases was increasing more sharply than in the previous wave, there should have been a greater attempt to understand what was going on and to model different scenarios. Those predictive models should also have been readjusted as more data came in.
The ministry’s model, SUTRA, is also not well formulated as an epidemiological model in the first place. The scientists involved are trying to make long-term predictions and appear to have much more confidence in the model than is justified.
They have thus been careless about explaining the many approximations and uncertainties involved. Models cannot make long-term predictions because things change over time. SUTRA scientists have treated the uncertainties associated with modelling in a cavalier manner.
Modelling doesn’t make sense in the abstract. You have to be alert to the increase in cases in different parts of the country. You must know which variants are coming in, what is known about them and how much more are they ransmissible.
The outcomes can change due to the interventions taken to control the pandemic too. If everyone went home and did not step out for four weeks, the disease would stop right in its tracks.
The researchers should have been aware of the potential emergence of variants. These would have shown up as a sharp increase in the number of infected people at a point in time. Genomic sequencing of these patients would have shown that this could be explained by variants.
Once alerted about the increase in cases, the team should have tried to account for this. But, because they do not come from a background of epidemiology, they seem to have consistently underrated this field and were not respectful of its nuances and depth.
VV: Why did independent modellers not intervene and help improve SUTRA?
GM: The original aim of the ministry was to do precisely this — to have a model which many groups could contribute to and which could be improved as a community effort. In fact, officials said the ministry had initiated the exercise to pool in expertise in the field and create one model for the entire country. Such a model would be subjected to rigorous tests required for evidence-based forecasting, routinely practiced in weather forecasting communities.
The officials also said:
The coordination team will consult and work with the research groups active in modelling, various software developers and reputed companies to ensure delivery of a suitable user interface and software.
However, the modellers in this case seem to have ignored this completely and there was no effort from the ministry to set this right.
There is a lack of transparency at the back of all this. We have no idea about the relation between the SUTRA modellers and the government. We do not know what advice they have given to the government and at what time.
There has been criticism of the model in public fora but we do not know if the team is following this criticism. There was no suggestion that independent modellers would be listened to and the government seemed perfectly happy about what they were hearing from the SUTRA modellers.
VV:Why is better modelling needed?
GM: Models are useful as they enable the government to understand what the health system would face during a pandemic — How much medicines need to be bought, how many intensive care unit beds would be needed, etc.
The UK variant led to a spike in cases in the UK too but unlike them, our health care system was not strong enough to handle such a considerable spike in cases post-February. We should have moved earlier and anticipated that the spike would be steep.
Once cases have crossed a certain threshold, there is very little you can do except fire-fight. This is a lesson for what happens when the public health system does not respond fast enough to an ongoing crisis. Now there are many variants in the country. There are many complicated lockdowns in different parts. If you act fast, you can still confine the spread of the infection.
VV: Do we have reliable data on COVID-19 in India to support robust models?
GM: Data is a problem across the world. In India, data is worse but there are advanced statistical methods that can help account for such uncertainty. There are ways to check if the number of deaths and cases are consistent with each other.
But for this, you need a clear understanding of the epidemiology of COVID-19. In India, I would tend to believe the Indian Council of Medical Research (ICMR) sero-surveillance data. I am surprised that the modellers are blaming this for their incorrect predictions.
I should say, however, that the distinction between the numbers from the large number of urban serological surveys may have painted a misleading picture. The disease had likely not spread in rural areas as much as in the cities.
The modellers were wrong to claim that in September / October, 60 per cent people were infected and we had reached herd immunity. This was contrary to the ICMR serological survey results which pointed out that we were far away from such a situation.
VV: Are you also involved in modelling the pandemic? What does your data say?
GM: Yes, we are studying the disease in certain states and cities, providing input to public health authorities and local governments. In general, our take-away is that talking of India as a whole does not make sense as places like Mumbai and Bengaluru.
These two megapolises are doing different things and the course of the disease is different — Bengaluru is in a lockdown, Mumbai is coming out of a lockdown. We work closely with data and we do not project too much in advance.
Based on our model, India will peak in some 10 days, in the middle of May, but we know that there are large uncertainties in this prediction, possibly two-three weeks each way. We will have to redo this calculation on a regular basis as more data comes in.
Modellers should always emphasise the role of uncertainties, assess the quality of input data and account for errors there as well and never claim that models are more accurate than they can be.
VV: How can modelling be improved?
GM: Most good models use age stratification in which the population is divided into age subgroups —10-20 years, 20-30 years. They then study how mortality and infectiousness are different between these groups.
They also incorporate social behaviour in terms of what we understand to be the amounts of contacts between these groups. For example, the fraction of 0-10-year-olds who mix with 20-30-year-olds, how 20-30-year-olds mix with 60-70-year-olds. This social mixing pattern also goes into the model which is important because a 80-year-old is at a far greater risk of dying due to COVID-19 than a 20-year-old.
This has not happened with SUTRA. They represented the whole population as a one homogenous block. People who work on models occasionally do simplify a complicated situation in this manner. But they need to remember that they are simplifying a situation to get a prediction and make this approximation explicit.
The modellers don’t seem to understand the intense current debate around what is the appropriate infection fatality ratio for a young country like India and how it might differ from countries like the UK or Brazil.
They seem to be treating this really as an exercise in fitting data, like engineers or data scientists and not as a problem in epidemiology. They are modellers who believe more in maths than in epidemiology.
Ideally, there should be multiple groups using different models, funneling their predictions into the corridors of government. These can be combined to make ensemble predictions which are better than individual predictions.
Projections need to be made at the granular level, more at district level than for the whole country. There is a lot of noise in the data and most of it is of low quality. Modellers should find a way to deal with these problems. The mistake that the government made was to put all their faith in just one model, not allowing external input to improve the models, nor treating epidemiological inputs with the seriousness they deserved.