Another bad CDC study on Long COVID in Kids

If you tweet bad science long enough; I have to conclude you are a bad scientist.

The CDC is doing what they do best: publishing poor quality science in MMWR. And twitter ‘experts’ are doing what they do best: uncritically parroting the results. One of them literally screenshot the study authors limitations passage, as if that were critical appraisal! Let’s take a look at the paper. Starting with the infographic…

Wow, scary stuff!

This is a large database study (running to Jan 2022). The study takes kids who sought care for possible COVID and compares those who had COVID to those who did not. They use more ICD codes to exclude COVID than to include it.

No COVID means no codes for “(A41.89, B34.2, B97.21, B94.8, J12.81, J12.82, J12.89, M30.3, M35.81, U07.1, or U07.2), a positive SARS-CoV-2 test result, or received treatment for COVID-19 (casirivimab/imdevimab, etesevimab/bamlanivimab, sotrovimab, bebtelovimab, nirmatrelvir, molnupiravir, or remdesivir)”

While COVID “equals B97.29 during March–April 2020” or U07.1 code April 2020–November 2021 or + covid test

Problem #1: No-COVID kids also had a lot of COVID. Since the study runs till Jan. 2022, maybe 60-80% of kids who didn’t have COVID, actually had COVID.

As such, it is actually a study of: kids sick enough to see a health care provider for COVID vs. kids who had to see a health care provider because they feared they might have COVID, met someone who had covid, needed a test (for some other purpose like travel) AND had an established doctor in whom that consultation was easy. It is not a study of kids who got COVID and otherwise similar kids who didn’t.

That’s tricky so read it twice.

They don’t look at vaccination status, which leads to Problem #2- claims that vaccines can prevent whatever they find, or this study shows “why kids vaccines are needed” are unjustified. The authors could have explicitly looked at that, but chose not to.

Next, they picked a huge and garbage list of ICD codes that might possible be linked to covid; It would have been great if we knew this was pre-specified and pre-registered & if a list of falsification codes were picked. (does anyone know?); they don’t say. Many of these codes are vague.

The ICD code was linked to the encounter if it came 31-365 days after the index date but not if it was also there 7- 365 days prior. And the index date was roughly when kids had COVID for the COVID patients, but a random claims data from the month in which the non-covid pt was matched to the covid patient for those who did not.

Problem #3 - COVID might be bringing in kids who weren’t getting regular care. Imagine this: Both groups of kids have other medical problems at equal rates, but the ‘COVID’ group has more kids in whom COVID brings them to the doc for the first time (at least in a while), prompting diagnosis of other problems. Control kids have been coming for many visits, and other problems already coded in system. A positive COVID test doesn’t ‘cause’ the other problems but is the event that prompts work up for someone who hasn’t been seen for >365 days.

It turns out this is quite plausible, “Patients without COVID-19 had a higher prevalence of previous hospitalization (4.5%) and complex chronic disease (15.6%), than did patients with COVID-19 (3.6% and 11.7%, respectively)” — More repeat health care user kids are in the without COVID group than the with COVID. Thus, no matter what they find, we cannot be sure if it isn’t just COVID that serves as a reason for a check up, and not a cause of malady.

Next, they build Cox models adjusting for ‘age, sex, race, U.S. Census Bureau region, payor type, previous medical complexity, and previous hospitalization’.

Their main results

• “Patients with COVID-19 were significantly more likely than were those without to develop the following assessed post-COVID symptoms: smell and taste disturbances (aHR = 1.17), circulatory signs and symptoms (1.07), malaise and fatigue (1.05), and musculoskeletal pain (1.02) (Table 2).”

The smell and taste disturbance is lower than what I would expect. COVID does do that.

• “Patients with COVID-19 were also more likely than were those without to develop the following assessed post-COVID conditions: acute pulmonary embolism (2.01), myocarditis and cardiomyopathy (1.99), venous thromboembolic event (1.87), acute and unspecified renal failure (1.32), type 1 diabetes (1.23), coagulation and hemorrhagic disorders (1.18), type 2 diabetes (1.17), and cardiac dysrhythmias (1.16).”

Scary stuff! But, let us just take PE, the absolute rate post COVID was 0.0167% post COVID, and 0.009%, a difference of 0.007%! If it were myocarditis post vax, they would say ‘we have no reports of it, and we specifically looked’ to —- oh, ok, shit, I mean it is ‘rare’ and ‘mostly mild.

• “Patients with COVID-19 were less likely than were those without to experience respiratory signs and symptoms (0.91), symptoms of mental conditions (0.91), sleeping disorders (0.91), neurological conditions (0.94), anxiety and fear-related disorders (0.85), mood disorders (0.78), and muscle disorders (0.94); no significant associations were found for the remaining five symptoms and conditions.”

Holy moly! Problem #4 COVID is curing many ailments of modernity! Less mental and sleeping disorders! Fewer fear related disorders! How could this be? I doubt COVID actually lowers these things (could anyone think that?), instead this reflects that the authors are comparing apples and oranges the whole time. Likely their shitty methods are comparing kids sick enough with COVID who go to the doctor to kids suffering from school closure, disruption in normal social life, and all the other awful restrictions placed on kids.

Now to Problem 5: The groups are not getting the same medical care after the diagnosis. Once someone has a COVID diagnosis, a doctor is likely more likely to look for other medical problems (especially if they read the Atlantic), and more likely to see the patient again and sooner. Also, having added one new billing code, the doctor might be more likely to add more new billing codes. This is why the sensitivity analysis also doesn’t save us. Again, we have no assurance these are comparable groups seeking care for comparable reasons (I feel sick vs. I need a test to go on this trip).

I want to conclude in 2 parts. First, even if this paper were persuasive— it isn’t, but even if some risks were higher, it would do nothing for today’s policy questions

• It doesn’t change our policy of mitigation. The horse is out the barn, nearly all kids have already had it. Masking, distancing, quarantining— these are all unproven, i.e. we don’t know if they ever slowed the spread, and they are especially illogical in a world where the majority of kids already had covid. Also if you don’t generate RCT data in 3 years, you don’t get to mandate anything.

• It doesn’t help the vaccine decision. The paper provides no evidence vaccination would have averted these outcomes, but worse: where is the evidence vaccination lowers the risk of these outcomes for a parent making that choice TOMORROW, where there is a 90+% chance their kid already had COVID?

But more importantly, the paper is not persuasive. It is trash. I still don’t know if a kid who had asymptomatic covid, mild covid, or severe covid has risks greater than not having had covid, or having had a milder version of covid. I am interested in this for biological and scientific reasons, but not so much policy reasons, as I note. If the authors cared about this question, they would need to reimagine the entire study. I think they would want to use seroprevalence to build a cohort and carefully stratify people by severity of illness and work from there. It would take actually work, not just playing with a claims data-set.

But worst of all is just how bad many doctors are at doing science, reading science, and making scientific arguments. We have social media influencers who are in over their head. We have poor thinkers working in all branches of COVID pandemic response (CDC, NIAID and the Czar branch). Politics not science is guiding these publications and decisions. When the dust settles on COVID19, I am not sure the public will have faith in medical science, and things weren’t great going into it. This paper was not worth the time I spent reading it. I would be embarrassed to tweet it out, or reference it in any way. You are best of unfollowing those who do.