Why no one is right about UCLA medical school: Top schools can't settle for *average* grads & Reporters have to analyze data properly
Why no one is right about UCLA medical school: Top schools can't settle for *average* grads & Reporters have to analyze data properly
Failing shelves & average test scores is bad & the Free Beacon didn't prove diversity is the problem
UCLA medical school admissions and test results are being discussed online as a referendum on the schools admission’s program that admits many under-represented minorities— aiming for a diverse class.
Critics say diverse admission policies mean that the students are underprepared for the rigors of being a doctor. Proponents say UCLA is a strong as ever.
Let me walk you through the data & conclusions we can draw.
It all got started with this figure, which was made public by the Washington Free Beacon
It’s hard to read but it shows shelf exam scores by rotation in the first block (1A & 1B?) of 22-23. Red is bad— it means they failed. And there is a ton of red.
Point 1: When students start on clerkships (aka block 1), they get destroyed by the shelf. Passing is a low bar. Failing is unacceptable. This is bad.
Aside 1: The shelf is a fair test of medicine, but not a perfect test. People who pass can still be lousy doctors (for sure!), and a few who fail are good doctors. But the questions are generally fair.
The Washington free beacon article tries to connect this failure to diversity in the class
“So when it came time for the admissions committee to consider one such student in November 2021—a black applicant with grades and test scores far below the UCLA average—some members of the committee felt that this particular candidate, based on the available evidence, was not the best fit for the top-tier medical school, according to two people present for the committee's meeting.
Their reservations were not well-received.
When an admissions officer voiced concern about the candidate, the two people said, the dean of admissions, Jennifer Lucero, exploded in anger.
"Did you not know African-American women are dying at a higher rate than everybody else?" Lucero asked the admissions officer, these people said. The candidate's scores shouldn't matter, she continued, because "we need people like this in the medical school."
I suspect this conversation did occur and this paragraph definitely warrants debate (including whether it is constitution per the new Supreme court ruling), but the article does not actually connect it to the failure rate.
Data are not provided that the reason kids are failing— and the specific kids who are failing. Are they the same kids who got in despite low test scores? And, was it better before?
Did you adjust for everyone doing worse b/c of COVID changes (aka the zoom mentality), and other changes (aka compression of the preclinical curriculum— they did compress it)? Finally, did you adjust for step 1 going pass-fail, which has shifted students priorities away from taking test taking skills? The answer is no.
The Free Beacon article would fail my class on research interpretation because it doesn’t provide actual evidence that this is the cause of the failure rate.
Now, UCLA issues a rebuttal. Basically arguing that our students are not failing at high rates and doing as well as average
Data point #2. UCLA provides recent shelf scores
This is the same plot but for block 3 (not the first block). It shows few kids fail.
Point #2: UCLA is cherry picking data. The original news story naturally only had 1 (leaked) data point, but UCLA has all the data points. It is dishonest to provide a random block. Provide all of the blocks over all of the years. Do not just provide pass and fail rates but percentile distribution of scores. If UCLA wants to rebut the claim that newer classes are as good as past classes, it has to provide better data. I find this rebuttal to be weak.
Data point #3 UCLA provides step 2 CK scores
Here UCLA aims to show that our students do just as well as the *national average* on the step 2 CK test. Another fair, but imperfect test of medicine.
Point #3: It is not sufficient to say that our students score the national average, your students are supposed to score…. spoiler alert…. way better than the national average because you aren’t an average school!
UCLA is ranked number 18 (tied) in the US for schools. Among test takers on Step 2 you also get a huge bolus of international graduates. UCLA should be submitting test takers in the top 5 or 10% of all Step 2 CK takers, to settle for 50th percentile is defeatist.
Imagine if I drafted an NBA team with Lebron, Stef Curry, and Nikola Jokic and then said, see, I was a good coach because we scored *average* for the league. You weren’t supposed to be average. You were supposed to be great. And if you can’t see that you shouldn’t be the coach.
Point #4: None of these data capture the percent of UCLA students who are delaying or deffering exams. These data are all conditioned on *taking the exam*. — a huge limit to the data. We don’t know the scores of people who don’t take the test. So you have to report rates of taking and delaying it.
Point #5: Taking extra years from school off because your students are *scared to take exams* is not precluded by presented data, and should be examined, as it is a hallmark of a failing education system. This is alluded to in the Free Beacon but data not provided by UCLA.
In response to these concerns, the Dean and some supporters of UCLA said, more or less
‘There has been no change to the minimum standards required to graduate from medical school and get into residency. All students still have to pass every shelf exam (with brand new questions written by the NBME on re-take), pass Step 1, and get a competitive score on Step 2.’
Point #6: I find this irritating. The broader claim is that UCLA is not training the caliber of doctors it *ought to train* and that caliber is not merely *able to clear a low hurdle* but should be *literally top 10 percentile doctors*.
UCLA needs to state clearly how it plans to provide a real education to train the best. Shelf scores are imperfect metric, so develop better metrics and show me you crush them. Also, while imperfect, shelf scores aren’t terrible, and getting to draft your dream team but settling for average is not good enough for UCLA. Aspire for more.
Conclusion:
The Washington Free Beacon provides smoke but no fire. It cannot actually connect changes in admission’s policies to a decline in test performance or other measure. It is sad to see this because I think the reporter has at other times done good work. In my opinion, the article is lazy and meant get a rise from supporters who oppose diversity for other reasons.
Whether and how to admit medical students— and how this should be done in accordance with the Supreme court’s ruling— must be debated and discussed. Sadly, this topic is off limits at UCLA and other medical schools, where debate is stiffled on such issues, and disagreement with the UCLA admission program is implicitly unacceptable, as suggested by the Free Beacon. UCLA and other schools need to have open debates and make policies transparent.
Getting to pick any basketball player you want, and then settling for average, or *passing* is a cope out. I would fire any educator who wants their students to be average. I want my students to be excellent. My job is take whatever I am given and make them excellent. Admissions committees have a duty to make sure that the students we take are up for that challenge, but we have a duty to push them to excel.
Personal point: I am concerned with medical education and the leadership in medical school. This is outside the scope of these data, but leadership has no spine in my opinion, capitulates to student requests too easily, has removed all grades from clerkships, engaged in other actions that undermine motivation, has filled the curriculum with non-medical topics, and forgotten the core purpose of medical school— to make good doctors. This is my bias. But I hope that bias doesn’t color my interpretation of the presented data because I had good teachers in medical school who taught me never to allow that.
UCLA should not dismiss criticism from the Washington Free Beacon merely because it speculates about the political motivations of the author. UCLA must grasp with the core questions: What makes a good doctor? How can you assess it? How can you pick students capable of achieving it? And how can educators develop the courage to push students to be better than they imagined? UCLA can’t abdicate the duty to train the best doctors and celebrate average Step 2 CK scores.
If you aren’t aiming for the best, get out of medical education.
I don’t want you to train my future doctor.
If you want to learn more about my thoughts on curricular reform, read my essays in the topic. Subscribe to this stack for more such posts
This is the state of America today. The right generally has a point, even if not perfectly articulated or well researched. And the left is WAY over the line and covers it up, protects and doubles down on bad policy, and gets their base fired up instead of being introspective. Politics over policies. Unlike the G W Bush era, the left is far more scary and deranged than the right these days
Vinay, I have much of the data to which you refer from two different medical schools (where I sit on/interact with the admissions committee). I have the correlation between "admitted because URM" and failing and it is over 0.7. The remediators in virtually every block are almost all URMs.
No one (especially in committee) will discuss this, but it is well understood. The keepers of this horribleness are those in charge, sadly. They will brook not a word against the narrative.
The premise, as idiotic as it is, is that Indian doctors do better at taking care of Indians, black doctors do better at taking care of blacks, etc. It is 100% wrong. I have yet to meet a doctor who really cared what the race of their patient was (yes, there is probably one somewhere, but not the point); the good doctors do the best for whoever is sitting on the exam table. EVERY race needs the best doctor...not the one that looks like them. Thinking otherwise is both demeaning and stupid and is characteristic of AWFUL people (like the UCLA admissions dean) and their male acolytes.
You could/should gather this information from UCSF. Maybe we could pool numbers and see how prevalent this all is. But everywhere I have looked, the results are the same.