Fixing Health Care at the Source

With application season upon us, graduating medical students across the nation are preparing right now to compete for coveted spots in residency programs, where they will receive their most critical training and become minted as certified practitioners of medicine.

As the controversy over health care funding rages on, we must not forget about the quality of the doctors themselves – and the social implications of our nation’s deficient resident selection process.

We need to pay close attention to how we are selecting MDs for residency – the final and most demanding stage of physician training for two basic reasons. For one, graduating more competent physicians could bolster our ability to make more prudent and cost-effective medical decisions, helping to slow down the explosive growth of health care expenditures in the nearer rather than later future; for two – and more fundamentally – these trainees will eventually be our doctors, providing care for us and those dearest to us at our neediest.

As an educator at a major academic medical center, I will see many medical students apprenticing at my department from now until the end of the fall – peak application season for residency. They will put their best faces forward in hopes of getting an interview invitation to be considered for a spot. But who among them is going to make for an outstanding physician? Given our current selection criteria, it is impossible to tell.

Ironically, despite preaching the virtues of evidence-based medicine, medical residency programs hardly practice such a method of selecting whom they will train. Instead, we look mainly at such factors as deans’ letters, personal statements, letters of reference, grades and, in the absence of objectivity among these, medical licensing exam scores. There is little evidence that any of these predict effective doctoring.

Let’s take a look at several of these individually.

The deans’ letter lays out the student’s performance in coursework. But it tends to exclude vital information not only about whether a student failed a course, or took a leave of absence, but also whether any ethical issues were raised – caught lying or showing up drunk to class.

The personal statement humanizes an application – with information like extra-curriculars, volunteerism, and research – and can substantiate a candidate’s reasons for entering medicine. But these essays can be easily manipulated and ghostwritten to portray a desirable image.

The letters of reference can provide significant information that reaffirms a candidate’s record. But they often are overly one-sided and indistinguishable from one another. In a 2006 study of 736 letters of reference that were received by our department, Dr. Anna Messner and I found not a single one that did not recommend an applicant; furthermore, there was an average of 2.6 standout terms (e.g., “outstanding,” “exceptional,” “excellent”) per letter describing each candidate.

As for the medical licensing exam, it is at best an inconsistent predictor of residency test-taking ability; studies show conflicting results correlating medical licensing exam scores with exams residents take during residency.

And finally there are grades. On a superficial level, they might seem an obvious criterion for determining if a medical student has the basic science and clinical skills to become an effective doctor. But what does a grade of “Honors” mean when 45% of the class is getting it? And how does “Honors” at one school compare to a grade of “A” at another?

Clearly, present criteria reveal nothing about whether a physician will be able to make sound judgments regarding the necessity of exorbitantly costly procedures for the purpose of cost-effectiveness. Nor do they predict a surgeon’s propensity to operate with precision, comfort a patient, or assimilate the latest studies and apply them to practice.

Any residency program director can tell you at least one story about a resident who had an immaculate application who, as a resident trainee, lied to his team, got arrested for drunken driving, or could not even properly address patients by name. Residency interviewers are wary of residency applicants who present themselves impressively during interviews, coming across as assiduous workers with intense scientific curiosity and a passion for patient care. This is because, as residents, one too many has surprised us with lack of skill or pejorative behaviors – arrogance, obsequiousness, chronic tardiness for example – that we did not expect based on the selection criteria.

At the national level, medical competence and professionalism are enough of a concern that the Accreditation Council for Graduate Medical Education (ACGME) has sanctioned rigorous educational standards. To achieve national accreditation, a program must show how it is teaching and assessing the enumerated educational objectives of the ACGME. To graduate a resident – in theory, anyway – a program must be able to demonstrate that a resident has met various competencies encompassing his ability to make ethical decisions, appraise scientific evidence, possesses a breadth of medical knowledge on specific subjects, and provide compassionate patient care.

Training residents, who were chosen on the basis of misleading or unreliable data, is gruelingly complicated, taxing and costly. More compellingly, it is potentially dangerous to the patient population. But an effective selection process, one that distinguishes medical students who have the greatest promise to meet the rigors of residency from those who are likely to fail or flounder, could help to minimize these dicey predicaments.

Some preliminary ideas for improving the current selection process are: implementing standardized grading systems so that all medical schools award the same kinds of grades and do so in the same proportions; using a universal reference form with numerical ratings for particular skills and behaviors instead of free-form letters of reference, and developing simulated patient or surgery exams that are rigorously graded, based on specific criterion.

To be sure, the question of how to chose the most qualified candidates for doctor training will require extended and dedicated study. But the medical community is answerable to society as to why we use the factors we do in minting physicians. So until physician-educators commit themselves to this matter, the current flawed tradition of selecting residents will continue on its complacent path, and we will be guilty of not having tried harder to develop the best physicians we could.

Our current health care crisis illuminates the essential need for compassionate physicians who can make cost-effective, ethical decisions about life and death matters that are informed by the latest studies and research. If we could discover how to weed out weak candidates, or identify those who have the greatest promise to become superb physicians from the beginning of training, we might be able to strengthen our current physician force even more while reducing costs associated with sub-par doctoring.

The nation’s healthcare system is ultimately as good as the quality of our physicians. They – not insurance companies or alternative funding systems – are the pillars of medicine. Investing in how they are chosen could be a very good prescription for the nation’s health.