Research Design for Pilot
The experimental design for the pilot was constructed in a way to minimize any disruption to your current process while still allowing us to answer the question: does an additional prescreen designed to assess interpersonal skills (e.g., interpersonal communications, professionalism, leadership and empathy) using remote, on-demand interviewing and simulations result in greater benefits to your program’s match process and outcomes. In other words, does it improve: 1) efficiency in the process; 2) quality of candidates; and 3) likelihood of matching the candidates you most desire. More about the specific measures later.
Since it is a quality improvement, it does not need IRB approval.
You maintain your screening and selection practices for deciding whom you bring in for a site visit and face-to-face interview. Let’s call the group you bring in your “green group.” You almost certainly had a “yellow group”, a group who were close and basically met your criteria but just didn’t make the cut for the face-face interview, often because you have a limited number of slots. The yellow group is the focus of our pilot.
For the pilot, we are asking that you open a few additional face-to-face interview slots. You will send us a fixed number of your “yellow” candidates. We will then use our remote interviewing/simulation process to evaluate the interpersonal and communication skills, empathy, leadership and professionalism on these “yellow” candidates and screen this group down. We will not evaluate technical skills or knowledge. We will then return a small number of candidates for you to include in your on-site interviews. Our “on-demand” interviews for candidates take about 30 minutes and can be completed anytime, 24-7 through any internet connected device (laptop, table or smart phone). We will provide a window for the candidates to complete the interview, typically on the order of a 1 to 2 week window.
You then agree to fold these additional candidates into your on-site interview process and blindly treat them as any other candidate in the mix.
We envision three primary measures to determine the impact of the prescreen:
The results of the three measures can help answer the question: can a prescreen on interpersonal skills result in fewer on-site, face-to-face interviews while maintaining or improving candidate quality and achieving an enhanced match ratio for your better candidates.
The null hypothesis is, of course, no difference on the three measures.
We will aggregate data across several pilot programs to increase sample sizes.
We would like to limit the pilot to internal medicine, family medicine, pediatrics and emergency medicine programs.
It is not a perfect study and won’t definitively answer all our questions, but as a pilot it will certainly advance our understanding of how to better screen candidates for residency programs. While we all seek great breakthroughs, moving our understanding incrementally forward is the more common outcome. That is often the role of scientific inquiry in practice.
What’s in it for your program to participate in this pilot? In addition to the opportunity to improve the three measures above, there are other benefits:
Thank you for your consideration. If you are interested in participating in a pilot, please submit your email below. There are a limited number of opportunities so we cannot guarantee everyone who is interested will be able to participate in this pilot.