How do I make sense of application files?

Question

I'm at the stage of my career where people are asking my opinion on applications such as for PhD students etc.

I find it somewhat challenging to sift through everything. I got a stack of 150 or so applicants for a particular program and they come with transcripts, CVs, letters, etc. Many of the letters are filled with superlatives and many GPAs are decently high. It's pretty overwhelming to look at honestly. I know that this task is important so I don't want to do a bad job. But how do you assess applications? How do you deal with the overwhelm?

Answer 1

I have not done admissions work, I only hire people into a range of job positions from student interns to very senior technical staff, so take with a grain of salt.

Experience helps, so it will get better as you do it more, so keep that in mind. Also realize that no selection system is perfect, mistakes of both kinds (accept and reject) will be made but should be fully expected to happen. Learn from them.

Evaluating applicants for a job is different than your situation (except for student interns), but the basic principles apply:

0. What is your role in the process? Review all of them for fit, or find a few that might fit in your area and leave others to find fits with their area?
1. Can you define what you are looking for with some specificity? But don't be so specific that there aren't going to be at least a few in the bunch who might fit. My job postings have Required elements and Desired elements in them - they are the criteria I use to evaluate against.
2. I take the stack of materials and go through quickly to check for the Required elements (often HR does this but I've learned to be light on Required stuff that is technical because, well, they are not in a position to evaluate technical details). This is where administrative folks can really help by passing on only packages that meet the Required stuff.
3. On the first, fast pass I'm also looking for elements that meet the Desired attributes, and put a check mark for each Desired attribute that might be met. Do not go into detail, this is still screening.
4. OK, I now have a spreadsheet. Don't meet Requireds? Out. Don't meet any Desireds? Out. Of the rest, break into 3 roughly equal groups, consider them High, Medium, and Low based on the number of Desired attributes checked off by you. Start with the High pile and review in greater detail, making notes on just what catches your eye about the candidate.
5. That step may already provide you enough folks to consider, but it is still a good idea to go through the Medium and Low groups looking at those Desireds to catch those with one (or more) exceptional characteristics.
6. Final downselect. For one job opening, I only need 2-4 people to interview. You are likely looking for a larger pool (which makes the winnowing down easier). But, I pull about 6 packages out, trying to include at least one person with the one (or few) exceptional attributes. I then dig into whatever else is in the packages to try to reduce to the 2-4 I want to interview.

The point is to iterate through, looking more carefully at fewer people with each iteration. But you first have to be clear just what you are looking for.

Does this take time? Well, yes, but consider that one of the most important things I can do is hire good people. If I do that the rest of my job is really easy. Same for you - get in good students who will do good work and your job is much easier. Time 'saved' on this task causes grief further down the road.

Answer 2

It's a lose-lose-lose proposition: the applicants hate to have to gather all the paperwork and spend time copy pasting word-salad answers from LLMs, those writing letters of reference run out of synonyms for 'outstanding', and nobody wants to go through the pile of applications.

Here's how I deal with it:

1. Randomly pick 5-10 applications and read through them slowly and carefully, making notes. You will immediately start noticing patterns in the applications, which will allow you to place them in separate buckets. For example, the suburban applicant who went to Costa Rica for a week, the ones asking to be selected on pity and not merit, the exaggerators (e.g. 5.2 GPA out of a max of 4.0), the incomplete applications, etc. Once you get a feel for it, you will be able to place them into separate piles after 1-2 minutes each applicant.
2. Some buckets you can reject immediately, e.g. incomplete applications, word salad statements, too low GPA, etc.
3. Come up with short-list criteria. This should be based on the criteria posted on the call for applications, although many programs don't list any criteria at all. Absent objective criteria from the program itself, come up with your own: how would the ideal candidates look? Give the pile a second look based on that criteria. The goal is to shorten the stack to about 2X the size of the amount of spots available, e.g. if the goal is to admit 20 students, make a short list of 40.
4. Go through the short list, reading each application with care and taking notes.
5. Arrive at a your final list.
6. Destroy your notes.