Matching Keywords, Partial Keywords, and Keyword Logic

On a conceptual basis, job keywords make sense to everyone.  However, when you dig into the details of how computers identify keywords, many challenges arise.  I am going to explain the logic to clear up the confusion and hopefully start an ongoing discussion in the comments section about keywords.  I will use laymen terms to remove the complexity of understanding computational linguistics.  For those looking for more info on the subject, a good introductory article from Microsoft is here.

Computers Look for Different Keywords than Humans

As for the job keywords, lots of advice suggest focusing on industry keywords and functional keywords.  While this advice is still valid, a new era has emerged where computers are looking for keywords in a résumé before a hiring manager reviews a résumé.

Unlike humans, computers do not try to decipher meaning from individual words (e.g., does “manage” mean managing people or managing products).  Instead, they apply complex mathematical formulas to determine the words and phrases that can precisely and compactly represent the content of the job description.  Then, these phrases are searched for in the résumé.  Based on the search, a complex ranking system is used to compare one candidate’s résumé to anothers’.  Complex?  Yes!

I think a good example of how a computer identifies keywords is to use a sample job description.  Let’s focus on just three lines of the Requirements section:

Requirements:
Bachelors degree in a relevant scientific discipline or equivalent.
At least 2 years of relevant experience as a CRA in the biotech / pharmaceutical industry.
3+ years CRA experience is preferred 
Knowledge of GCP and ICH guidelines

Computers identify keywords by determining how often phrases are used among other job descriptions, then the computer looks for the phrases in a candidate’s résumé and ranks the candidate based on the findings.  To understand the process, we can break it into 4 parts: 1. Computer must identify keyword phrases, 2. Computer must determine frequency of the phrases, or “significance”, 3. computer searches résumé for matching or partially matching keyword phrases, 4. computer assigns a rank based on the matches and partial matches.

1. Identifying the Keyword Phrases

The computers first begin by analyzing the job description to identify all the keyword phrases in the job description.  What is a “phrase”.  A “phrase” is one or more words in succession from the job description.  Phrases can be single words like “CRA” from our example, or longer strings of words like “Bachelors degree in a relevant scientific discipline.”

2. Determine Frequency of the phrases

With the phrases identified, next, the computer identifies how many times that phrase is found in all the other job descriptions.  The more it is found, a higher score is assigned to the phrase.  The less it is found, the lower the score.  For instance, let’s look at the first two lines of our sample job description.

Requirements:
Bachelors degree in a relevant scientific discipline or equivalent.
At least 2 years of relevant experience as a CRA in the biotech / pharmaceutical industry.

If we had 10 other job descriptions and counted the frequency of the phrases, we might end up with something like this:

What the computer does is start with a word and get a count for its frequency (i.e., how many times was it found in all job descriptions). Then, it will add on additional words and get a count.

Once the frequency is determined, then the computer decides what are the keywords for a job.  With the information we have above, we could claim the words that show up less frequently are the most important phrases for this job, and the words that show up more frequently are too generic. For instance, if a phrase appears in 10 job descriptions, we may think this is not important (this is the case with “Bachelors degree”).  However, the phrase “CRA in the biotech pharmaceutical” is very unique to this job.  Therefore, we could assert “any phrase with a count of 3 or less is a keyword phrase”.

But these don’t look like Keywords

The reason it appears to be incomplete phrases or gibberish is due to the added words in the phrase that make it less frequent. The computer is not looking for grammatical or commonly used phrases.  For example, you may believe “2 years of experience” is the keyword but a computer may say “At least 2 years of” is the keyword, because “2 years of experience” shows up in too many job descriptions.

3. Searching Résumés for Keyword Phrases

Once the computer has a set of keyword phrases, next is searches a résumé for the keywords.  If it finds a match or partial match, it will give it a score.  Let’s use “at least 2 years experience” as the keyword phrase.  If the résumé had “I have more than 2 years experience in…”, we would get a partial match with “2 years experience” being the overlap.  Changing tense of a word, and/or adding or removing plurality or possession will result in getting a partial match.  Partial matches are not bad.  It is unlikely any résumé will match the job description exactly without copying it word for word. Therefore, the goal is to eliminate any missing keywords and fill your résumé with matched and partially matched keywords.

4. Assigning a Résumé Rank (i.e., Job Fit Rating or Resumeter Rating)

Once the computer has a list of all the matched and partially matched keywords, the computer assigns a rank or value.  The rank is weighted based on the matches and the frequency of the keyword phrase.  A keyword phrase that is less frequently found will get a higher weight than a keyword phrase that is more frequently found.  An exact match will get a higher weight than a partial match.  Within the partial match, the closer to the exact phrase you can get, the higher the rank.  The computer takes all of these into account and assigns a weighting.

For every résumé that comes in, a rating can be assigned.

Does it work?

Using our example, let’s do a simple test to see if the process works.  Let’s assume we get hundreds of résumés. If 10 résumés have “CRA” or “CRA in the biotech” versus 100 résumés that have “bachelors degree”, a hiring manager could quickly narrow the applicant list to just 10.  While the hiring manager may miss out on a strong candidate who does not have this term, they do avoid having to read through hundreds of résumés.   There are plenty of arguments on why this may not result in the best hiring decisions, but in today’s economy where employees are required to do more with less, these systems are here to stay.

Technology: Foe or Friend?

Resume keywords can fool your eye

Can you identify keywords better than a computer? Try it here.

Job seekers and experts in the field are familiar with résumé keywords.  However, identifying keywords to make your résumé standout is not as easy as you think.

Let’s look at an example. If you are applying for a sales manager position, commonly held views are to include “sales” and “manage” in your résumé. However, to quote Brenda Bernstein, a professional résumé writer and owner of TheEssayExpert,

“Pardon me for saying so, but the above advice is 1) rudimentary, 2) a no-brainer and 3) limited in its value. The problem is that 99% of the people applying for a sales manager job are going to have the words “sales” and “manage” in their résumés! Therefore, you will not get higher on any list by including these keywords.”

Brenda’s statements are accurate.  Over 80% of all hiring companies use computers to screen and rank résumés.  These computers have created new hurdles for job seekers.  Therefore, we wanted to evaluate the best ways to identify keywords.

So what is the best way to identify keywords?

The Résumé Keyword Test

We created a test to compare the effectiveness of a professional résumé writer and Preptel’s Resumeter to determine the best method for identifying résumé keywords.

We had each identify the keywords from a job description. We added the job keywords to a résumé and submitted the résumé. The judge of each résumé is the hiring system at a Fortune 500 company. The hiring system, known as an ATS, ranks candidates based on their résumé (learn more about ATS systems).  To keep it fair, we used the same base résumé and only added the keywords.

The Job Description

Below is our test job description.

Clinical Researcher

May monitor study sites for data collection, source data verification, review of regulatory documents/files and drug accountability.

May review and edit documents including protocols, informed consents, case report forms, monitoring plans, edit specifications, abstracts, presentations, manuscripts and clinical study reports.

May write protocols and protocol amendments, with supervision.

Will present at investigator meetings.

May manage CROs and/or contract CRAs with guidance from CPM.

Chair or participate in meetings or conference calls with CROs and multi-disciplinary study team.

Assist in review of data and preparation of safety, interim and final study reports, and resolution of data discrepancies.

Assist in setting and updating study timelines.

Participate in abstract preparation, presentation preparation and manuscript development.

Train clinical site staff to ensure protocol and regulatory compliance.

May assist in training CRA I and CRA Assistant

Requirements

Bachelors degree in a relevant scientific discipline or equivalent.

At least 2 years of relevant experience as a CRA in the biotech / pharmaceutical industry.

3+ years CRA experience is preferred 
Knowledge of GCP and ICH guidelines

The Professional Résumé Writer Keywords

We want to thank Norine Dagliano for participating in our test. Norine is a professional résumé writer, certified by the NRWA, and oversees certifications of other résumé writers.

Below are the keywords identified by Norine. We have marked them in yellow highlights.

Clinical Researcher

May monitor study sites for data collection, source data verification, review of regulatory documents/files and drug accountability.

May review and edit documents including protocols, informed consents, case report forms, monitoring plans, edit specifications, abstracts, presentations, manuscripts and clinical study reports.

May write protocols and protocol amendments, with supervision.

Will present at investigator meetings.

May manage CROs and/or contract CRAs with guidance from CPM.

Chair or participate in meetings or conference calls with CROs and multi-disciplinary study team.

Assist in review of data and preparation of safety, interim and final study reports, and resolution of data discrepancies.

Assist in setting and updating study timelines.

Participate in abstract preparation, presentation preparation and manuscript development.

Train clinical site staff to ensure protocol and regulatory compliance.

May assist in training CRA I and CRA Assistant

Requirements

Bachelors degree in a relevant scientific discipline or equivalent.

At least 2 years of relevant experience as a CRA in the biotech / pharmaceutical industry.

3+ years CRA experience is preferred 
Knowledge of GCP and ICH guidelines

The Preptel Resumeter Keywords

Preptel’s Resumeter is a computer-based approach to identifying keywords. Resumeter uses the same approach hiring systems use to identify job keywords.

Below are the keywords identified by Resumeter.

Clinical Researcher

May monitor study sites for data collection, source data verification, review of regulatory documents/files and drug accountability.

May review and edit documents including protocols, informed consents, case report forms, monitoring plans, edit specifications, abstracts, presentations, manuscripts and clinical study reports.

May write protocols and protocol amendments, with supervision.

Will present at investigator meetings.

May manage CROs and/or contract CRAs with guidance from CPM.

Chair or participate in meetings or conference calls with CROs and multi-disciplinary study team.

Assist in review of data and preparation of safety, interim and final study reports, and resolution of data discrepancies.

Assist in setting and updating study timelines.

Participate in abstract preparation, presentation preparation and manuscript development.

Train clinical site staff to ensure protocol and regulatory compliance.

May assist in training CRA I and CRA Assistant

Requirements

Bachelors degree in a relevant scientific discipline or equivalent.

At least 2 years of relevant experience as a CRA in the biotech / pharmaceutical industry.

3+ years CRA experience is preferred 
Knowledge of GCP and ICH guidelines

The Comparison

To compare the job keywords, we show both our Professional Résumé Writer’s and Resumeter’s keywords.

Clinical Researcher

May monitor study sites for data collection, source data verification, review of regulatory documents/files and drug accountability.

May review and edit documents including protocols, informed consents, case report forms, monitoring plans, edit specifications, abstracts, presentations, manuscripts and clinical study reports.

May write protocols and protocol amendments, with supervision.

Will present at investigator meetings.

May manage CROs and/or contract CRAs with guidance from CPM.

Chair or participate in meetings or conference calls with CROs and multi-disciplinary study team.

Assist in review of data and preparation of safety, interim and final study reports, and resolution of data discrepancies.

Assist in setting and updating study timelines.

Participate in abstract preparation, presentation preparation and manuscript development.

Train clinical site staff to ensure protocol and regulatory compliance.

May assist in training CRA I and CRA Assistant

Requirements

Bachelors degree in a relevant scientific discipline or equivalent.

At least 2 years of relevant experience as a CRA in the biotech / pharmaceutical industry.

3+ years CRA experience is preferred 
Knowledge of GCP and ICH guidelines

The Results

As shown above, even though many words were marked as keywords by both approaches, our research shows significant differences in how hiring systems rate résumés based.  To demonstrate this point, let’s see how they ranked.

We added the keywords from our Professional Résumé Writer to a base resume and named the person “Julie Wilson“.

Next, we added the keywords from Resumeter to the same base résumé and named the person “Peter Comp“.

We submitted both résumés for this job.

To see which one ranks higher, we then logged into the hiring system.  For those not familiar with these hiring systems, when a hiring manager or recruiter logs into the hiring system and chooses the job, they see a stacked list of candidates based on Job Fit. Job Fit is the hiring system’s evaluation of how well the résumé fits the job.

Based on the results and screenshots below, Resumeter outscored our Professional Résumé Writer by 34 percentage points. Resumeter scored a 83% job fit rating, and our Professional Résumé Writer scored a 59% job fit rating.

Implications

This test does not diminish the value of a professional résumé writer.  In fact, Professional Résumé Writers are the best way to create your résumé and are experts on industry best practices.

This test and results demonstrate the need for job seekers to 1) tailor their résumé to each and every job, 2) understand the different approaches to identifying keywords.

Most of the advice you may find on the web is outdated.  This test demonstrates the difficulty in identifying keywords and how just a few keyword differences may significantly alter your success.