Rubrics provide a meaningful way to operationalize and apply the selection criteria in a consistent and reliable manner, and are an effective mechanism to support the equitable evaluation of candidates for faculty positions. They should be tailored to match the specific selection criteria for the search as well as the description of the position in the job advertisement. Transparency helps to ensure a shared understanding among committee members and other reviewers. (Please refer to the search guide for more guidance on developing selection criteria.)
Scoring system
Search committees have found it useful to assign numerical scores to each section of their rubric. This is helpful in identifying and analyzing specific areas of agreement or disagreement as the committee discusses each candidate. The sample template below suggests assigning a range of scores for each section (with a score from 1 to 5 for each), rather than a single score for the entire application (e.g., this file was reviewed and it receives a 3), which can be difficult to defend with evidence (ie., no candidate is uniformly strong or weak in all areas, and relative strengths and weaknesses in one area or another is difficult to score with a single value). Some committees may, however, decide that one section or another should be weighted more heavily. Or, committees may decide that a different scoring system for each section more accurately reflects their departmental or disciplinary needs. What is most important is being intentional and explicit about the use and weighting of the system, having agreement about the evidence reviewers will use to assign the scores, and ensuring a shared understanding of what overall constitutes a particular score for a given category.
Calibration exercise
Calibration exercises assist search committees in using rubrics equitably, consistently, and reliably across all applicants. To best make use of a rubric, we strongly suggest conducting a calibration exercise in advance of reviewing the entire candidate pool. Search committees in the past have found the following calibration exercise useful:
-
Create a rubric for use in the particular search, including selection criteria, scores, and definitions.
-
Discuss ahead of time the kinds of evidence that could motivate low, medium, or high scores.
-
Select a random sample of 5 - 7 statements (or dossiers) from the applicant pool, redacted for candidate name.
-
Apply the rubric to the statements (or dossiers), with each committee member scoring the statements separately.
-
Analyze the scores assigned to each statement (or dossier) across all categories and by all committee members.
-
Discuss interpretations and discrepancies between reviewer scores.
-
Recalibrate the scoring/assessment system as needed.
-
Apply the agreed upon rubric to the entire applicant pool.
Sample rubric
The sample rubric below is a template for search committees to use for assessing candidates. This template should be adapted to specific searches, as appropriate, given departmental or disciplinary expertise.
Topic |
Score |
Criteria |
Definition |
Research |
1 - 5 |
Productivity |
|
1 - 5 |
Alignment |
|
|
1 - 5 |
Plans |
|
|
Teaching & Mentoring |
1 - 5 |
Teaching experience & approach |
|
1 - 5 |
Teaching interests |
|
|
1 - 5 |
Mentoring |
|
|
Service |
1 - 5 |
Experience & track record |
|
1 - 5 |
Plans for contributing |
|
Guidance for senate searches that would be conducted in AY 2025/26 is forthcoming