Imagine a candidate describing a useful intervention: asking a learner to predict reversed test order exposes a shared-state misunderstanding behind their proposed timeout fix. The candidate reports that the learner later handled a different fixture, but offers only their account of that application.
In this example, the mentor first uses a small fixture to show how state survives between tests. Rather than supplying the fix, they ask which state needs resetting and why. The later task involves a nested mutable value that survives a shallow copy, rather than the original scalar setting. The learner has syntax documentation and can ask clarifying questions, but receives no diagnosis or suggested fix.
That describes a meaningful application check. Whether it happened, and what it establishes about the mentor, require separate judgments.
Session counts cannot establish teaching judgment. A solved ticket alone cannot establish learning.
For a hiring manager, the decision is whether the evidence supports assigning the candidate responsibility for developing other engineers. For a candidate, the task is to make that judgment possible without exposing someone else’s private work or claiming ownership of their success.
Rate teaching judgment separately from outcome confidence
A polished story with an impressive outcome can overshadow weak teaching reasoning. A thoughtful candidate with limited access to past evidence can be unfairly discounted. Use two records to avoid blending those judgments.
Record one: teaching judgment
For each dimension, record demonstrated, partly demonstrated, or not demonstrated in this sample, with a supporting observation.
| Dimension | Demonstrated looks like |
|---|---|
| Diagnosis | Elicits the learner’s reasoning and distinguishes a learning need from a task or environment problem |
| Intervention choice | Connects the guidance to the diagnosed need and explains why that approach fits |
| Adaptation | Changes the explanation, task, or level of help in response to what the learner shows |
| Application check | Chooses a related but meaningfully different task and identifies what assistance was required |
“Not demonstrated in this sample” leaves room for further evidence. Use the same dimensions across candidates and record observations before discussing an overall recommendation.
Record two: confidence in the reported outcome
These categories describe confidence that the learner applied the skill, not confidence that the mentor caused that application.
- No historical outcome evidence: The assessment uses a simulation, with no account or supporting material about a real learner’s later application.
- Account only: The candidate describes later learning, but no supporting material is available.
- Supported application: An artifact or observation that is safe to share supports what the learner did, with the assistance level explained.
- Repeated application: Evidence shows the learner applying the skill on more than one relevant occasion, with context about other support.
A candidate-created reconstruction can support discussion of teaching judgment; note that status when assessing outcome confidence.
Even repeated application does not isolate the mentor’s contribution from practice, peers, prior knowledge, or other instruction. Promotion is not a clean measure of that contribution either. Continued support also needs context: someone can apply one concept independently while needing help with unfamiliar systems or higher-risk decisions. Appropriate escalation can itself be a useful learning outcome.
A candidate can demonstrate strong judgment in the standard exercise while having no historical outcome evidence. Missing records should not automatically lower the teaching-judgment rating.
Follow one learning need with a mentorship evidence worksheet
Ask candidates to walk through one example that is safe to share. Accept an anonymized account, a candidate-created reconstruction, or an artifact they have permission to share.
| Step | What to record | What the reviewer should look for |
|---|---|---|
| Starting point | The task, relevant prior knowledge, and the learner’s initial explanation or attempt | A specific learning need rather than a label such as “junior” or “weak at debugging” |
| Diagnosis | What the mentor asked, observed, or checked before intervening | A distinction between a knowledge gap, unclear requirements, missing access, and other obstacles |
| Guidance | The explanation, prompt, demonstration, or practice task, including any adjustment | A reason for choosing that intervention and evidence of responding to the learner |
| Application check | A later task, what help remained available, and what the learner actually did | Application beyond repeating the original steps, with the limits of the evidence stated |
In the test example, examine whether the mentor elicited the learner’s reasoning before explaining, addressed the actual misunderstanding, and adjusted the help as understanding developed. Then check whether the later task required applying the idea rather than copying the fix.
The same approach works for a review comment, a pairing session, or a longer development plan. Assess the reasoning behind the intervention rather than its format.
Ask: “What made you change your approach?” and “What would have made you conclude your diagnosis was wrong?” These probe judgment more directly than asking whether the mentoring relationship was successful.
Do not request private performance reviews, confidential code, or identifiable messages from a learner. Sanitization should remove unnecessary personal details, not merely replace a name. If safe sharing is impossible, use a standardized scenario.
Use a standard scenario when past evidence is unavailable
A candidate should not need private learner records to demonstrate teaching judgment. Give candidates the same bounded exercise, with equivalent time, materials, and interviewer behavior.
Reuse the test-suite example with this minimal Python fixture:
settings = {"timeout_ms": 100}
def test_long_timeout():
settings["timeout_ms"] = 500
assert settings["timeout_ms"] == 500
def test_default_timeout():
assert settings["timeout_ms"] == 100Tell candidates that each test passes in a fresh process. Running testlongtimeout() followed by testdefaulttimeout() in the same process fails; reversing them passes. The learner can read the code but believes increasing a timeout will fix the failure.
Ask the candidate to:
- Choose an opening question or diagnostic task.
- Respond to the prepared learner explanation.
- Adjust their guidance after a second prepared response.
- Design a follow-up task and explain what would count as independent application.
After the opening question or task, give every candidate this first response: “I think both orders should pass if the timeout is long enough. Each test starts fresh.” If asked directly about reversing order, use that same response.
Choose the second response using the candidate’s next action:
| Candidate action | Second prepared learner response |
|---|---|
| Explains or demonstrates that the dictionary persists between tests | “Then I’ll reset everything before every test. I’m not sure which value the failing test depends on.” |
| Supplies a reset fix without exploring the learner’s reasoning | “I can copy that reset. I still don’t understand why each test doesn’t start fresh.” |
| Continues questioning without making persistence clear | “The dictionary is created at the top, so I assumed it was created again for each test.” |
If actions overlap, use the first applicable row. Deliver the response without adding hints, then let the candidate adapt. Record what they change and why.
Offer this related fixture when discussing the application check:
defaults = {"labels": []}
def make_config():
return defaults.copy()
def test_add_label():
config = make_config()
config["labels"].append("slow")
assert config["labels"] == ["slow"]
def test_empty_labels():
config = make_config()
assert config["labels"] == []Here, a new outer dictionary still shares the nested list. Ask the candidate what the learner should predict, explain, and change before receiving help. Keep syntax documentation and clarifying questions available, but withhold a diagnosis or fix during the proposed independent attempt. The candidate can suggest another task if they explain its meaningful difference.
Use these observable anchors alongside the four judgment dimensions:
| Dimension | Observable anchor |
|---|---|
| Diagnosis | Demonstrated when the candidate elicits the order prediction and uses it to identify the shared-state misunderstanding. Giving the reset fix immediately leaves diagnosis not demonstrated in this sample. |
| Intervention choice | Demonstrated when the candidate connects a trace, explanation, or practice task to that misunderstanding and explains the choice. |
| Adaptation | Demonstrated when the candidate responds to the second answer by targeting the remaining confusion, such as tracing the relevant value rather than repeating “reset everything.” |
| Application check | Demonstrated when the candidate requires prediction and explanation on a meaningfully different task, states what help remains available, and specifies what learner behavior would support application. |
Score observable choices, not fluency. Use the guide rather than improvising a more cooperative learner for some candidates. Allow reasonable accommodations and keep the exercise focused on mentoring, not obscure syntax or performance under surprise.
The exercise makes instructional choices visible under comparable conditions; it cannot verify learning by a real mentee. Keep that limit in the hiring notes.
For a role that includes mentoring, I would use this structured exercise rather than treat technical depth or a mentoring claim as sufficient evidence. I would not require historical artifacts when sharing them would violate confidentiality.
Start with a five-minute evidence check
Take one mentoring claim from a profile, interview note, or your own experience. Write four short entries: starting point, diagnosis, guidance, and application check. Mark unknowns rather than filling them with favorable assumptions.
Then record one provisional teaching-judgment rating for a dimension, with a supporting observation, and one outcome-evidence category. Ask: “What did the learner do on a different task, and what help did they need?”
Use missing answers to identify follow-up questions. This short check practices the two-record method; it does not support a hiring decision by itself.