← TALENT2ROLE

Technical note v0.1 — how the scores are computed

Published 5 September 2026 · engine 2.3.0 · mapping prompt map-v2 · NICE Framework v2.2.0 · item bank items-v1.3. Written so that a reader can reproduce a score, and structured after the headings of the Standards for Educational and Psychological Testing (AERA/APA/NCME) so that it is clear which evidence exists and which does not yet.

PurposeWhat is measuredScoringCareer stagesAI layerConfidence bandConsistency flagsKnowledge checkReliabilityValidityFairnessLimitationsVersions
Plain statement. The readiness score is an evidence-alignment score: a measure of how well a person's self-reported evidence matches the competency statements of a NICE work role. It is not a psychometric test score, and on its own it is not a measure of competence. The knowledge check adds a timed, randomised test of role knowledge; its psychometric statistics will be published as data accumulates (see §9–10).

1. Purpose and intended use

The scores exist to help an individual decide which NICE work roles their evidence points to, which competency statements they are missing, and what to learn next. They are intended for career development and workforce planning. They are not intended or validated for employment selection, and must not be the sole basis of hiring, promotion or termination decisions.

2. What is measured

The framework supplies the construct: each work role is a set of Knowledge (K) and Skill (S) statements (NICE v2.2.0: 42 roles, 693 K and 556 S statements; Task statements are counted but not scored). For a person and a role the engine produces:

Evidence is everything the person enters or imports: headline, résumé text, roles held with descriptions, education, projects and labs, self-listed skills and certifications, and — when they apply them — the structured fields an AI pass extracts from the résumé (§5).

3. Scoring

3.1 Statement matching (deterministic)

Text is lower-cased, tokenised on non-alphanumerics and stripped of stop-words and words shorter than three characters. Each framework statement has a set of distinctive tokens weighted by inverse document frequency over all statements: idf(t) = ln(N ÷ (1 + df(t))) + 1, with unseen terms given 3.2. A statement's hit is the IDF-weighted share of its tokens present in the evidence:

hit(s) = Σ_t∈s idf(t)·[t ∈ evidence] + 0.35·Σ_t∈s idf(t)·[t ∈ expansions]   ÷   Σ_t∈s idf(t)

Expansions come from the rule engine: when a trigger such as SIEM, IR or pentest appears in the evidence, the rule's framework vocabulary is added at 0.35 credit — enough to tip a near-miss, not to create a match. The rule set is admin-editable and versioned (r-…). A statement counts as matched when

hit(s) + 0.10 · w_AI(s) ≥ 0.30

where w_AI(s) ∈ [0, 1] is the weight of a validated AI proposal for that statement (§5), and 0 otherwise. Disputed statements never count.

3.2 Role score

coverage      = matched ÷ total
sizeFactor    = min(1, 0.42 + 0.58·√(total ÷ 95))          // damps coverage on small roles
semantic      = cosine(tf(evidence), tf(role description + statements))
matchedMass   = matched ÷ (matched + 25·bar)                 // saturating count of matches
raw           = 0.35·(covForScore·sizeFactor) + 0.30·semantic + 0.35·matchedMass
score₀        = min(99, round(raw^0.62 · 132))
score         = round(0.5·score₀ + 0.5·stagePct)             // see §4

Why matched mass. A finite résumé can only evidence so many statements, so ratio coverage structurally favours small roles however hard it is corrected; counting absolute matches, saturating, is what stops a 30-statement role out-ranking the role the person actually does. The weights (0.35/0.30/0.35), the size factor and the curve were set by judgement and checked on sample profiles; they are not fitted to outcome data.

4. Career stages

The framework has no levels. Talent2Role adds five stages from years of relevant experience and classifies every statement once as Foundational, Practitioner or Advanced by deterministic rules on its text (admin overrides possible). Each stage carries weights per tier and a bar: the weighted coverage that counts as "ready" at that stage. A role is measured at the higher of the person's stage and the stage the role typically starts at.

StageRelevant yearsWeights F / P / ABar
Entering0–11.0 / 0.6 / 0.20.35
Early career1–31.0 / 0.8 / 0.350.50
Established4–70.8 / 1.0 / 0.60.65
Senior8–120.6 / 1.0 / 1.00.80
Leader13+0.5 / 1.0 / 1.00.90

stagePct = min(99, 100·weightedCoverage ÷ bar), and covForScore = min(1, weightedCoverage ÷ bar). The bars are judgement values. A modified-Angoff standard-setting session with practitioners is the planned replacement (the admin tooling exists); when it runs, the bars and the session record are published here and the engine version bumps. Raw coverage is never adjusted by stage.

5. The AI layer — proposes, never decides

  1. Extraction. On import, the model returns structured fields (skills, knowledge, capabilities, roles) from the résumé text; the person reviews and applies them. Applied fields become evidence like any other text.
  2. Candidates. Only statements in the uncertain band — lexical hit between 0.10 and 0.30 — across the person's strongest roles are sent (≤ 90, sorted by ID, batches of 30, temperature 0, pinned model).
  3. Proposals. The model returns statement IDs with a verbatim quote from the evidence and a confidence.
  4. Validation. A proposal is dropped unless the quote is found verbatim in the evidence (normalised) and the confidence meets the tier's threshold (0.6; 0.75 for Advanced-tier statements). It is confirmed (weight 1) if the quote alone hits the statement at ≥ 0.35 or fires a rule whose distinctive vocabulary (IDF ≥ 4.5) reaches the statement; otherwise it is suggested (weight 0.5). Advanced-tier statements require confirmation.
  5. Credit. The engine adds 0.10 × weight to the lexical hit. Consequently a confirmed proposal can only flip a statement whose own hit is ≥ 0.20, and a suggested one ≥ 0.25. Calibration on the sample profile (2026-09-05): with every candidate confirmed, roles scoring ≥ 80 rose from 15 to 28 at 0.10 versus 34 under the previous binary override, and the top role was preserved; 0.20 reproduced the old override exactly.
  6. Record. Results are cached by content (evidence, candidates, prompt, rule set, ontology, model), so identical inputs return the identical stored result. Before an alignment certificate is issued the mapping is re-run three times and only statements agreed by at least two runs are kept. Every match keeps its quote, rule, scores and prompt version, visible under "why".

6. Confidence band

The band expresses how much, and how verifiable, the evidence is. Its half-width in points is

half = clamp( 17 − 0.018·min(words, 600) − 1.6·verif , 3, 17 )

where words is the evidence word count and verif sums: certifications listed (+1), a LinkedIn profile linked (+1), two or more roles held (+1), projects described (+0.5), structured extraction applied (+0.5), and a passed knowledge check for the role (+3). A 100-word profile with nothing else shows ±15; a 600-word profile with certifications, LinkedIn, roles and a passed check shows ±3. The band is displayed as "72% (64–80)". It is a heuristic uncertainty statement, not a statistical confidence interval, and it is labelled as such.

7. Consistency flags

Deterministic checks mark a result provisional — on the certificate face, the verify page and the registry record — when the claims do not add up:

8. Knowledge check

9. Reliability

10. Validity evidence

TypeStatus
ContentRole statement sets are the framework's own; item stems cite the K statements they test and were checked against the role's statement list. Practitioner review of the tiering rules and item banks: pending.
Response processEvery match is explainable and disputable; quotes and rules are recorded. Think-aloud studies: not done.
Internal structureWeights and curve are judgement-set, checked on one sample profile per edition. Factor or dimensionality analysis: not done.
Relations to other variablesCriterion study (score vs. relevant years, certifications held, knowledge-check score) — the admin analytics compute this once n ≥ 100; no result yet.
ConsequencesIntended use is development, not selection (§1). Misuse safeguards: terms, provisional flags, no title conferred.

11. Fairness

The engine is language-based and currently English-only; evidence written in other languages, or in non-native English, will match fewer statements. Statement matching depends on vocabulary overlap, so profiles written in framework language score higher than equally experienced profiles written in plain language — the rule engine and the AI layer exist to reduce, not remove, that effect. No protected attributes are collected; adverse-impact analysis by region will be run on de-identified aggregates when volumes allow and reported here.

12. Known limitations

13. Versions and reproducibility

Every credential and mapping record carries: engine version, rule-set hash, ontology version, mapping prompt version, item-bank version and the model name. Changing any of them bumps the corresponding version; records issued earlier keep theirs. This note is versioned with the engine. Questions about the method: info@quantumtaskai.com.

See also: Issuing policy · Trust, AI & integrity · Framework attribution