Learn more about the benefits of registering on the new BJS website

Construct and criterion validity testing of the Non‐Technical Skills for Surgeons (NOTSS) behaviour assessment tool using videos of simulated operations. BJS 2018; 105: 719-727.

Published: 30th March 2018

Authors: S. Yule, A. Gupta, D. Gazarian, A. Geraghty, D. S. Smink, J. Beard et al.


Surgeons' non‐technical skills are an important part of surgical performance and surgical education. The most widely adopted assessment tool is the Non‐Technical Skills for Surgeons (NOTSS) behaviour rating system. Psychometric analysis of this tool to date has focused on inter‐rater reliability and feasibility rather than validation.


NOTSS assessments were collected from two groups of consultant/attending surgeons in the UK and USA, who rated behaviours of the lead surgeon during a video‐based simulated crisis scenario after either online or classroom instruction. The process of validation consisted of assessing construct validity, scale reliability and concurrent criterion validity, and undertaking a sensitivity analysis. Central to this was confirmatory factor analysis to evaluate the structure of the NOTSS taxonomy.


Some 255 consultant surgeons participated in the study. The four‐category NOTSS model was found to have robust construct validity evidence, and a superior fit compared with alternative models. Logistic regression and sensitivity analysis revealed that, after adjusting for technical skills, for every 1‐point increase in NOTSS score of the lead surgeon, the odds of having a higher versus lower patient safety score was 2·29 times. The same pattern of results was obtained for a broad mix of surgical specialties (UK) as well as a single discipline (cardiothoracic, USA).


The NOTSS tool can be applied in research and education settings to measure non‐technical skills in a valid and efficient manner.

Full text

Your comments