Observations of surgeons as a tool to assess competency

Medical doctors receive extensive education and training before and during their time spent working at a hospital or clinic. There are many training sessions organized by groups of medical specialists, hospitals, and professional societies worldwide that doctors can choose from to continue improving their communication and technical skills.

How can we consistently evaluate performance?

Skills and knowledge should be evaluated on a regular basis, shouldn’t they? Many researchers are receiving funding to investigate how skills are best taught and practiced.

Also, effective communication and positive team interactions are becoming increasingly recognized as crucial components for success in the workplace. Studies are investigating the best way for a physician to speak to a patient while simultaneously obtaining the appropriate medical information and putting the patient at ease.

Below you can find a summary of an observational research study conducted by Miskovic et al. (2012) about a new method used to assess the competency level of surgical specialists in advanced laparoscopic surgery.

“Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level”

Recently, Miskovic et al. (2012) collected 32 video recordings of laparoscopic colorectal resections. Laparoscopic surgery is a modern method in which small incisions are made. It is meant to minimize post-operative pain by using smaller incisions and speed up recovery times. The surgeon uses small tools to interact with tissue. Experts and delegates of the National Training Program in England performed all surgeries.

In total, Miskovic et al. evaluated performances of 4 experts and 27 delegates of which 17 passed the National Training Program in England. With this material, Miskovic et al. were able to evaluate a new method called OCHRA that has been developed to detect errors and near misses during surgeries.  

Video analysis is at the core of this method. Miskovic et al. used The Observer XT to code events and calculate frequency and duration. The full coding scheme can be found in Surgical Endoscopy. All videos were analyzed by experts (expert reviews) and by observers using the predefined coding scheme: time spent on Dissecting (D) and on Exposing (E) tissue were measured. And, all errors were marked as point events. If an error led to a direct consequence (i.e. when the patient started bleeding), this was also recorded.

After accurately coding the videos, the complete error count was extracted from The Observer XT for further analysis in SPSS. 

FREE TRIAL: Try The Observer XT yourself!

Request a free trial and see for yourself how easy behavioral research can be!

• Work faster
• Reduce costs
• Get better data

Free Trial

Expert review and behavioral analysis

In this study, two methods were used: expert review and behavioral analysis. Miskovic et al. explain that novel methods are required to further rationalize and objectify the process of measuring competence. Individual assessments of operating videos can easily be combined with detailed behavioral analysis to allow for more accurate assessments. 

Concluding

The researchers concluded that OCHRA is a valid method for assessing performance of surgical specialists. The video analysis delivered the researchers with proof that experts have a higher D/E ratio, in other words work more efficient, in comparison to delegates. Note that it is not the operating time that determined the outcome. It is the combination between time spent on for example cutting tissue and exposing tissue time (i.e. grabbing a different piece of tissue).

OCHRA can now be integrated into clinical training programs. Miskovic et al. emphasize the importance of assessing the competence of surgeons at this level by evaluating real operations in a real environment.

Reference

Miskovic, D.; Ni, M.; Wyles, S.M.; Parvaiz, A.; Hanna, G.B. (2012). Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level. Surgical Endoscopy, 26,796-803.