Intra-and inter-observer reliability of qualitative behaviour assessments of housed sheep in Norway

Abstract

This study tested the reliability of a Qualitative Behavioural Assessment (QBA) protocol developed for the Norwegian Sheep House (FåreBygg) project. The aim was to verify whether QBA scores were consistent between different observers, i.e., inter-observer reliability, and between scorings of the same observers on different time points, i.e., intra-observer reliability. Six trained observers, including two veterinary students, two animal welfare inspectors and two sheep farmers observed sheep in 16 videos, and independently scored 14 pre-defined behavioural descriptors on visual analogue scales (VAS). The procedure was repeated one week after the first scoring session. QBA scores were analysed using Principal Component Analysis. Inter- and intra-observer agreement was assessed using Kendall's coefficient of concordance (W). Principal component 1 (PC 1) and 2 (PC 2) combined explained >60% of the total variation in the QBA scores in both scoring sessions. PC 1 (44.5% in sessions 1 and 2) ranged from the positive descriptors calm, content, relaxed and friendly to the negative descriptors uneasy, vigilant and fearful, and was therefore labelled mood. PC 2 (18% in session 1, 16.6% in session 2) ranged from bright to dejected and apathetic, and was therefore labelled arousal. Kendall's coefficient of concordance of PC 1 for all observers was high in the two scoring sessions (W = 0.87 and 0.85, respectively), indicating good inter-observer reliability. For PC 2, the agreement for all observers was moderate in both video sessions (W = 0.45 and 0.65). The intra-observer agreement was very high for all observers for PC 1 (W > 0.9) except for one, where the agreement was considered to be high (W = 0.89). For PC 2, Kendall's coefficient was very high for the veterinary students and interpreted as moderate for the two farmers and welfare inspectors. This study indicates that the QBA approach and the terms included in the Fårebygg protocol were reliable for assessing video recordings of sheep behaviour when applied by trained observers, regardless of whether they were a veterinary student, animal welfare inspector or sheep farmer. Further work is needed to examine the reliability of the QBA protocol when tested on-farms for sheep managed under Norwegian housing systems.