Inter-Rater Reliability of the New Observational Method for Assessing an Exposure to Risk Factors Related to Work-Related Musculoskeletal Disorders (WMSDS)

The Entire Body Risk Assessment (ENBORA) method is a new observational method used to evaluate risk factors linked to Work-related Musculoskeletal Disorders (WMSDs). Hence, this study aims to verify the inter-rater reliability of the ENBORA method. 16 raters were involved in a one day ENBORA training session. All of the participants needed to assess three different tasks using the ENBORA paper checklist. The tasks involved were; lifting from pallet to conveyor for Task A, creels of wire onto spindles for Task B, and bottling line inspection for Task C. The Cohen’s Kappa (K) coefficient and percentage of agreement were calculated. As a result, the inter-rater reliability for Task A has a Cohen’s Kappa value (K) of 0.88, while Task B and Task C achieved K=0.91 which indicated very good agreement. This study found that trained practitioners can reliably use the ENBORA method to assess exposure to risk factors related to WMSDs.


Introduction
Work-related musculoskeletal disorders (WMSDs) fall under the category of musculoskeletal disorders (MSDs) that are caused by occupational exposure which could be the reason for work restriction, work-time loss, and at times work leave [1]. It has been reported that WMSDs can affect a person's quality of life and career with frequent absenteeism at work, increasing work restriction, job hopping, or even incapability compared to the other groups of diseases with a huge economic toll on oneself, the organisation, and the public all together [2]. It is a condition that affects the nerves, tendons, muscles and the body's supporting structures which are triggered by cumulative trauma associated with non-neutral body postures, sustained static postures, repetitive motions, as well as psychosocial factors and individual characteristics [3]. Many job sectors which have contributed to WMSDs include car tyre service centres [4], mould making manufacturing industry [5] and the construction industry [6]. Burdorf and Beek explained that, there are three types of methods to assess exposure to WMSD risk factors namely; subjective judgment (e.g., questionnaire and measurement scale), systematic observation, and direct measurement [7]. Takala et al, claimed that the most common method used by practitioners is the observational method [8]. Checklists which act as assessment toolsfor observational methods are applicable to determine the existence and acuteness of WMSDs risk factors in the workplace. Other than that, the checklists also provide ergonomic specialists that have a stable structure for the purpose of analysing of risk factors in different occupations [9]. Several observational methods for evaluating the exposure of risk factors linked to WMSDs are; Ovako Working Posture Assessment System (OWAS) [10], Posture, Activity, Tools and Handling (PATH) [11], Quick Exposure Check (QEC) [12], Rapid Entire Body Assessment (REBA) [13], Workplace Ergonomic Risk Assessment (WERA) [14], and Individual Risk Assessment (ERIN) [15]. However, the majority of the tools only emphasise on evaluating the physical risk factors of WMSDs specifically in terms of different body postures which do not involve other factors of WMSDs. Other risk factors of WMSDs include psychosocial risk factor, work organisation risk factor, and individual risk factors. The Entire Body Risk Assessment (ENBORA) method is a new observational method for assessing exposure to risk factors related to WMSDs that assesses physical risk factors, psychosocial risk factors, work organisational risk factors and individual risk factors. Reliability is required to prove validity as it ought to be operated on any kind of new measurement method [16]. Reliability is also known as the concord between the object's recurring measurements and the one that has been tested to support the propositioned method [17]. A measurement's reliability implies a thorough and scrupulous measurement that will guarantee consistency through numerous items in the instrument [18]. Reliability of a measure also depicts the stability and regularity of the instrument that determines the concept while assisting in determination of its quality at the same time [18]. The inter-rater reliability has been analysed for the ENBORA method. Inter-rater reliability is defined as the results' level of agreement derived by numbers in monitoring similar work conditions [19]. Streiner et al, acknowledged that inter-rater reliability resembles a tool's capability to come out with consistent outcomes. However, it also depends on the person in charge of the measurement [20]. Hence, this research aims to analyse the inter-rater reliability of the ENBORA method.

Study design
Researchers from Company A conducted the inter-rater reliability analysis which involved 16 participants in a-day ENBORA workshop. This research concentrated on the assessment and investigation of inputs delivered from 16 of the observers. Individually, the observers were also needed to measure distinct job task using the ENBORA method. Among 16 of the observers from Company A, they possess different expertise where ten observers were categorized as executive level group including process engineer, production executive, health, safety and environment (HSE) executive, quality management executive, service executive, parts executive and warehouse assistant manager, while other six observers were categorized as non-executive level including mechanic, administration officer, production operator, production officer, production line leader, and health, safety and environment supervisor. Six of them had been staying in the same position within the company for two to six years. Nevertheless, four of them had been in employment in the same job scope for 11 to 18 years and six observers had been employed in their current position for longer than 20 years.

Data collection
Prior to the execution of the ENBORA method, all participants were required to undergo risk assessment. They were also provided with proper training to execute the method. A 90 minute briefing was given to introduce the and step by step procedure of the method. Additionally, observers were presented with examples of the assessment to familiarise them with the method. There were three different tasks which were assessed by the observers. These tasks were shown through individual video clips which include lifting from pallet to conveyor for Task A, creels of wire onto spindles for Task B, and bottling line inspection for Task C. 30 minutes were spent by the observers to evaluateeach case study. Observers were given a brief talk regarding the purpose of the tool, instructions, scoring system, and terms used in the ENBORA method before the videos were played. After that, the videos were played as the observers assessed every recorded task using the ENBORA method. The total time used to evaluate the three tasks was one and half hours. In the case where any subject failed to finish the evaluation within the given period of time, the video films were shown again to make certain that all observers managed to complete the evaluation on the specific task.

Data analysis
The reliability of the ENBORA method has been assessed using inter-rater reliability. This test aimed to determine the applicability of the ENBORA method in assessing the workplace risks for all sectors and job scopes that involve the entire body. For the analysis of the inter-rater reliability of the ENBORA method, Cohen's Kappa coefficients were calculated for the assessment by 16 observers including the executive and non-executive level team, who evaluated three recorded tasks. Many researchers have used Kappa statistics in posture reliability studies [10,12,14,21]. Knowing that the subject's occupation may have some influence in defining the results, further investigation of the results was performed. If the Cohen's Kappa (K) value are less than 0, it indicates lesser value than change agreement, K=0.01-0.20 is poor agreement, K=0.21-0.40 is fair agreement, while K=0.41-0.60 is moderate agreement, K=0.61-0.80 is a good agreement and if K=0.81-1, it is very good agreement [22]. In order to obtain Cohen's Kappa value, the differences between the values of observed agreement and expected agreement were compared. Observed agreement is the existing agreement, while expected agreement is the agreement that is supposed to occur only by chance [23]. The formula below was used for the inter-rater reliability analysis [22].
Where, po = observed agreement, pe = expected agreement The percentage of agreement was also calculated for the observers group, which included the executive and non-executive level team. The mean, standard deviation (SD) were used to evaluate and analyse the demographic data of the 16 participants in ENBORA training session.

Inter-rater reliability
The results of inter-rater reliability test for Task A until C are shown in Table 2   In general, the observers obtained the highest K value for the assessment of flexion and extension posture compared to bending and rotation posture. Task A's Cohen's Kappa value for neck flexion and extension was 0.93, and K=0.86 for the neck rotation. On the other hand, the K value for shoulder flexion and shoulder extension was 0.93, and K=0.86 for shoulder abduction, adduction and lateral bending. Additionally, the back flexion and extension for Task B recorded the highest value where K=0.93 compared to the K value for back rotation which was 0.86. The K value for bending and rotation posture for Task C was 0.86. However, the K value for flexion posture for the six main body regions was 0.93.
The other physical items including repetition, forceful exertion, contact stress, vibration and task duration show good to very good agreement with K values ranging from 0.77 to 0.93 for Task A until Task C. For psychosocial risk factors, for the K values for Task A ranged between 0.86 to 0.93, and 0.93 for each item for Task B and Task C.For work schedule items, Task A scored a K value of 0.86 while Task B and Task C both obtained a K value of 0.93. The individual items resulted in a moderate to very good agreement of K=0.55-0.86 for Task A whereas Task B and C obtained very good agreement because the value of K was 0.93. The total agreement of the inter-observer reliability for Task A resulted in a high Kappa value of 0.88 with a total agreement of 90%. The total agreement for Task B was K=0.91 with a total agreement of 92.3%. Nevertheless, Task C's total agreement also resulted in a high Kappa value of K=0.91 with a total agreement of 91.8%. The results indicated very good agreement.
Based on the feedback obtained from the observers in the reliability analysis, the observers mentioned that they had difficulty understanding the neck rotation and bending angles due to the ambiguous illustration of the posture while the head position was also a bit confusing. The same problem occurred during the evaluation of the medial rotation and lateral rotation of the shoulders. The observers found the illustrations to be confusing because the view from the head position was difficult to understand. In addition, there are similarities between the inter-rater reliability obtained in this research and the results presented by other researchers which were used to examine body posture. For example, Coenen et al.,had obtained very good agreement for trunk flexion angle, trunk lateral flexion, elbow flexion and shoulder flexion. However, poor agreement was obtained for trunk rotation and shoulder abduction [24]. Meanwhile, Burt and Punnett had discovered that gross body motions including back flexion were easier to observe which will then develop a better agreement compared to smaller motions such as wrist deviation [21].

Conclusion
The main purpose of the Entire Body Risk Assessment (ENBORA) is to provide a quick way to identify exposure to physical risk factors, psychosocial risk factors, work organisational risk factors and individual risk factors associated with work-related musculoskeletal disorders (WMSDs). The results for inter-rater reliability were (K=0.88) for Task A, (K=0.91) for Task B, and (K=0.91) for Task C which showed very good agreement. To conclude, the ENBORA method is a reliable tool which can be applied by well-guided practitioners to assess the exposure to risk factors related to WMSDs.With this tool, it will be easier for employers to recognise and identify the improvements required in their work stations in order to reduce the high occurrence of musculoskeletal disorders and improve employee productivity.