Coherence Coefficient of Core Competencies used Predict Job Performance for Medical Employees in a Hospital
Tsai Lin Huang1,2,3, Jung Hua Wu1, Willy Chou4 and Tsair Wei Chien2,5*
1Department of Resources Engineering, National Cheng Kung University, Taiwan
2Department of Hospital and Health Care Administration, Chia Nan University of Pharmacy and Science, Taiwan
3Department of Planning and Management, Chi Mei Medical Center, Taiwan
4Department of Physical Medicine and Rehabilitation, Chi Mei Medical Center, Taiwan
5Research Department of Chi Mei Medical Center, Taiwan
Submission: March 09, 2018; Published: March 27, 2018
*Corresponding author: Tsair Wei Chien, Department of Hospital and Health Care Administration, Chia Nan University of Pharmacy and Science, Taiwan; Email: smile@mail.chimei.org.tw
How to cite this article: Tsai Lin Huang, Jung Hua Wu, Willy Chou, Tsair Wei Chien. Coherence Coefficient of Core Competencies used to Predict Job Performance for Medical Employees in a Hospital. Glob J Add & Rehab Med. 2018; 5(2): 555661. DOI: 10.19080/GJARM.2018.05.555661.
Abstract
Background: To investigate whether coherence coefficients of core competencies can predict job performance for rehabilitation-related therapists in a hospital.
Methods: Rehabilitation-related therapists (n = 133) from 3 hospitals in southern Taiwan were enrolled for this competency survey using the Web Competencies Assessment System developed by 1111 Job Bank in Taipei. A t test was used to examine mean differences between the system's 15 core competencies and demographic variables. Regression analysis was used to inspect the association between the personal coherence coefficient of the core competencies and job performance.
Result: The participant's core competencies were Job Control, Conscientiousness, Executive Ability, Self-Enhancement, Stress Tolerance, Optimism, and Customer Relationships. Their scores were higher than those of the reference group (nurses). The coherence coefficient can significantly predict job performance.
Conclusion: The coherence coefficients of core competencies associated with their own worksite were beneficial for recruiting and hiring new employees, and for career development. The coefficient of core competencies can be used to promote and train future rehabilitation department supervisors.
Keywords: Rehabilitation related therapists; Human resource management; Job competency; Coherence coefficient
Abbreviations: CC: Coherence Coefficient; CI: Confidence Interval; DRG: Diagnosis Related Group; EFA: Exploratory Factor Analysis; IRT: Item Response Theory; IRB: Institutional Review Board; SD: Standard Deviation; VBA: Visual Basic for Applications
Background
Explicit professional certification that someone is qualified for a particular kind of job is not the only requirement considered for recruitment and staffing in a hospital, especially because many qualified job seekers apply for promising positions, particularly at first-rate organizations. Instead, the implicit core competencies, attitude, communication skills, for example, have recently become an important focus for human resource management in hospitals [1]. The implicit features such as selfconcept, personality traits and job motivation are referred to as core competences [2,3]. Medical technicians are as involved in treatment as are doctors and therapists who assist patients with diagnoses and prognoses. The rehabilitation-related technicians (therapists), for instance, help doctors taking care of patients in many fields [3,4]: physical therapists help cure patients with physical methods [5-8], occupational therapists, by designing various functionally effective activities [9,10], and language (speech) therapists, treating swallowing and expression problems [11]. All of them are required to have not only the explicit professional qualification, but also the implicit core competences involved in their mind and service attitude [5,12].
Core competencies were explored early in the history of management studies and widely used human resources managers [13-15] to recruit new employees [16], to implement a standard of workload allocation and personnel development as the basis for the right employee in the right place [17], and to regularize performance evaluations in terms of rewards and promotions [18]. However, there are only two studies on the core competencies of nurses [19,20], and on rehabilitation-related therapists, another major group in the healthcare industry, especially important in this era of a rapidly increasing elderly population in developed countries [21]. In the present study we detected the aberrant behavior of students with a coherence coefficient frequently used in educational and psychometric research [22-25]. We investigated:
a) Whether there is an association between the core competencies and job performance of therapists and
b) Whether it is possible to program a computer routine to identify capable technicians before new ones are recruited and hired.
Methods
Data Source
We recruited 135 therapists from 3 hospitals (A, B, and C). Two of the therapists were excluded because their annual performance scores were missing from the study dataset. The competency website assessment of the so-called Nine Competencies Star system was developed by the 1111 Job Bank [26] and was thus responded to by the participants. The system is commonly used in Taiwan to help job seekers and to help employers recruit employees [27]. More than 400 thousand job seekers have assessed their job competence on the website of the assessment system [26]. Those public norm data (men = accounting for 42.76%; women = 57.24%; mean age: 28.63 ± 6.96 years; 1,287 (69.39%) were 18-30 years old; 434 (23.38%) were 31-40; 112 (6.05%) were 41-50; and 22 (1.18%) were 51) were used as the comparison reference against the sample of the current study [27]. We compared the hospital sample of 1,855 nurses who participated in the same project with our therapists. The study was approved by the Chi-Mei Institutional Review Board (IRB) with a certification.
15 Core Competencies
Sixty-six items of psychological scenarios in the system were assessed and then 15 were included as the core competencies for the respondent. All scenario items without any correct answer were related to each competence consisting of at least 3 items. For instance, when encountering a difficult problem to solve, I will:
a) Avoid it and pretend that nothing happened;
b) Boldly face and solve it with my own knowledge and skills
c) Seek help from others;
d) Prepare a plan to collect data and then solve it.
In addition, the reliability and validity of the assessment were taken into account. Any respondent who took too much time to answer any one item or who skipped an item and then attempted to go back and answer it was excluded from the dataset. The Cronbach's reliability coefficient of the assessed items ranged between 0.70 and 0.79. The validity was verified with a correlation coefficient between 0.69 and 0.78 (p <.001) using two variables of their personal job performance score and their core competence score [26,28]. In this study, we used two variables of personal coherence coefficients and their job performance scores to investigate the validity of the job competence assessment.
The Examinee's Coherence Coefficient
The point biserial correlation (rpbis) was used to evaluate the examinee responding pattern of an assessment [29,30]. It is similar to using exploratory factor analysis (EFA) item loading to evaluate the item correlation coefficient to the factor (or domain) in education and psychometrics, or to using the item response theory (IRT) discrimination parameter to assess the association between a respective item and its factor [31]. Alternatively, one examinee responding a test (similar to the 15 core competence of this study) has a point biserial correlation (called coherence coefficient in the current study and short for CC) to evaluate the relation between the respondents. When the coefficient less than 0.20 is deemed to an aberrant behavior occurred in the assessment [32,33]. The rpbis for an examinee n can be shown with the formula [30] below:
Where L = item length, N = sample size, and X = the observation responses of examinees against the respective item. The higher the value of the coherence coefficient, the greater the response association with others. We designed 2 criterion- referenced scenarios of all examinees' responses (i.e., members in the 3 hospitals, and in the self-hospital) used for regressing coherence coefficients (independent variables) to the job performance scores (dependent variable) to examine whether the job competence assessment between these two variables is valid. The job performance scores (dependent variables) were retrieved from the personal profile of each examinee's annual evaluation by their supervisors.
Selecting a Lower Coherence Coefficient from the Sample
According to the criterion for a residual correlation coefficient less than 0.20 can be unidimensional for a domain factor [34]. The Z-score was computed using the formula: (observed score-the group mean) standard deviation = the deviation from the average. The value |Z|>2 indicates an item with a value significantly higher or significantly lower than expected.
Developing a Module Showing Personal Core Competencies
We designed a computer program to calculate the personal coherence coefficient of each group, and screen out the significantly (p <.05) unexpected responses with a graphical representation and a fit reference for the hospital's Human Resources Department to recruit, select, place, develop, and promote therapists.
Statistical Analyses
Microsoft Excel VBA (Visual Basic for Applications) was used to design a computer program to calculate coherence coefficients and compare them with the group response pattern. MedCalc 9.5.0.0 for Windows [35] was used for single regression analyses of job competencies and performance scores and to create a predictive plot of its trend. ANOVA were used to examine differences across hospitals and demographic groups. T tests were used to compare competence differences between the nurses and therapists of the 3 hospitals.
Results
Descriptive Statistics and Demographic Analyses
One hundred thirty-three therapists participated in the current study: 67 (59.29%) from Hospital A, 34 (30.09%) from Hospital B, and 12 (10.62%) from Hospital C. Their mean age was 35.07 ± 5.44 years old and work tenure was 10.46 ± 6.02 years. For the nurse groups, the mean age was 31.83 ± 6.68 years old and work tenure was 8.15 ± 6.24 years. There were no significant demographic differences in any groups (Table 1).
SD: Standard Deviation,*p <.05
Comparing Core Competencies with the Nurse Sample
Some of the therapists' competencies-Job control, Conscientiousness, Executive ability, Self-enhancement, Adaptability, Stress tolerance, Optimism, and Customer relationships-were, in one hospital at least, significantly superior to those of the nurses' (Table 2). None of the therapists' competencies was inferior to those of the nurses.
Investigating the Validity of the Competence Assessment
The ratio of count number dispersion (at bottom) and relation between the CCs (on vertical axis) and performance scores (at top) across 3 hospitals show a slightly positive correlation (Figure 1). The regression analyses showed that all the CCs but the third in the 1st scenario (Figure 2, upper right- hand panel) have positive effects between the core competencies performance scores (p <.05). The mean CCs were not different and job performance, because there were too few therapists between types of hospitals across 2 scenarios (refers to all 3 in Hospital C (n = 12), can be used for predicting annual job hospitals and a single hospital, respectively) (Figure 3).
Developing a Module Showing Personal Core Competencies
A therapist with a coherence coefficient of -0.06, for instance in Figure 4, presented his scores of all the 15 core competencies in comparison with the mean and 95% CIs of his service department. The core competencies of team cohesion and stress tolerance with Z-scores less than -2.0 were significantly below the section expectation (Table 3), indicated that he did not meet the requirements of the section he was working for.
Discussion
We found that only coherence coefficients yielded by the analysis of a single hospital, rather than of a group of hospitals, can be used for predicting an employee's annual job performance scores (p <.05), and that the computer program designed by the authors can be used as a guideline for recruiting, hiring, placing, developing, and promoting an employee based on the box plot (Figure 3) and (Table 3) showing the personal coherence coefficient and Z-scores across core competencies. We also found that in at least one hospital, score for 8 core competencies:
a) Job control,
b) Conscientiousness,
c) Executive ability,
d) Self-enhancement,
e) Adaptability,
f) Stress tolerance,
g) Optimism and
h) Customer relationships.
Were significantly higher for therapists than for nurses, which indicated they are essential for therapists. This is consistent with reports [9,11,36,37] that core competencies [4 6] are required for therapists in clinical settings. Furthermore, in a clinical setting, therapists usually and frequently deal with patients under with high pressure and tension [8,36]. They have to be able to tolerate stress, and they must be conscientious and optimistic to provide effective treatment and therapy.
The required core competencies are different between front-line therapists, middle-ranking supervisors, and top- level supervisors. The first two focus on skills that enable them to effectively interact with patients, especially interpersonal communication, but the third focuses on policy and solving administrative problems. We also found that the competence patterns of coherence coefficients were somewhat different for therapists in a small study: male therapists had higher Communication skills, Adaptability, and Customer Relationships scores than did female therapists in that study.
Wu et al. [27] investigated the core competencies for technicians and suggested that they be applied to human resource management in hospitals. No studies have researched job competency using coherence coefficients to measure the degree of competency within a section or group. We verified that this is a valid method for evaluating hospital-based therapists and developed a computer program and showed that coherence coefficient can be used to evaluate potential and current hospital employees. Many studies have investigated the relationship between items and the specific domain factor using item-total correlation or exploratory factor analysis. Some have used person fit statistics to detect aberrant response patterns [23,24] but still based their analyses and conclusions on dichotomous and polytomous responses [25]. We applied the coherence coefficients of continuous variables to detect aberrant response patterns of job competences. Additional studies of coherence coefficients to evaluate other kinds of clinical professionals (e.g., doctors, dietitians, social workers, X-ray technicians, pharmacists, etc.) and to investigate the pattern similarities and differences in the structure of medical fees within diagnosis related groups (i.e., so called DRGs in medical settings).
The association between the coherence coefficients and job performance is evident not only within a hospital but also between hospitals. This means that coherence coefficients can be valid and suited for all therapists working in different hospitals if the sample is larger than 30. However, a coherence coefficient generated for one kind of clinical professional cannot be generalized to other kinds of clinical professionals [26].
CI: Confidence Interval; SD: Standard Deviation; Obs.: Observed
Conclusion
We recommend a graphical representation that
a) Compare the coherence coefficients and 95% CIs of work teams,
b) Present comparative individual core competencies, and
c) Display a table with personal coherence coefficient yielded by an analysis of the employee's group.
Acknowledgement
We thank Frank Bill who provided medical writing services on behalf of Meiwin translation Ltd.
References
- Cofsky KM (1993) Critical keys to competency based pay. Comp Benef Rev 25(6): 46-52.
- Rothwell WJ, Lindholm JE (1999) Competency identification, modeling and assessment in the USA. Int J Train Dev 3(2): 90-105.
- Spencer LM, Spencer SM (1993) Competence at work: Model for superior performance. John Wiley & Sons, New York, USA, pp. 384.
- Jennings L, Skovholt TM (1999) The cognitive, emotional, and relational characteristics of master therapists. J Couns Psychol 46(1): 3-11.
- Peloquin SM (1995) The fullness of empathy: reflections and illustrations. Am J Occup Ther 49(1): 24-31.
- Rodger S, Clark M, Banks R (2009) A comparison of international occupational therapy competencies: Implications for Australian standards in the new millennium. Aust Occup Ther J 56(6): 372-383.
- Jette DU, Bacon K, Batty C (2003) Evidence-based practice: Beliefs attitude, knowledge, and behaviors of physical therapists. Phys Ther 83(9): 786-805.
- Rose SJ (1989) Physical therapy diagnosis: Role and function. Phys Ther 69(7): 535-537.
- Prodinger B, Stamm TA (2011) The emergence of occupational science in Austria: An insider perspective. J Occup Sci 19(2): 127-137.
- Yerxa EJ990) An introduction to occupational science, a foundation for occupational therapy in the 21st century. Occup Ther Health Care 6(4): 1-17.
- Zipoli RP Jr, Kennedy M (2005) Evidence-based practice among speech- language pathologists. Am J Speech Lang Pathol 14(3): 208-220.
- Rodriguez D, Patel R, Bright A (2002) Developing competency models to promote integrated human resource practices. Hum Resource Manage 41(3): 309-324.
- Armstrong M (2009) Armstrong's handbook of human resource management practice. (11th edn.), Kogan Page, London, pp. 201-217.
- Lievens F, Sanchez JI, De Corte WD (2004) Easing the inferential leap in competency modeling: The effects of task-related information and subject matter expertise. Pers Psychol 57(4): 881-904.
- Zhu Y, Chen I, Warner M (2000) HRM in Taiwan: an empirical case study. Hum Resource Manage 10(4): 32-44.
- Wanberg CR, Kanfer R, Rotundo M (1999) Unemployed Individuals: motives, job-search competencies, and job-search constraints as predictors of job seeking and reemployment. J Appl Psychol 84(6): 897-910.
- Lieven F, Sanchez JI (2007) Can training improve the quality of inferences made by raters in competency modeling? A quasiexperiment. J Appl Psychol 92(3): 812-819.
- Levenson AR, Van der Stede WA, Cohen SG (2006) Measuring the relationship between managerial competencies and performance. J Manage 32(3): 360-380.
- Andrew S, Gregory L, Cowin LS (2008) Psychometric properties of the Australian nurse competency 2000 standards. Int J Nurs Stud 45(10): 1512-1515.
- Lewis R, Yarker J, Donaldson Feilder E (2010) Using a competency- based approach to identify the management behaviors required to manage workplace stress in nursing: a critical incident study. Int J Nurs Stud 47(3): 307-313.
- Tran D, Hall LM, Davis A (2008) Identification of recruitment and retention strategies for rehabilitation professionals in Ontario, Canada: Results from expert panels. BMC Health Serv Res 8: 249-265.
- Sijtsma K, Molenaar IW (2002) Introduction to Nonparametric Item Response Theory. Quality of Life Research 14(4): 1201-1202.
- Linacre JM (2012) A Comment on the HT Person Fit Statistic. Rasch Meas Trans 26(1): 1358.
- Li MF, Olejnik S (1997) The power of Rasch person-fit statistics in detecting unusual response patterns. Appl Psych Meas 21(3): 215-231.
- Linacre JM (1997) An All Purpose Person Fit Statistic? Rasch Meas Trans 11(3): 582-583.
- 1111 Manpower Bank (2014) The assessment system tool of the nine job competence stars.
- Wu HM, Yuan KS, Su DCJ, Chou W (2013) Investigating the core competencies for rehabilitation related therapists in three hospitals belonging to a medical association in southern Taiwan. Tw J Phys Med Rehabil 41(4): 203-213.
- Schmidt FL, Hunter JE (1998) The validity and utility of selection methods in personnel psychology: practical and theoretical implication of 85 years of research findings. Psychol Bull 124(2): 262-274.
- Donlan TF, Fischer FE (1968) An index of an individual's agreement with group determined item difficulties. Educ Psychol Meas 28: 105113.
- Linacre JM (2014) Correlations: Point-biserial, point-measure, residual. Winsteps User's Guide.
- Wang WC (2010) Recent Developments in Rasch Measurement. The Hong Kong Institute of Education Press, Hong Kong, China.
- Karabatsos G (2003) Comparing the aberrant response detection performance of thirty-six person-fit statistics. Appl Meas in Educ 16(4): 277-298.
- Linacre JM (2012) A Comment on the HT Person Fit Statistic. Rasch Meas Trans 26(1): 1358.
- Bjorner J, Kosinski M, Ware JE (2003) Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the Headache Impact Test (HIT-superTM). Qual Life Res 12: 913-933.
- MedCalc Software (2014) MedCalc for Windows. Version 9.5.0.0 version. Mariakerk, Belgium.
- Adam K, Gibson E, Lyle A (2010) Development of roles for occupational therapists and physiotherapists in work related practice: An Australian perspective. Work 36(3): 263-272.
- Adam K, Gibson E, Strong J (2011) Knowledge, skills and professional behaviours needed for occupational therapists and physiotherapists new to work-related practice. Work 38(4): 309-318.