Research Article

IT Evaluation - are Individual Differences Relevant?

Tuan Yu*

Lecturer in Organizational Management and E-Commerce, University of Kent, United Kingdom

Submission: July 02, 2019; Published: July 17, 2019

*Corresponding author: Tuan Yu, Lecturer in Organizational Management and E-Commerce, Kent Business School, University of Kent, United Kingdom

How to cite this article:Tuan Yu. IT Evaluation - are Individual Differences Relevant?. 2019; 3(5): Ann Soc Sci Manage Stud. 555621. DOI: 10.19080/ASM.2019.03.555621

Abstract

The importance of IT evaluation is not in doubt as all companies seek a Return on Investment in this modern technology intensive economy. The challenge is how to evaluate IT investment in an effective way to improve performance. Although it is clear that ultimately it is the IT user who creates value with the technology through the work done it has been assumed implicitly in evaluation that IT users could be treated as a homogeneous whole when it comes to IT use and evaluation. Recently, researchers have found that user predispositions play a part in cognitive tasks and since evaluation relies heavily upon cognition the outcome of the evaluation would vary based on user disposition. This mini review traces the models for evaluating IT success and highlights the resurgence of individual differences in IT evaluation research. It selects an area (“Need for Cognition”) that might serve as a useful example for study.

Keywords: Individual differences; IT evaluation; Cognition; Need for Cognition

Mini Review

IT implementations do not a high success rate. Only about 25% are successful according to traditional criteria like being delivered on time and within budget [1]. This has led to the popularization of models of evaluation that may help with understanding reasons for this low performance rate whilst being itself easy to use. The most used model for evaluating IT systems that has emerged so far is the Technology Acceptance Model (TAM) which was introduced by Fred Davis in his PhD thesis in1986 and published as an article in [2] and in later publications [3,4]. The central ideas of the theory are based on the Theory of Reasoned Action [5] and the Theory of Planned Behavior [6]. The theory states that external factors influence the user’s perceived ease of use which then influences user’s perceived usefulness which then leads to beliefs and attitudes which translate into behavioral intention and then the use behavior itself. In addition to being popular, it appears to be moderately effective with [1] claiming that 40% of the system’s use is explained by this model. Later versions of TAM have added more variables to the basic model [7].

Further, [8] introduced a Unified Theory of Acceptance and Use of Technology (UTAUT) by incorporating 8 different evaluation models: “the theory of reasoned action, the technology acceptance model, the motivational model, the theory of planned behavior, a model combining the technology acceptance model and the theory of planned behavior, the model of PC utilization, the innovation diffusion theory, and the social cognitive theory”. which outperforms the eight individual models with an adjusted R2 of 69 percent [8].

In 1992, DeLone & McLean [9] put together the DeLone and McLean IS Success Model which is the updated version of the model). Measurement of success is problematic, not least because the notion is ambiguous and unclear. Nevertheless, it is necessary to assess the efficacy and value of a system. The model is based on theoretical and empirical IS research undertaken in the 1970s and1980s. “In the (original) D&M IS Success Model, “systems quality” measures technical success; “information quality” measures semantic success; and “use, user satisfaction, individual impacts,” and “organizational impacts” measure effectiveness success” [10]. “Service Quality” was added to the 2003 version of the model.

Individual impacts lead to organizational impacts; system quality, information quality, service quality lead to intention to use, use, user satisfaction and so on. An indication of the model’s success is that the model has been adopted as a common framework for reporting and comparing IS success or effectiveness research work, so it has helped to make it easier to compare different research findings.

Regarding the role of individual differences in the adoption, use and impact of IT this has been around since the late 1970s. For instance, [11] proposed three categories of individual-difference variables: demographics, personality, and cognitive style. There could arguably be more categories - ability (intelligence) and motivation spring readily to mind – but this was a good starting list. Since that time, other researchers have picked up on this strand of research (e.g. [12-15] in particular, make a strong case for continuing to research this area as they found differences in frequency and volume of IT use not predicted by TAM.

To select an example of an individual difference for study, “Need for Cognition” has been defined as the “an individual’s tendency to engage in and enjoy effortful cognitive activity” [16]. Put in another way, [17] regard NFC as “a motivation for cognitive challenge” [18,19] distinguished NFC from Intelligence but the NFC scale was correlated with Openness, Emotional Stability, and goal-oriented traits (Personality traits). However, NFC was distinct from Personality [17]. Why is NFC important? The reason is that it determines how much cognitive effort will be expended. Most importantly it determines the cognitive processing mode employed and even decides how cognitive attention is allocated for a task.

To show the diversity of application of the NFC construct, [17] lists the domains where NFC has been studied: Health, false memory, learning and problem solving, ethnic prejudice, paranormal beliefs, advertising, lucid dreaming intellectual task performance, decision making and website evaluations.

Since that list, another area where interest seems to be growing is education where NFC is known to play a part in student engagement initiatives through its dimensions of curiosity, perseverance, effort and complex problem solving

“Need for Cognition” was first used by [20] when the psychology of needs was in its heyday, especially with the success of Maslow’s Hierarchy of Needs [21] being in everyone’s minds at the time. He defined it as the “individual’s tendency to organize his experience meaningfully” [22] However, the main inspiration for the construct was the achievement motivation construct [23]. Cohen quickly dropped the name and it was much later in the early 1980s when it was picked up by [24] who reconceptualized it away from early motivation psychologists with their language of energy and drive reduction. It now took on the form of a trait or disposition to engage in and enjoy thinking [25,26].

The construct also became part of a larger theoretical framework, Elaboration Likelihood Model [27] with applications in advertising research concerning recall of messages [28]. identified four dimensions of the NFC construct-the enjoyment of cognitive stimulation, preference for complexity, commitment of cognitive effort, and the desire for understanding. Individuals higher in NFC are more likely to attend to deeper levels of argument, be less concerned with aesthetics and be likely to use critical thinking. Early work using the NFC construct in the area of advertising investigated the processing of advertising messages and how persuasive they were perceived to be [27].

The NFC construct was envisaged to be domain free and was assumed to be generally applicable to various settings and situations. This does not mean that specific types of NFC cannot be explored. The connection with IT evaluation is that evaluation requires the use of cognitive processing and NFC predisposes IT users towards more elaborate processing (high in NFC) or simplified heuristic processing (low in NFC).

The general framework for understanding and explaining the use of NFC is the Elaboration Likelihood Model [27]. High NFC individuals were likely to use elaborate processing before forming attitudes. An example given was voting behaviors of high voters were found to correspond better with their attitudes than those with low NFC. Transferring this to the IT evaluation task, dual mode processing [27] may be used – a central vs peripheral route where the central involves more elaborate cognition and the peripheral more intuitive and heuristic cognition. Also, since NFC is predisposition towards cognitive motivation, IT users with high NFC are likely to expend more effort in the process than those with low NFC.

The amount of effort expended in evaluation will also vary between those with high vs low NFC as well as what attention is focused upon (central vs peripheral messages) and the level of critical thinking (higher or lower level of cognition). The above suggests that IT evaluation could consider more how individual differences might impact on the design, use, and evaluation of an IT system rather than considering IT users as a homogeneous whole. That way, evaluations can be more sensitive to the traits, behavior and relationships of IT users with the systems they use and leverage technology better to enhance work performance.

References

1. Paul Legris, John Ingham, Pierre Collerette (2003) Why do People use Information Technology? A Critical Review of the Technology Acceptance Model. Information & management 40(3): 191-204.
2. Fred D Davis (1989) Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13(3): 319-340.
3. Fred D Davis, Richard P Bagozzi, Paul R Warshaw (1989) User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science 35(8): 982-1003.
4. Fred D Davis, Richard P Bagozzi, Paul R Warshaw (1992) Extrinsic and Intrinsic Motivation to Use Computers in the Workplace. Journal of Applied Social Psychology 22(14): 1111-1132.
5. AjzenIcek, FishbeinMartin (1977) Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychological bulletin 84(5): 888-918.
6. Icek Ajzen (1991) The theory of planned behavior. Organizational Behavior and Human DecisionProcesses 50(2): 179-211.
7. Andrew Burton-Jones, A Hubona GS (2006) The mediation of external variables in the technologyacceptance model. Information & Management 43(6): 706-717.
8. Viswanath Venkatesh, Michael G Morris, Gordon B Davis, Fred D Davis (2003) User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly 27(3): 425.
9. DeLone WH, McLean ER (1992) Information Systems Success: The Quest for the Dependent Variable. Information Systems Research 3(1): 60-95.
10. Delone WH (2003) The Delone and Mclean Model of Information Systems Success: A Ten-Year Update. Journal of Management Information Systems 19(4): 9-30.
11. Zmud RW (1979) Individual Differences and MIS Success: A Review of the Empirical LiteratureManagement Science 25(10): 966-979.
12. Magid Igbaria, Saroj Parasuraman (1989) A Path Analytic Study of Individual Characteristics, Computer Anxiety and Attitudes toward Microcomputers. Journal of Management 15(3): 373-388.
13. Ritu Agarwal, Jayesh Prasad (1998) A Conceptual and Operational Definition of Personal Innovativeness in the Domain of Information Technology. Information Systems Research 9(2): 204-215.
14. Deborah Compeau, Christopher A Higgins, Sid Huff (1999) Social Cognitive Theory and Individual Reactions to Computing Technology: A Longitudinal Study. MIS Quarterly23(2): 145.
15. Burton-Jones Andrew, Hubona, Geoffrey S (2005) Individual differences and usage behavior. ACM SIGMIS Database 36(2): 58-77.
16. Cacioppo JT, Petty RE, Feng Kao C (1984) The Efficient Assessment of Need for Cognition. Journal of personality assessment 48(3): 306-307.
17. Adrian Furnham, Jeremy D Thorne (2013) Need for Cognition. Journal of Individual Differences 34(4): 230-240.
18. Fleischhauer M, Enge S, Brocke B, Ullrich J, Strobel A, et al. (2009) Same or Different? Clarifying the Relationship of Need for Cognition to Personality and Intelligence. Personality and Social Psychology Bulletin 36(1): 82-96.
19. Benjamin D Hill, Joshua D Foster, Emily M Elliott, Jill Talley Shelton, Jessica McCain, et al. (2013) Need for cognition is related to higher general intelligence, fluid intelligence, and crystallized intelligence, but not working memory. Journal of Research in Personality 47(1):22-25.
20. Cohen AR, Stotland E, Wolfe DM (1955) An experimental investigation of need for cognition. The Journal of Abnormal and Social Psychology 51(2):291.
21. Maslow A (1970) Motivation and personality. (2^nded), Harper & Row, New York, USA.
22. McClelland DC, Atkinson JW, Clark RA, Lowell EL (1976) The achievement motive. Oxford, Irvington, England.
23. John T Cacioppo, Richard E Petty (1982) The need for cognition. Journal of personality and social psychology 42(1): 116-131.
24. Cacioppo, John T, Petty Richard E, Kao, Chuan Feng Rodriguez, Regina (1986) Central and peripheral routes to persuasion: An individual difference perspective. Journal of personality and social psychology 51(5): 1032-1043.
25. Cacioppo John T, Petty Richard E, Feinstein, Jeffrey A Jarvis, W Blair G (1996) Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological bulletin 119(2): 197-253.
26. Richard E Petty, John T Cacioppo(1986) The Elaboration Likelihood Model of Persuasion. Advances in Experimental Social Psychology 19: 123-205.
27. John T Cacioppo, Richard E Petty (1983) Effects of need for cognition on message evaluation, recall, and persuasion. Journal of personality and social psychology 45(4): 805-818.
28. Kenneth R Lord, Sanjay Putrevu (2006) Exploring the dimensionality of the need for cognition scale. Psychology & Marketing 23(1): 11-34.