The importance of IT evaluation is not in doubt as all companies seek a Return on Investment in this modern technology intensive economy. The challenge is how to evaluate IT investment in an effective way to improve performance. Although it is clear that ultimately it is the IT user who creates value with the technology through the work done it has been assumed implicitly in evaluation that IT users could be treated as a homogeneous whole when it comes to IT use and evaluation. Recently, researchers have found that user predispositions play a part in cognitive tasks and since evaluation relies heavily upon cognition the outcome of the evaluation would vary based on user disposition. This mini review traces the models for evaluating IT success and highlights the resurgence of individual differences in IT evaluation research. It selects an area (“Need for Cognition”) that might serve as a useful example for study.
Keywords: Individual differences; IT evaluation; Cognition; Need for Cognition
IT implementations do not a high success rate. Only about 25% are successful according to traditional criteria like being delivered on time and within budget . This has led to the popularization of models of evaluation that may help with understanding reasons for this low performance rate whilst being itself easy to use. The most used model for evaluating IT systems that has emerged so far is the Technology Acceptance Model (TAM) which was introduced by Fred Davis in his PhD thesis in1986 and published as an article in  and in later publications [3,4]. The central ideas of the theory are based on the Theory of Reasoned Action  and the Theory of Planned Behavior . The theory states that external factors influence the user’s perceived ease of use which then influences user’s perceived usefulness which then leads to beliefs and attitudes which translate into behavioral intention and then the use behavior itself. In addition to being popular, it appears to be moderately effective with  claiming that 40% of the system’s use is explained by this model. Later versions of TAM have added more variables to the basic model .
Further,  introduced a Unified Theory of Acceptance and Use of Technology (UTAUT) by incorporating 8 different evaluation models: “the theory of reasoned action, the technology acceptance model, the motivational model, the theory of planned behavior, a model combining the technology acceptance model and the theory of planned behavior, the model of PC utilization, the innovation diffusion theory, and the social cognitive theory”.
which outperforms the eight individual models with an adjusted R2 of 69 percent .
In 1992, DeLone & McLean  put together the DeLone and McLean IS Success Model which is the updated version of the model). Measurement of success is problematic, not least because the notion is ambiguous and unclear. Nevertheless, it is necessary to assess the efficacy and value of a system. The model is based on theoretical and empirical IS research undertaken in the 1970s and1980s. “In the (original) D&M IS Success Model, “systems quality” measures technical success; “information quality” measures semantic success; and “use, user satisfaction, individual impacts,” and “organizational impacts” measure effectiveness success” . “Service Quality” was added to the 2003 version of the model.
Individual impacts lead to organizational impacts; system quality, information quality, service quality lead to intention to use, use, user satisfaction and so on. An indication of the model’s success is that the model has been adopted as a common framework for reporting and comparing IS success or effectiveness research work, so it has helped to make it easier to compare different research findings.
Regarding the role of individual differences in the adoption, use and impact of IT this has been around since the late 1970s. For instance,  proposed three categories of individual-difference variables: demographics, personality, and cognitive style. There
could arguably be more categories - ability (intelligence) and
motivation spring readily to mind – but this was a good starting
list. Since that time, other researchers have picked up on this
strand of research (e.g. [12-15] in particular, make a strong case
for continuing to research this area as they found differences in
frequency and volume of IT use not predicted by TAM.
To select an example of an individual difference for study,
“Need for Cognition” has been defined as the “an individual’s
tendency to engage in and enjoy effortful cognitive activity” .
Put in another way,  regard NFC as “a motivation for cognitive
challenge” [18,19] distinguished NFC from Intelligence but the
NFC scale was correlated with Openness, Emotional Stability,
and goal-oriented traits (Personality traits). However, NFC was
distinct from Personality . Why is NFC important? The reason
is that it determines how much cognitive effort will be expended.
Most importantly it determines the cognitive processing mode
employed and even decides how cognitive attention is allocated
for a task.
To show the diversity of application of the NFC construct,
 lists the domains where NFC has been studied: Health,
false memory, learning and problem solving, ethnic prejudice,
paranormal beliefs, advertising, lucid dreaming intellectual task
performance, decision making and website evaluations.
Since that list, another area where interest seems to be
growing is education where NFC is known to play a part in student
engagement initiatives through its dimensions of curiosity,
perseverance, effort and complex problem solving
“Need for Cognition” was first used by  when the
psychology of needs was in its heyday, especially with the success
of Maslow’s Hierarchy of Needs  being in everyone’s minds at
the time. He defined it as the “individual’s tendency to organize
his experience meaningfully”  However, the main inspiration
for the construct was the achievement motivation construct .
Cohen quickly dropped the name and it was much later in the
early 1980s when it was picked up by  who reconceptualized
it away from early motivation psychologists with their language of
energy and drive reduction. It now took on the form of a trait or
disposition to engage in and enjoy thinking [25,26].
The construct also became part of a larger theoretical
framework, Elaboration Likelihood Model  with applications
in advertising research concerning recall of messages .
identified four dimensions of the NFC construct-the enjoyment
of cognitive stimulation, preference for complexity, commitment
of cognitive effort, and the desire for understanding. Individuals
higher in NFC are more likely to attend to deeper levels of
argument, be less concerned with aesthetics and be likely to use
critical thinking. Early work using the NFC construct in the area
of advertising investigated the processing of advertising messages
and how persuasive they were perceived to be .
The NFC construct was envisaged to be domain free and
was assumed to be generally applicable to various settings and
situations. This does not mean that specific types of NFC cannot
be explored. The connection with IT evaluation is that evaluation
requires the use of cognitive processing and NFC predisposes
IT users towards more elaborate processing (high in NFC) or
simplified heuristic processing (low in NFC).
The general framework for understanding and explaining
the use of NFC is the Elaboration Likelihood Model . High
NFC individuals were likely to use elaborate processing before
forming attitudes. An example given was voting behaviors of high
voters were found to correspond better with their attitudes than
those with low NFC. Transferring this to the IT evaluation task,
dual mode processing  may be used – a central vs peripheral
route where the central involves more elaborate cognition and the
peripheral more intuitive and heuristic cognition. Also, since NFC
is predisposition towards cognitive motivation, IT users with high
NFC are likely to expend more effort in the process than those with
The amount of effort expended in evaluation will also vary
between those with high vs low NFC as well as what attention is
focused upon (central vs peripheral messages) and the level of
critical thinking (higher or lower level of cognition). The above
suggests that IT evaluation could consider more how individual
differences might impact on the design, use, and evaluation of
an IT system rather than considering IT users as a homogeneous
whole. That way, evaluations can be more sensitive to the traits,
behavior and relationships of IT users with the systems they use
and leverage technology better to enhance work performance.