Confirmatory Factor Analysis of the End-User Computing Satisfaction…
Confirmatory Factor Analysis of the End-User Computing Satisfaction Instrument: Replication within an ERP Domain
[Somers, Nelson, Karimi (2003)]
- Orgs make sig investments in ERP (Enterprise Resource Planning)
- Investments in ERP depends on:
- supporting effective use of information technology (IT)
- satisfying IT users (Most important determinant of the success of ERP)
- Sample of 407 end users of ERP systems
- working within the framework of confirmatory factor analysis (CFA),
- Aim: this study examines the structure and dimensionality, and reliability and validity of the end-user computing satisfaction (EUCS) instrument posited by Doll and Torkzadeh (1988).
- In response to Klenke (1992) motion to cross-validate management information system (MIS) instruments and to retest the end user computing satisfaction instrument using new data
- this study’s results, consistent with previous findings:
- confirm that the EUCS instrument maintains its psychometric stability when applied to users of enterprise resource planning application software.
- Implications of these results for practice and research are provided. #
ERP: Enterprise resource planning systems, also referred to as enterprise-wide systems or enterprise systems, offer the “seamless integration of all the information flowing through a company—financial and accounting information, human resource information, supply chain information, and customer information” (Davenport, 1998, p. 121) and present a holistic view of the business from a single information and information technology (IT) architecture (Gable, 1998). #
ERP penetration is 67%, and 15% of companies that do not have ERP today plan to implement it within the next 12 months (Scott
- many ERP installations achieve only partial implementation.
- only 10% of newi nformation systems failures can be attributed to technological problems (Bikson&Gutek, 1984)
- Many ERP projects fail to achieve anticipated benefits because managers underestimate the efforts in- volved in managing change (Appleton, 1997) HUMAN PROBLEM
- One important issue that to date has remained largely unexplored is the nature of end-user computing satisfaction with ERP (Esteves & Pastor, 2001)
Yet while EUCS has been validated and found generalizable across several applications (Doll & Torkzadeh, 1988), it has not been validated with users of ERP systems.
This paper reports the results of a study that further examines the theoretical meaning, structure and dimensionality, and reliability and validity of EUCS when used to measure end-user satisfaction with ERP software applications.
USER INFORMATION SATISFACTION (UIS)
- Extent to which users perceive that the IS available to them meets their information requirements.
- UIS measures the success or failure of an IS
- why a measure of success?: high degree of face validity, development of reliable tools for measure, and conceptual weakness and unavailability of other measures.
END-USER COMPUTING SATISFACTION (EUCS)
- End-user satisfaction is “the affective attitude towards a specific computer application by someone who interacts with the application directly” (Doll & Torkzadeh, 1988, p. 261).
- 12-item survey instrument that was a synthesis of the Ives et al. (1983) measure of UIS, and which is a widely used, validated, and generalizable instrument
- EUCS is a multifaceted construct that requires subjective self-reports of five subscales that measure end-user satisfaction with the content, accuracy, format, timeliness, and ease of use of a computer application and a single overall second-order construct called EUCS.
- Past research has demonstrated instrument validity (content validity, construct validity, and reliability [Straub, 1989]) as well as internal validity and statistical validity
NEED TO CROSS VALIDATE
- Klenke (1992) highlighted the importance of cross-validation of measurement models and stressed the need to retest EUCS with different samples.
- Since the instrument was established, a number of researchers have applied it to various advanced information technologies.
- Responding to the call for “reinstating replication as a critical component of research” (Berthon et al. 2002), we believe EUCS as developed by Doll and Torkzadeh (1988) should be reinvestigated in light of emerging technologies with new data to demonstrate robustness of the measurement model.
- Data used in this study were collected via a nationwide mail survey of users of ERP systems.
- End users were asked to indicate the ERP module(s) they were using and to answer questions about their specific application(s)
- 1,162 firms in the United States
- resulted in 407 usable questionnaires representing a 12.19% response rate
- Data came from 214 organizations with the number of respondents per organization ranging from a minimum of one to a maximum of three
TABLE 1: End User Characteristics
- Seventy-nine per- cent had revenues of $500 million or more,
- almost half (48%) had revenues exceeding $1 billion.
- Approximately 32% were manufacturing organizations,
- 18% were financial services organizations, and
- 13% were utility organizations.
- 37% came from a variety of sectors, including education, insurance, retail, high tech, health care, or government.
- Approximately 86% of the organizations implemented ERP systems regionally or nationally
- 14% reported global implementations
TABLE 2: Profile of end users.
- Table 2 shows that end users were college educated with approximately 92% possessing bachelor’s or master’s degrees.
- Typically, end users had used ERP systems for approximately 3 years.
- Doll and Torkzadeh’s (1988) 12-item instrument to measure EUCS along with descriptive statistics is shown in Table 3.
- ERP systems are profoundly complex pieces of software that require large investments of money, time, and expertise (Davenport, 1998).
- The EQS 5.6 (Bentler, 1995) program with maximum likelihood estimation was used to estimate the confirmatory and structural equation models in this study.
- We tested for multivariate normality of the observed variables to ensure observations were independently and identically distributed.
- No significant outliers were detected.
- Test for identification: The solutions converged at the same point each time, indicative of model identification
- 4 proposed by Doll et al. (1994)
- Tested the fit of each hypothesized model to determine its
consistency and applicability with the sample data of end users of ERP applications.
- Comparison studies examined in this paper have assumed that a second-order factor structure applies, consisting of a single factor called EUCS.
- Figure 1: Model 1: One first-order factor.
Hypothesized one first-order factor (EUCS) accounting for all the common variance among the 12 items.
- Figure 2: Model 2: Five first-order factors (uncorrelated).
Hypothesized that the 12 items form into 5 uncorrelated or orthogonal first-order factors defined as content, accuracy, format, ease of use, and timeliness.
- Figure 3: Model 3: Five first-order factors (correlated).
Hypothesized that the five first-order factors are correlated with each other.
- Figure 4: Model 4: Five first-order factors and one second-order factor.
Hypothesized five first-order factors and one second-order factor (EUCS). The second-order factor EUCS exists but cannot be directly measured by indicator variables. It can only be inferred from the first-order factors, which in turn are measured by their respective indicator variables.
- Model 4 is the only model that has been tested in previous studies over the past decade
The CFA estimation proceeded in a two-step approach in which the confirmatory factor models were tested prior to testing the structural model.
TABLE 5: The results of testing the four models
- Six common model-fit measures, assess each model’s overall goodness of fit.
- Hair et al. (1998) χ2 is appropriate for sample sizes between 100 and 200, with the significance test becoming less reliable with sample sizes outside this range.
- Índice de ajuste normalizado, índice de bondad de ajuste (GFI), índice ajustado de bondad de ajuste (AGFI) y residuo cuadrático medio (RMSR)
- The null model serves as a good baseline model against which to compare alternative models for purposes of evaluating the gain in improved fit, and to establish a zero point for the normed fit index (NFI)
- Doll and Xia (1997) found that neither models 1 nor 2 were even close to being considered a good fit with the sample data. Our results concur with their observations. Model 3 either, model 4 is the best fit.
- Compares model 4 goodness-of-fit indices from the present study with those of previous studies.
- In sum, the absolute indexes (GFI, AGFI, and RMSR) in this study compare favorably with the values reported by the other studies.
Convergent validity was evaluated using three criteria recommended by Fornell and Larcker (1981):
- all indicator factor loadings (λ) should be significant,
- construct reliabilities should exceed .80, and
- average variance extracted (AVE) by each construct should exceed .50.
- The λ-values for all scale items in the CFA models were significant at p ≤ .001
- Composite reliabilities (ρc) of the latent constructs ranged between 0.72 and 0.89 (content .86; accuracy .89; format .74; ease of use .72; timeliness .75).
- AVE ranged from .56 to .61 (content .61; accuracy .80; format .59; ease of use .56; timeliness .60).
Ensure that variance extracted by selected items was greater than that due to measurement error.
TABLE 7: Shows standardized parameter estimates for the observed variables for model 4
- Convergent validity is established if the loadings of the measures to their respective constructs are at least .60 (Bagozzi & Yi, 1988).
- All of the original 12 items demonstrated loadings of .62 or greater and are statistically significant (t > |2.00|)
- R-square, the proportion of the variance in the observed variables that is accounted for by its latent variable, ranges from 0.39 to 0.88 providing evidence of acceptable reliability for all individual items
TABLE 8: standard structural coefficients, corresponding t-values, and R-square values for the latent variables.
- The structural coefficients exceed .70 and are significant, indicating good construct validity of the latent factors comprising the EUCS construct.
- R-square values range from .62 to .95, providing evidence of acceptable reliability for all factors.
- The results are in consonance with the other studies and support the structural equation model and underlying theory.
- A higher-order EUCS factor is confirmed as accounting for or explaining all variance and covariance related to the first-order factors in capturing end users’ satisfaction with ERP systems.
- The replication of these results enhances our confidence in the generalizability of EUCS, and its robustness as a measure of UIS.