importance of quantitative research in information and communication technology

Null Hypothesis Significance Testing: a Guide to Commonly Misunderstood Concepts and Recommendations for Good Practice [version 5; peer review: 2 approved, 2 not approved]. Imagine a situation where you carry out a series of statistical tests and find terrific indications for statistical significance. As a second example, models in articles will sometimes have a grab-all variable/construct such as Environmental Factors. The problem here is similar to the example above. 443-507). Unfortunately, unbeknownst to you, the model you specify is wrong (in the sense that the model may omit common antecedents to both the independent and the dependent variables, or that it exhibits endogeneity concerns). Data are gathered before the independent variables are introduced, but the final form is not usually known until after the independent variables have been introduced and the after data has been collected (Jenkins, 1985). After observing the situation to be investigated, the researcher forms a hypothesis and then uses deductive reasoning by predicting how the data should look if the hypothesis is true, after collecting the data and analyzing it to confirm or rejectthe hypothesis. If samples are not drawn independently, or are not selected randomly, or are not selected to represent the population precisely, then the conclusions drawn from NHST are thrown into question because it is impossible to correct for unknown sampling bias. Ideally, when developing a study, researchers should review their goals as well as the claims they hope to make before deciding whether the quantitative method is the best approach. Research Methods in Social Relations (6th ed.). Journal of the Association for Information Systems, 18(10), 727-757. One benefit of a high-quality education is learning the purposes and advantages of the various methodologies and how to apply them in your own research. What are theories? A correlation between two variables merely confirms that the changes in variable levels behave in particular way upon changing another; but it cannot make a statement about which factor causes the change in variables (it is not unidirectional). A Comparison of Web and Mail Survey Response Rates. The idea is to test a measurement model established given newly collected data against theoretically-derived constructs that have been measured with validated instruments and tested against a variety of persons, settings, times, and, in the case of IS research, technologies, in order to make the argument more compelling that the constructs themselves are valid (Straub et al. Churchill Jr., G. A. Essentially, time series data is single variable data that has another dimension of time. It is a special case of MANOVA used with two groups or levels of a treatment variable (Hair et al., 2010). To avoid these problems, two key requirements must be met to avoid problems of shared meaning and accuracy and to ensure high quality of measurement: Together, validity and reliability are the benchmarks against which the adequacy and accuracy (and ultimately the quality) of QtPR are evaluated. For this reason, they argue for a critical-realist perspective, positing that causal relationships cannot be perceived with total accuracy by our imperfect sensory and intellective capacities (p. 29). The quantitative approach holds the researcher to remain distant and independent of that being researched. Larsen, K. R. T., & Bong, C. H. (2016). Accordingly, scientific theory, in the traditional positivist view, is about trying to falsify the predictions of the theory. In E. Mumford, R. Hirschheim, & A. T. Wood-Harper (Eds. If they omit measures, the error is one of exclusion. Other tests include factor analysis (a latent variable modeling approach) or principal component analysis (a composite-based analysis approach), both of which are tests to assess whether items load appropriately on constructs represented through a mathematically latent variable (a higher order factor). Likewise, problems manifest if accuracy of measurement is not assured. They could legitimately argue that your content validity was not the best. Random assignment is about randomly manipulating the instrumentation so that there is a very unlikely connection between the group assignments (in an experimental block design) and the experimental outcomes. Several detailed step-by-step guides exist for running SEM analysis (e.g., Gefen, 2019; Ringle et al., 2012; Mertens et al., 2017; Henseler et al., 2015). Regarding Type II errors, it is important that researchers be able to report a beta statistic, which is the probability that they are correct and free of a Type II error. (2001). We might say that archival data might be reasonably objective, but it is not purely objective By any stretch of the imagination. Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach (2nd ed.). This discovery, basically uncontended to this day, found that the underlying laws of nature (in Heisenbergs case, the movement and position of atomic particles), were not perfectly predictable, that is to say, deterministic. Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004). Using Multivariate Statistics (4th ed.). quantitative or qualitative methods is barren, and that the fit-for-purpose principle should be the central issue in methodological design. The American Statistician, 59(2), 121-126. ACM SIGMIS Database, 50(3), 12-37. While this is often true, quantitative methods do not necessarily involve statistical examination of numbers. The moving average part adds a linear combination of the error terms of the previous observations. A TETRAD-based Approach for Theory Development in Information Systems Research. A procedure for the analysis of LInear Structural RELations among one or more sets of variables and variates. Prentice Hall. This is why often in QtPR researchers often look to replace observations made by the researcher or other subjects with other, presumably more objective data such as publicly verified performance metrics rather than subjectively experienced performance. Univariate analysis of variance (ANOVA) is a statistical technique to determine, on the basisof one dependent measure, whether samples come from populations with equal means. Neyman, J., & Pearson, E. S. (1928). . Their paper presents the arguments for why various forms of instrumentation validity should be mandatory and why others are optional. Meta-analyses are extremely useful to scholars in well-established research streams because they can highlight what is fairly well known in a stream, what appears not to be well supported, and what needs to be further explored. Journal of Management Analytics, 1(4), 241-248. (2009). Quantitative Research is a systematic approach to collect data through sampling method like online polls, online surveys, Questionnaires etc. Like the theoretical research model of construct relationships itself, they are intended to capture the essence of a phenomenon and then to reduce it to a parsimonious form that can be operationalized through measurements. The original online resource that was previously maintained by Detmar Straub, David Gefen, and Marie-Claude Boudreau remains citable as a book chapter: Straub, D.W., Gefen, D., & Boudreau, M-C. (2005). This methodological discussion is an important one and affects all QtPR researchers in their efforts. ), The Handbook of Information Systems Research (pp. Quantitative Research in the field of business is significant because through statistical methods, high possibilities of risk can be prevented. Central to understanding this principle is the recognition that there is no such thing as a pure observation. It focuses on eliciting important constructs and identifying ways for measuring these. Gefen, D., Ben-Assuli, O., Stehr, M., Rosen, B., & Denekamp, Y. The units are known so comparisons of measurements are possible. Journal of Management Information Systems, 19(2), 129-174. Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). Masson, M. E. (2011). Gelman, A., & Stern, H. (2006). Overall, modern social scientists favor theorizing models with expressed causal linkages and predictions of correlational signs. But setting these exact points in the experiment means that we can generalize only to these three delay points. Springer. Time-series analysis can be run as an Auto-Regressive Integrated Moving Average (ARIMA) model that specifies how previous observations in the series determine the current observation. 50th Hawaii International Conference on System Sciences, Waikoloa Village, Hawaii. Mohajeri, K., Mesgari, M., & Lee, A. S. (2020). Scholars argue that we are living in a technological age. Measurement for the Social Sciences: The C-OAR-SE Method and Why It Must Replace Psychometrics. This idea introduced the notions of control of error rates, and of critical intervals. Information and Organization, 30(1), 100287. Miller, I., & Miller, M. (2012). necessarily reflect the official policy or position of Grand Canyon University. The role & application of ICT in research and Higher Education academic work can be broadly divided into 4 major areas: -. But the effective labelling of the construct itself can go a long way toward making theoretical models more intuitively appealing. You can learn more about the philosophical basis of QtPR in writings by Karl Popper (1959) and Carl Hempel (1965). It differs from construct validity, in that it focuses on alternative explanations of the strength of links between constructs whereas construct validity focuses on the measurement of individual constructs. Likewise, QtPR methods differ in the extent to which randomization is employed during data collection (e.g., during sampling or manipulations). The Difference Between Significant and Not Significant is not Itself Statistically Significant. Find more answers Ask your question New questions in English (2013). One of the main reasons we were interested in maintaining this online resource is that we have already published a number of articles and books on the subject. Interpretation of Formative Measurement in Information Systems Research. These nuances impact how quantitative or qualitative researchers conceive and use data, they impact how researchers analyze that data, and they impact the argumentation and rhetorical style of the research (Sarker et al., 2018). Unless the persons weight actually changes in the times between stepping repeatedly on to the scale, the scale should consistently, within measurement error, give you the same results. Sage. Kluwer Academic Publishers. Reinhart, A. Hayesa, A. F. and Coutts, J. J. Statistically, the endogeneity problem occurs when model variables are highly correlated with error terms. Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. Experimentation in Software Engineering: An Introduction. Consider that with alternative hypothesis testing, the researcher is arguing that a change in practice would be desirable (that is, a direction/sign is being proposed). While the positivist epistemology deals only with observed and measured knowledge, the post-positivist epistemology recognizes that such an approach would result in making many important aspects of psychology irrelevant because feelings and perceptions cannot be readily measured. econometrics) and numerical methods such as mathematical modeling. Interpretive Case Studies in IS Research: Nature and Method. The posterior can also be used for making predictions about future events. Random item inclusion means assuring content validity in a construct by drawing randomly from the universe of all possible measures of a given construct. Use Omega Rather than Cronbachs Alpha for Estimating Reliability. Consider, for example, that you want to score student thesis submissions in terms of originality, rigor, and other criteria. ber den anschaulichen Inhalt der quantentheoretischen Kinematik und Mechanik (in German). Figure 2 also points to two key challenges in QtPR. Descriptive analysis refers to describing, aggregating, and presenting the constructs of interests or the associations between the constructs to describe, for example, the population from where the data originated, the range of response levels obtained, and so forth. In D. Avison & J. Pries-Heje (Eds. Traditionally, QtPR has been dominant in this second genre, theory-evaluation, although there are many applications of QtPR for theory-generation as well (e.g., Im & Wang, 2007; Evermann & Tate, 2011). The integrated part of the model is included when there is a trend in the data, such as an increase over time, in which case the difference between the observations is calculated rather than modeling the actual observed values. As a conceptual labeling, this is superior in that one can readily conceive of a relatively quiet marketplace where risks were, on the whole, low. Development And Measurement Validity Of A Task-Technology Fit Instrument For User Evaluations Of Information Systems. The table in Figure 10 presents a number of guidelines for IS scholars constructing and reporting QtPR research based on, and extended from, Mertens and Recker (2020). 130 Information Technology Research Topics And Quick Writing Prompts. accurate as of the publish date. Data computing equipment makes it possible to process and analyze data quickly, even with large sample sizes. Kaplan, B., and Duchon, D. Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study, MIS Quarterly (12:4 (December)) 1988, pp. Why not? Revisiting Bias Due to Construct Misspecification: Different Results from Considering Coefficients in Standardized Form. For a better experience, please consider using a modern browser such as Chrome, Firefox, or Edge. Therefore, QtPR can involve different techniques for data collection and analysis, just as qualitative research can involve different techniques for data collection (such as focus groups, case study, or interviews) and data analysis (such as content analysis, discourse analysis, or network analysis). This distinction is important. For example, several historically accepted ways to validate measurements (such as approaches based on average variance extracted, composite reliability, or goodness of fit indices) have later been criticized and eventually displaced by alternative approaches. In other words, QtPR researchers are generally inclined to hypothesize that a certain set of antecedents predicts one or more outcomes, co-varying either positively or negatively. The same thing can be said about many econometric studies and other studies using archival data or digital trace data from an organization. In this technique, one or more independent variables are used to predict a single dependent variable. Aside from reducing effort and speeding up the research, the main reason for doing so is that using existing, validated measures ensures comparability of new results to reported results in the literature: analyses can be conducted to compare findings side-by-side. There are numerous ways to assess construct validity (Straub, Boudreau, and Gefen, 2004; Gefen, Straub, and Boudreau, 2000; Straub, 1989). Christensen, R. (2005). The importance of quantitative research is that it offers tremendous help in studying samples and populations. Quantitative research is a powerful tool for anyone looking to learn more about their market and customers. We note that these are our own, short-handed descriptions of views that have been, and continue to be, debated at length in ongoing philosophy of science discourses. Scandinavian Journal of Information Systems, 22(2), 3-30. University of Chicago Press. In what follows, we give a few selected tips related to the crafting of such papers. Also, QtPR typically validates its findings through testing against empirical data whereas design research can also find acceptable validation of a new design through mathematical proofs of concept or through algorithmic analyses alone. Researchers use quantitative methods to observe situations or events that affect people.1Quantitative research produces objective data that can be clearly communicated through statistics and numbers. Similarly, the choice of data analysis can vary: For example, covariance structural equation modeling does not allow determining the cause-effect relationship between independent and dependent variables unless temporal precedence is included. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. Elsevier. NHST originated from a debate that mainly took place in the first half of the 20th century between Fisher (e.g., 1935a, 1935b; 1955) on the one hand, and Neyman and Pearson (e.g., 1928, 1933) on the other hand. In reality, any of the included stages may need to be performed multiple times and it may be necessary to revert to an earlier stage when the results of a later stage do not meet expectations. Reliable quantitative research requires the knowledge and skills to scrutinize your findings thoroughly. This methodology models the real world and states the results as mathematical equations. Field experiments involve the experimental manipulation of one or more variables within a naturally occurring system and subsequent measurement of the impact of the manipulation on one or more dependent variables (Boudreau et al., 2001). Multivariate Data Analysis (7th ed.). 3. Journal of Consumer Research, 30(2), 199-218. Shadish et al. It is the most common form of survey instrument use in information systems research. Cesem, Cisee, K-fist (l2), K-fist (l1), Smysr, Rftt, Arp Proposal Format 2015 . Advertisement Still have questions? Secondary data sources can be usually found quickly and cheaply. Journal of Personality Assessment, 80(1), 99-103. Vessey, I., Ramesh, V., & Glass, R. L. (2002). The experimenter might use a random process to decide whether a given subject is in a treatment group or a control group. Another problem with Cronbachs alpha is that a higher alpha can most often be obtained simply by adding more construct items in that alpha is a function of k items. Random assignment makes it highly unlikely that subjects prior knowledge impacted the DV. Manipulation validity is used in experiments to assess whether an experimental group (but not the control group) is faithfully manipulated and we can thus reasonably trust that any observed group differences are in fact attributable to the experimental manipulation. Hence the external validity of the study is high. (1980), Causal Methods in Marketing. Integrated communications and technology (ICT) encompasses both . Recker, J. Taking steps to obtain accurate measurements (the connection between real-world domain and the concepts operationalization through a measure) can reduce the likelihood of problems on the right side of Figure 2, affecting the data (accuracy of measurement). Evermann, J., & Tate, M. (2014). (1972). Following the MAP (Methods, Approaches, Perspectives) in Information Systems Research. In simple terms, in QtPR it is often useful to understand theory as a lawlike statement that attributes causality to sets of variables, although other conceptions of theory do exist and are used in QtPR and other types of research (Gregor, 2006). Historically, internal validity was established through the use of statistical control variables. With the caveat offered above that in scholarly praxis, null hypotheses are tested today only in certain disciplines, the underlying testing principles of NHST remain the dominant statistical approach in science today (Gigerenzer, 2004). 2017; Gefen, Straub, and Boudreau 2000; Gefen 2003). Applied Multivariate Statistics for the Social Sciences (4th ed.). W. H. Freeman. The Free Press. A more reliable way, therefore, would be to use a scale. It can include also cross-correlations with other covariates. Hempel, C. G. (1965). It is also important to regularly check for methodological advances in journal articles, such as (Baruch & Holtom, 2008; Kaplowitz et al., 2004; King & He, 2005). The importance of information communication technology, visual analysis, and web monitoring and control are all examples of Information Communication Technology (ICT). Data analysis techniques include univariate analysis (such as analysis of single-variable distributions), bivariate analysis, and more generally, multivariate analysis. Data analysis concerns the examination of quantitative data in a number of ways. Because developing and assessing measures and measurement is time-consuming and challenging, researchers should first and always identify existing measures and measurements that have already been developed and assessed, to evaluate their potential for reuse. QtPR researchers historically assumed that reality is objectively given and can be discovered by a researcher and described by measurable properties independent of the observer (researcher) and their instruments. Cambridge University Press. An example would be the correlation between salary increases and job satisfaction. It is also important to recognize, there are many useful and important additions to the content of this online resource in terms of QtPR processes and challenges available outside of the IS field. Information sharing - How quickly & easily information can be shared across the globe. The variables that are chosen as operationalizations to measure a theoretical construct must share its meaning (in all its complexity if needed). Communications of the Association for Information Systems, 16(45), 880-894. Here are some examples of studies conducted in educational settings using the above research methodologies:3, Quantitative researchers aim to create a general understanding of behavior and other phenomena across different settings and populations. What could this possibly mean? Limitation, recommendation for future works and conclusion are also included. In other words, many of the items may not be highly interchangeable, highly correlated, reflective items (Jarvis et al., 2003), but this will not be obvious to researchers unless they examine the impact of removing items one-by-one from the construct. The ASAs Statement on P-values: Context, Process, and Purpose. The current ICT master plan for the Philippines dates back to 2006. Laboratory experiments take place in a setting especially created by the researcher for the investigation of the phenomenon. In QtPR practice since World War II, moreover, social scientists have tended to seek out confirmation of a theoretical position rather than its disconfirmation, a la Popper. We do this in a systematic scientific way so the studies can be replicated by someone else. Research Methodologies and MIS Research. Journal of the Royal Statistical Society. Intermediaries may have decided on their own not to pull all the data the researcher requested, but only a subset. CT Bauer College of Business, University of Houston, USA, 15, 1-16. Explanatory surveys ask about the relations between variables often on the basis of theoretically grounded expectations about how and why the variables ought to be related. (Wikipedia.org). It implies that there will be some form of a quantitative representation of the presence of the firm in the marketplace. Another important debate in the QtPR realm is the ongoing discussion on reflective versus formative measurement development, which was not covered in this resource. Communications of the Association for Information Systems, 37(44), 911-964. Sira Vegas and colleagues (Vegas et al., 2016) discuss advantages and disadvantages between a wide range of experiment designs, such as independent measures, repeated measures, crossover, matched-pairs, and different mixed designs. They may well be. Comparative research can also include ex post facto study designs where archival data is used. Many of these data collection techniques require a research instrument, such as a questionnaire or an interview script. Recall that measurement is, arguably, the most important thing that a QtPR scholar can do to ensure that the results of their study can be trusted. Chalmers, A. F. (1999). Equity issues. In such a situation you are in the worst possible scenario: you have poor internal validity but good statistical conclusion validity. The higher the statistical power of a test, the lower the risk of making a Type II error. (2019). This allows comparing methods according to their validities (Stone, 1981). In other words, SEM allows researchers to examine the reliability and validity of their measurements as well as the hypotheses contained in their proposed theoretical model. Selection bias means that individuals, groups, or other data has been collected without achieving proper randomization, thereby failing to ensure that the sample obtained is representative of the population intended to be analyzed. Edwards, J. R., & Berry, J. W. (2010). Straub, Boudreau, and Gefen (2004) introduce and discuss a range of additional types of reliability such as unidimensional reliability, composite reliability, split-half reliability, or test-retest reliability. Allyn & Bacon. The ability to explain any observation as an apparent verification of psychoanalysis is no proof of the theory because it can never be proven wrong to those who believe in it. When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment. Studying something so connected to emotions may seem a challenging task, but don't worry: there is a lot of perfectly credible data you can use in your research paper if only you choose the right topic. Psychonomic Bulletin & Review, 16(4), 617-640. Written for communication students, Quantitative Research in Communication provides practical, user-friendly coverage of how to use statistics, how to interpret SPSS printouts, how to write results, and how to assess whether the assumptions of various procedures have been met. It can also include other covariates. Journal of the Association for Information Systems, 19(8), 752-774. Why is the Hypothetico-Deductive (H-D) Method in Information Systems not an H-D Method? This video emphasized the Importance of quantitative research across various fields such as Science, Technology, Engineering, and Mathematics (STEM), Account. Lindman, H. R. (1974). Accounting principles try to control this, but, as cases like Enron demonstrate, it is possible for reported revenues or earnings to be manipulated. Q-sorting consists of a modified rank-ordering procedure in which stimuli are placed in an order that is significant from the standpoint of a person operating under specified conditions. They are stochastic. How does this ultimately play out in modern social science methodologies? We can have correlational associated or correlational predictive designs. Creating model over findings ie. Mathematically, what we are doing in statistics, for example in a t-test, is to estimate the probability of obtaining the observed result or anything more extreme in the available sample data than that was actually observed, assuming that (1) the null hypothesis holds true in the population and (2) all underlying model and test assumptions are met (McShane & Gal, 2017). (1989) Structural Equations with Latent Variables. To transform this same passage into passive voice is fairly straight-forward (of course, there are also many other ways to make sentences interesting without using personal pronouns): To measure the knowledge of the subjects, ratings offered through the platform were used. In turn, there are theoretical assessments of validity (for example, for content validity,), which assess how well an operationalized measure fits the conceptual definition of the relevant theoretical construct; and empirical assessments of validity (for example, for convergent and discriminant validity), which assess how well collected measurements behave in relation to the theoretical expectations. Methods of Psychological Research, 7(1), 1-20. Quantitative research has the goal of generating knowledge and gaining understanding of the social world. Experimental research is often considered the gold standard in QtPR, but it is also one of the most difficult. Despite this buzz, however, many students still find it challenging to compose an information technology research topic. When authors say their method was a survey, for example, they are telling the readers how they gathered the data, but they are not really telling what their method was. Initially, a researcher must decide what the purpose of their specific study is: Is it confirmatory or is it exploratory research? Lee, A. S., Mohajeri, K., & Hubona, G. S. (2017). Providing a strong conceptual orientation to techniques and procedures that range from the "moderately basic" to . Statistical Significance Versus Practical Importance in Information Systems Research. Burton-Jones, A., Recker, J., Indulska, M., Green, P., & Weber, R. (2017). This notion that scientists can forgive instances of disproof as long as the bulk of the evidence still corroborates the base theory lies behind the general philosophical thinking of Imre Lakatos (1970). The alpha protection levels are often set at .05 or lower, meaning that the researcher has at most only a 5% risk of being wrong and subject to a Type I error. For example, one key aspect in experiments is the choice of between-subject and within-subject designs: In between-subject designs, different people test each experimental condition. Philosophical Transactions of the Royal Society of London. Likewise, with the beta: Clinical trials require fairly large numbers of subjects and so the effect of large samples makes it highly unlikely that what we infer from the sample will not readily generalize to the population. Greene, W. H. (2012). This stage also involves assessing these candidate items, which is often carried out through expert panels that need to sort, rate, or rank items in relation to one or more content domains of the constructs. (2019). It examines the covariance structures of the variables and variates included in the model under consideration. Hedges, L. V., & Olkin, I. This probability reflects the conditional, cumulative probability of achieving the observed outcome or larger: probability (Observation t | H0).
William Schur Regis Grandson, 173rd Airborne Vietnam Roster, Westside District 66 Salary Schedule, List Of Satellites In Graveyard Orbit, Articles I