An assessment on the acceptance of big data in statistics

Abstract

The data revolution which has caused an explosion of data volumes and increased data demands is expected to have a big impact in research institutes. To support this data revolution and improve the quantity, frequency, disaggregation, and availability of relevant statistics, there is an essence to use Big Data in statistics. The study sought to assess the adoption of Big Data in research institutes in Kenya, establish the risks and challenges of using Big Data in statistics, identify the determinants of adoption of Big Data in statistics, and validating the relevance of TAM based model for predicting the adoption. Big Data is a transformative tool for statistics and has great potential to fill data gaps, leveraged to reduce costs and improve the availability of data to monitor development goals. The study used a descriptive survey where quantitative data was collected using self-administered questionnaires. The data was collected from sampled staff sampled from research institutes. Data were statistically analyzed using Stata. Composite reliability was used to assess reliability while Factor loadings and average variance extracted were used to assess convergent validity. Descriptive statistics for each construct of the TAM-based model were generated. The test of the structural model which includes estimating the path coefficients was done using Structural equation Modelling. The study found that research institutes are adopting Big Data in statistics by developing and using Big Data strategies. Legal and regulatory issues; gaining access to data; gaining access to associated methodology and metadata; establishing dataset quality are the main challenges of using Big Data in statistics. Inconsistent access and continuity; privacy breaches and data security; resource constraints and cut-backs; and resistance of Big Data providers and populace were noted as the most prominent risks. The study establishes that external influence, subjective norms, perceived usefulness, compatibility, attitude towards use, and self-efficacy as the key factors influencing acceptance of Big Data in Statistics. The limitation of the study was that Market research companies, credit reference bureaus, private research institutes, and Big Data Analytics companies deal with statistics and were not included in the study. Research institutes agree that Big Data can complement traditional sources of data to generate statistics and are ready to adopt it. However, the risks and challenges highlighted in the study must be overcome for successful adoption. The study recommends sensitization, training, and capacity building of data professionals, resolving of legal and regulatory issues, improvement of statistical methodologies of sampling and analysis, and allocation of more resources to Big Data projects.