Share on Facebook

Share on Twitter

Share on LinkedIn

Share on Email

Share More



The audit profession has gone through many changes from time to time from review of the financial figures to shifting the focus to risk assessment and internal control, and consideration of fraud in the financial statement that could perpetrate externally or internally. These changes were due to how the business works and the changes taking place in the business scenario. With the emergence of new technology and the way the business is carried, new risks emerges and the audit profession is expected to be updated and to acquire or at least be updated with the business changes that are taking place in the business environment. In the changing scenario, big data has become popular for the organizations which are widely spread across the globe and which are growing exponentially.

With the advent of Big data, the auditing profession has to start looking at how to leverage big data in audits. Rapid changes in the business model possess a threat to the audit profession to match the foot step of the way the business is done. The auditor is needed to acquire new skill and has to start looking into every transaction structured or unstructured rather than audit by exception and analyse the same.

This article tries to bring out the challenges of using the big data and the emerging role of auditors in the changing scenario.

Big data today has become a popular term which is used to describe the exponential growth and availability of data both structured and unstructured.  Big data is collection of data sets so large and complex that it becomes difficult to process using on-hand data management tools or the traditional data processing applications. The big data generally includes complex data sets that exceed the processing capabilities of traditional infrastructure due to their size, format diversity and speed of generation of data. Unlike the traditional Enterprise Resource Planning (ERP) data, where the data used is homogeneous, big data has variety of heterogeneous sources like the customer feedback, promotional calls made, exchange of various mails in the organization.

Large organisations today need to maintain large amount of both structured and unstructured data for operational and compliance purposes. The resultant is huge growth in computing power and storage capacity. Big data provides solution to handle such huge data.  Businesses today are using big data not only to boost performance but also to reduce risk and prevent loss. The big data is mostly used by marketing, sales, engineering, product development, operations, logistics, and strategy development groups to do smarter and faster jobs. Big data takes into consideration the compliance with applicable privacy laws, reputational risks, security considerations and data destruction policies. The Board of Directors and the top management of company are needed to be aware of the compliance requirements, reputational risks, and ethical considerations surrounding privacy and manipulation of sensitive personal data for the commercial gains. Companies need to be sure they are capturing the right data and delivering it in real time to the right people. Big data inherently has its own challenges which are needed to be addressed by the top management. The general principle which is followed where big data is employed is; the greater the amount of data, the greater the risk. Any time an organization uses big data to make decisions, there is risk that it could be misused or used ineffectively. Auditors and other risk professionals obviously should monitor big data use to address this risk.


Inconsistent data formats: Each company or each department in a company has different method of describing the data. It can be describing the ledger or expenses or describing the location.

1. Understanding significance of each variable: Assessment of each variable in its own right to avoid the problem of generalization.

2. Ratifying expert institution: An experience expert who has dealt with similar data can state whether the data has any inconsistencies.

The real challenge of big data is not that you are acquiring large amount of data. It is what you do with the data that matters. Big data can improve the decision making process and increase the profitability, but it also creates significant risks from potential data breaches to privacy and compliance concerns. Big data provides important opportunities to deliver value from information but an enterprise will be more successful in long run if proper policies and procedures are in place.

Some important areas which are needed to be answered include:

a. Reliability of data

b. Protection of sources, processes and decisions from theft and corruption.

c. Collection of information without exposing the enterprise to legal and regulatory compliances.

d. Trend of actions taken that can be exploited by rivals

The audit professionals should use this opportunity to re-examine what aspects of technology should be part of the audit practitioners skill set; given that society’s attitude towards technology has clearly changed. The audit profession should understand what data is present in the big data and how this data can be correlated with financial data to make it more effective and efficient audit. The task of auditor in this regard environment increases. They should not only help organization identify the risks that the use of big data can pose. At the same time, they must be alert to the opportunities the big data can offer. The auditors should be approaching big data in proactive manner, identifying opportunities to use large sets in new ways that will enable them to operate with greater efficiency. The auditors has important role to play in relation to big data and exploiting big data to identify and mitigating the existing risk. The opportunities and risks that big data presents are significant and complex –too significant and complex to be considered solely or even primarily a technology concern.


The role of auditor, CFO and controller can be separated into three components

1. The risk and business control components that defines the traditional role of finance and audit professional

2. Enhancements to internal and external audit. These are so significant they deserve their own category.

3. Applications of big data and predictive analysis that promise to support finance professionals as the new pro-active stewards of better corporate governance.

Role of auditor while handling big data in traditional audit risk and business control includes:

a. Planning and forecasting: Big data adds new data sources and increased detail and makes the forecasts fast to update. Improvements to predictive analytics make them more accurate.

b. Strategic Financial management: Big data adds depth to the detail, speed to updates, and the opportunity to incorporate new meaningful data types both internal to the company and from outside sources. Dashboards can track both leading and lagging indicators to give a more accurate view of the future.

c. Enhance financial closing, statutory reporting, and analysis of variances: Big data technologies can allow incorporation of both new and traditional data types at very high levels of detail and allow the data to be retrieved and processed in a fraction of time. The new technologies used in the big data can achieve this integration cost effectively.

d. Compliance and regulation: The amount of risk, time and resources drained from these activities can be substantially reduced through application of both big data technologies and predictive analysis.

e. Fraud detection: Using the greater variety and detail of data available through big data to power predictive analytics to highlight potential fraud increases the likelihood of detection. It also decreases the incidence of deviation resulting in better, faster detection and protection and less cost in human resources.

f. Asset recovery: Detection of duplicate invoices and duplicate payments is one area where many accountant professionals start with big data. It is now practical to use automated analytic sorting to evaluate years of detailed invoice and payments to identify and recover duplicates due to low cost, high speed Hadoop – based big data storage and retrieval systems.

g. Depth of audit:  Big data technologies removes the requirement of sampling by handling massive amounts of data and many different varieties of data at a cost, and in a retrieval and analysis time frame that was previously impossible. On the operational side, it is this ability to look literally at everything that now makes it possible for technology to enforce compliance in real time.

h. Self-service discovery: Big data tools and analytics have largely automated tests allowing continuous monitoring without the need for an increase in IT audit human resources. These new technologies can deliver results up to 90% faster than traditional tools.

i. Audit test automation: Even the most complex financial transactions can be modelled and examined with the new analytic tools shortening audit cycles, lowering cost, and allowing deeper audits.

j. Data reconciliation: Big data analytics tests and compares transactions across multiple systems and simultaneously tests for compliance to policy and procedures.

k. Report verification: Analytics can compare the accuracy of reports prepared by separate systems to test and ensure consistency and accuracy

l. Recalculation: Based on the analytics ability to model and replicate complex transactions and postings, testing, reproducing, and confirming the accuracy of transactions  under multiple scenario is easy and quick


a. Efficiency of execution: Big data technologies and analytics empower the auditors to constantly monitor and recalibrate predictions based on enhanced ability to understand and analyse internal processes, competitive pressures, and customer response. Big data also helps in quickly finding answers to specific problems and modifying forward forecasts in near real time allows financial professionals to play an influential role in steering their companies.

b. Growth and Innovation: The big data and analytic capabilities that evaluate efficiency also offer up suggestions for growth opportunities in markets or complimentary products and services. Big data enables deeper and broader insights through analytics that can be the difference between successful and unsuccessful acquisition and business development strategies.

c. Valuation and customer equity: Traditional valuation techniques based on discounted free cash flows can be greatly enhanced with a clear understanding of customer lifetime value. Big data and analytics offers a clear path to better understanding customer behaviour, buying patterns, and wallet share which can be aggregated up to customer equity measures.

d. Customer risk: The risk involved in the customer equity is customer risk. Big data and analytics offers a path to better understanding of customer behaviour. Big data and analytics allows this with new depth and accuracy and this data can be used to correlate these measures with cash flows, profit and other financial measures to guide strategic decisions.

e. Reputational risk: Reputational risk runs parallel to the operational risk. This risk can arise from customers, non-customers and even from employees. The audit professionals has to recognized the need to constantly monitor and respond to the threats from social media which may be customer comments about products, service, price of quality, but it can also be inappropriate leaks of sensitive or incorrect private financial or transactional plans and data. Big data and analytics now offers a way for financial professionals to develop and maintain a listening post to spot and defend against these risks which have the potential materiality impact financial results.


Detecting of fraud has never been an easy process. For many years, auditors have relied on the whistle-blowers, or a wrong step taken by perpetrators or good amount of luck to uncover wrongdoing both inside and outside the companies. In most of the cases, where the rule based queries were used to detect fraud, it was easily evaded by smart thieves who know how to hide the numerical patterns.

In the rule based queries, many of the bribery schemes in the cases on record lasted for years before they were detected, and some involved several executives. Similarly in many of the cases, detecting improper payments largely comes down to what misguided employees enter into free-text field of a payment description in the account payable system.

The traditional way of detecting fraud can be changing with the advent of big data era. Big data which are high powered analytical tools can crunch through enormous quantities of structured and unstructured data, producing an exponential greater set of comparisons within the data on any given time. Big data also helps to analyse text and other unstructured data with e-mails and social media activity yielding plenty of insights about what is going on beyond the numbers.

With the exponential growth of the business across the globe and with the huge data collected through various means, big data is here to stay and it is the time the audit profession also take the leverage of big data for the purpose of audit.


1. Big data towards a “Data – driven audit” by Kenneth Cukier and Viktor Mayer Schonberger.
2. Role of auditor and other finance professional by Bill Vorhis; October 31, 2013
3. and insight


Published by

CA Amrita Chattopadhyay
(Audit & Assurance)
Category Audit   Report

1 Likes   28 Shares   9312 Views


Related Articles


Popular Articles

caclubindia books caclubindia books Book

CCI Articles

submit article

Stay updated with latest Articles!