Data is hugely significant in the business world, yet its true value lies not only in volume but also quality. Bad data can hinder your business growth and lead to wrong decisions and missed opportunities. This article explores the practical aspects of data quality management to help you discover the true potential of information and make decisions based on data that is reliable and accurate.
Bad data can have considerable financial consequences for businesses. In 2022, for example, Unity Software reported a loss of USD 110 million in revenue and USD 4.2 billion in its market capitalisation after ingesting bad data from a large customer. IBM studies suggest that bad data annually costs the US economy about USD 3.1 trillion. However, the consequences of bad data are not limited to financial losses, they also affect innovation opportunities and reputation. Experian Data Quality studies show that only 3% of corporate data meets quality standards and 90% of data is not used for strategic purposes. These findings show the criticality of data quality in today’s business environment and the importance of investing in data improvements.
Data quality management is not a universal solution. It requires an adapted approach that meets each organisation’s unique issues and needs. Let us look at a few tried and tested strategies for improving data quality.
Master data management (MDM)
MDM allows you to create a single reliable source for your key business data, securing accuracy and consistency across different systems and departments. This helps you prevent data fragmentation and reduce errors. Popular MDM solutions include SAP Master Data Governance, Informatica MDM and IBM InfoSphere MDM.
Data standardisation
You can use data profiling (extract, transform, load) tools, such as Talend or Informatica Data Quality, to identify and correct errors, discrepancies and duplicates, and to generate data standardisation rules. These rules secure data compliance by determining formats, title conventions and validation criteria for particular data elements. For example, a rule might require that all customer addresses should fit a certain format or all product codes should be a combination letters and numbers.
Data validation and enrichment
You can use data validation tools, such as Great Expectations or Apache Spark, to secure the integrity of your data. Apache Spark can be used to create validation pipelines that will check the completeness, consistency and accuracy of your data. This can also enrich your data by combining it with external data sets or using machine-learning models to obtain new insights.
Using data quality monitoring tools, such as Collibra or Ataccama, allows you to track key data indicators and determine areas that need improving. Regular monitoring of data quality allows you to identify issues early and take corrective action, ensuring data always meets your business needs and strategic goals.
|
Indicator |
Values in an average data set |
Consequences if the indicator is low |
|
Completeness – what percentage of entries contain all required values |
80–95% |
Incomplete information can lead to wrong decisions and operational inefficiencies. For example, if contact details are missing from your customer database, you can find it difficult to communicate with and serve your customers. |
|
Accuracy – what percentage of entries correspond to reality and can be verified |
90–99% |
Inaccurate data can lead to wrong financial statements or have legal consequences. For example, if a customer in your accounting system is wrongly defined as an EU company with a registered VAT number eligible for 0% VAT but is in fact an individual subject to 21% VAT, this can lead to wrong taxes and wrong bills. Such an error can have financial consequences for the company, as well as causing legal issues if inappropriate tax payments are discovered. |
|
Consistency – how consistent your data points are across your systems |
95–99% |
Data discrepancies can cause confusion and stall your decision-making process. For example, if a product on an e-commerce platform is priced lower than the cost appearing in your accounting system, this can cause a loss, as you would be selling the product below cost. If the price on the platform is too high, this can adversely affect your sales volumes. So it’s important to ensure your pricing and accounting data is consistent. |
|
Timeliness – how often data is updated |
While it depends on the data type, data is usually considered timely if it has been updated within the last 24 hours. |
Obsolete data can lead to wrong decisions and missed opportunities. For example, outdated information on share prices can lead to unprofitable investments. |
|
Uniqueness – the percentage of unique entries |
95–100% |
Data duplicates can create misunderstandings, hinder analysis and lead to wrong conclusions. For example, if your customer database has two or more entries for the same customer, this can cause problems in serving your customers and personalising your marketing campaigns. |
These data quality indicators highlight the importance of ensuring your data is not only big but also accurate, complete, consistent, timely and unique. Only then you can make full use of your data potential and make decisions based on reliable information.
In today’s competitive environment, data quality is no longer a luxury – it’s a strategic imperative. By prioritising data quality and adopting proactive management practices, organisations can utilise their data potential, drive innovation and achieve sustainable success. Make data quality a habit in your organisation!
If you have any comments on this article please email them to lv_mindlink@pwc.com
Ask question
