WHEN THE QUALITY OF SOURCE DATA FAILS
In our discussion of data quality, we explained how organizations with high data quality use data as a valuable asset that ensures competitiveness, boosts efficiency, improves customer service, and drives profitability. Alternatively, organizations with poor data quality spend much time working with contradictory reports—deviating from business plans (budgets), which leads to misguided decisions based on dated, inconsistent, and erroneous figures. There is, in other words, a strong business case for improved data quality. The question in this section is how organizations can work efficiently to improve the data quality in their source systems (when data is created).
Poor data quality in source systems often becomes evident in connection with profiling when data is combined in the data warehouse, and the trail leads from there to the source system. To improve data quality efficiently, we need to start at the source with validation. For instance, it should not be possible to enter information in the ERP system without selecting an account—it must be obligatory to fill in the account field. If this is not the case, mistakes will sometimes be made that compromise financial reporting. In terms of sales transactions, both customer number and customer name must be filled in.
If these details are not registered, we can't know, for example, where to send the goods. Data quality can typically be improved significantly by making it obligatory to fill in important fields in the source systems. Business transactions should simply not go through unless all required fields are completed.
Another well-known data quality problem arises when the same data is entered twice into one or more source systems. In many international organizations, customers are set up and maintained in a local language and alphabet source system as well as in an English system. The first system can handle specific letters such as the double-letter s that is used in the German language or the special letters used in Scandinavia; the other can't. The solution is, of course, to design the system so as to ensure that customer data can be entered and maintained in one place only.
The keys to improved data quality in source systems are to improve the company's validation procedures when data is created, and to hold a firm principle to create and maintain data in one place only.
In this chapter we went through typical data-generating systems in the business's immediate environment and the difference between primary and secondary data, as well as external and internal analyses. We looked at initiatives to improve the data quality of source systems. Finally, we present a way in which a business can prioritize which source systems to collect project-related data from.
We also explained that if we do not see the potential in source data, we will not be able to lead our business with confidence into the future using information as a strategic resource.