AFTER a decade of enterprise resource planning (ERP) and two decades into enterprise data warehousing (EDW), many business executives are still frustrated over their inability to trust their company's data. They have spent millions on new technologies, only to find that the state of their data assets keeps deteriorating instead of improving. This significantly reduces the business value of their investments. One big reason for this continuing data chaos is that companies do not manage their data as a business asset.
In order to manage your data as a business asset you need an enterprise-wide data governance programme, which includes a set of policies, processes, resources, roles, and responsibilities to standardise data, control data redundancy, and share a single version of the truth across the organisation.
Why you need data governance
Just imagine
A chief financial officer (CFO) is approached by the chief executive officer (CEO) and is asked for an accounting of the company's financial assets. The CFO gives a vague response indicating a lack of knowledge of the corporate bank accounts, has little idea what is in each account, and has no idea about the status of accounts receivables. When asked about the intended use of the financial assets, the CFO replies "there is no plan for their use." The CFO (and probably the CEO) would quickly lose their jobs.
Now imagine
A chief information officer (CIO) is approached by the CEO and is asked for an accounting of the company's data assets. The CIO gives a vague response indicating no knowledge of the inventory of data, has little idea where the data is stored, and has no idea how much data is duplicated and whether or not it is consistent. When asked about the intended use of the data assets, the CIO replies "there is no plan for their use." Interestingly enough, the CIO would not get fired.
Not managing your data as a business asset is analogous to a company allowing each department and each person within each department to develop their own financial chart of accounts. This empowerment would allow each person in the organisation to choose his or her own numbering scheme. Existing charts of accounts would be ignored as each person exercises his or her own creativity. The resulting chaos is obvious and easy to predict. The same goes for data assets.
There are many good business reasons to institute a data governance programme. Publicly traded companies are governed by a Security and Exchange Commission and must pass their audits. Then there is the risk of losing investor confidence, and maybe even losing the investors. Government agencies will impose fines, and in some cases jail sentences, if they find irregularities in the data reported to them.
There is always the risk of losing customers due to poor data quality. Business performance cannot be optimal if there is loss of productivity because of data redundancy and time-consuming reconciliation efforts. And finally, mergers and acquisitions continue to be painful and costly when data is not standardised.
Definitions
Let us start with some basic definitions of data governance and the principal responsibilities the business people and IT technicians must share in order to manage data as a business asset.
Data governance
According to the Encarta Dictionary (English: North America), "governance" is defined as having authority over something, controlling or restraining something, having influence over or being the law for something. Translating that to data governance, it means having authority over data, controlling or restraining data, having influence over or being the law for data.
Data ownership
If data governance is all about "authority," then identifying data owners is of paramount importance. Since data assets are owned by the organisation, senior business people who represent the organisation in a business process capacity inherently have authority to define business rules and business policies for data under their control. Thus, data ownership is about having authority over data assets.
Data stewardship
Data stewardship is about accountability. Data stewards are the business analysts and subject matter experts who are given the responsibility (accountability) for the quality of the data under their stewardship. They must be careful not to manage their data within the narrow focus of their own business unit (department or division); instead, they must ensure that their data is managed from an enterprise perspective so that it can be used and shared by all business units.
Enterprise information management
This (or EIM, in short) is about data administration. It is an IT function for maintaining, cataloging, and standardising corporate business data. This is done by establishing data-related standards, policies, and procedures, which are reflected in the enterprise data model and the business metadata.
Roles and responsibilities
Data governance is a practice that cannot be assigned solely to one specialised group, but it requires the collaboration between business people and IT technicians. The business people must participate in the roles of data owners and data stewards. The more technical data administration function is performed by a specialised group called Enterprise Information Management (EIM), which is typically staffed by a data administrator, enterprise data modeller, metadata administrator, and data quality analyst. Finally, data custodians in IT are the developers and database professionals who have the final responsibility for data integrity.
Data owner
Data owners are usually the business process owners, but they may also be the primary users of the data. In either case, data owners are senior business people who have the authority to set policies and make business rules for data elements. A data owner must specify what data can be accessed, when it can be used, and by whom. A data owner also should set the scheduled hours for when the data needs to be available, decide how frequently data needs to be updated, and specify data quality thresholds. He or she establishes business rules that will be coded into program edits, validation code, data cleansing code, DBMS specifications, and as metadata in the metadata repository.
Data steward
Data stewards should come from the business area as well, not from IT. The most important duty of a data steward is to continuously evaluate and improve the processes that contribute to data quality. Without data stewards, it would be very difficult to implement a data governance programme because data stewards have the responsibility to enforce the data policies and business rules established by the data owners, find and correct dirty data, resolve data disputes among business units, help to reconcile data from disparate systems, and ensure data integrity.
Data administrator
The purpose and benefits of data administration (DA) have not been well understood. As a result, many companies to do not have this function or confuse this function with database administration. The data administrator establishes and enforces DA principles, such as rules for data modelling, writing formal data definitions, creating fully qualified data names, documenting data domains, and creating standards for metadata.
Enterprise data modeller
An enterprise data modeller is usually a data administrator who has the responsibility to merge application-specific logical data models into one enterprise data model. This is an extremely important activity because many data collisions are not discovered until data names, data definitions, and data domains from different application-specific logical data models are merged together. The enterprise data modeller must resolve the data differences by working with the data stewards and the data owners on the business side.
Metadata administrator
The metadata administrator is responsible for maintaining the metadata repository. Metadata is collected in various tools, such as data modelling tools, OLAP tools, Excel spreadsheets, Word documents, DBMS tables, and so on. The metadata administrator is responsible to integrate and load metadata from these sources into the metadata repository. He or she is also responsible to deliver the metadata to business people and technicians through reports, online access, or a help function (wizard).
Data quality analyst
The most important responsibility of the data quality analyst is to collaborate with the data stewards and help them profile the operational source data to find data values that violate the business rules. The data quality analyst delivers data quality report cards to the data owners. He or she may also get involved in writing the programming specifications for data cleansing and helping the data stewards to identify the root causes of the data quality problems.
Data custodian
Any IT technician who touches data is considered to be a data custodian. This can be a database architect or DBA who is responsible for the physical databases, or it can be a developer who creates, updates, or deletes data from files and databases. A data custodian must be mindful not to introduce errors into his or her processes that might corrupt the data in the files and databases. Data custodians are also responsible for collecting technical metadata, process metadata, and data usage metadata, and helping the metadata administrator to populate these metadata components into the metadata repository.
Data governance organisation
If you are planning to institute a data governance program in your organisation, it is best to start with a readiness assessment to determine how your business people feel about treating data as a business asset.
Do your business executives accept that data has business value and should be managed like other assets that have business value? Are they willing to sponsor and fund a data governance programme in your organisation? Will they assign business people to be data owners and data stewards? Will they fund resources with appropriate skills to staff an EIM group?
Next, perform a data quality assessment across your critical systems. Share your assessment results with the business executives and establish goals for improving your data environment. That could mean launching a data quality improvement process, implementing master data management, eliminating redundant files or databases, creating official and ratified definitions for critical data elements and noting the exceptions, or creating a high-level enterprise data model.
In order to be effective, data governance responsibilities must be implemented at three levels, as indicated in Figure 1 - the Data Governance Organisation.
Figure 1 -- Data Governance Organisation
Strategic layer
Business executives should provide collective sponsorship for the data governance programme by assigning data owners, giving them the authority to set policy for the data assets in the organisation, and communicating their authority down through all layers of management.
The data owners in turn will assign data stewards in their business units and make them accountable for the quality and integrity of the data. The business people at the strategic layer can also be considered to be the data governance council.
Tactical layer
The data stewards must collaborate with the data administrators, the enterprise data modeller, the metadata administrator, and the data quality analysts from the EIM group. Together, they are responsible for creating the enterprise data model, managing the metadata, administering the lexicons abbreviations and class words, correcting dirty data, and so on. This layer contains the active workforce of a data governance programme.
Operational layer
The custodians implement and execute the data policies and business rules at the operational level. Database architects and DBAs implement the data policies and business rules in their database definitions and processes.
Developers and other maintenance technicians embed the data policies and business rules in their programs when they create, modify, move, copy, or delete data. It is also their responsibility to communicate and escalate any discovered data discrepancies to the data stewards and the EIM staff.
Conclusion
Implementing a data governance programme requires a new mental model regarding the business value of data and some organisational changes. These changes include new practices, disciplines, methods, applications, infrastructure, tools and techniques, roles and responsibilities, policies and procedures.
Making these changes must be systemic and holistic, not isolated and sporadic, which requires executive leadership committed to treating data as a business asset.
If you are interested in learning more about Data Governance, sign up for my upcoming Data Governance seminar. It is scheduled to be held on Nov 18, at the Grand Millennium Hotel, Kuala Lumpur. Contact Behlul Shaljani at (03) 7880-9894 or send e-mail to behlul@kbase.com for more imnformation.
(Larissa T. Moss is founder and president of Method Focus Inc. She has over 30 years of IT experience, with over 20 years in Data Warehousing and Business Intelligence.)
Related Stories:
Data governance, Part II