A Risk Management Approach to Improving Information Quality for Operational and Strategic Management Alexander Borek ([email protected]) Department of Engineering, University of Cambridge Philip Woodall Department of Engineering, University of Cambridge Ajith Kumar Parlikad Department of Engineering, University of Cambridge

Abstract Information has long been recognized as a key resource for every organisation. As it influences organisational success on every level of an organisation, its effects on objectives in operations and strategy need to be assessed and mitigated following a bestpractice risk management approach. This paper extends a process for Total Information Risk Management (TIRM) for managing risk that arises from poor quality information resources by incorporating a Ten Step approach for improving information quality. This approach guides managers through an effective information quality improvement programme that integrates best practices from the risk management and the information quality disciplines. Keywords: Information risk management, Information quality improvement, Information risk treatment Introduction Intangible assets, mainly in form of information and knowledge, are becoming increasingly important for a sustainable competitive advantage. While times are getting tougher, organisations need to cope with the intensifying pressure to adapt, innovate and speed up their processes quicker than their global competitors and therefore come to the realization that specialized knowledge, embedded in processes and routines in the organization, is the most effective differentiator when no other traditional market barriers exist (Klein 1998). This leads to a change of paradigm as the value of intangible assets often excels the value of tangible assets (Bontis 2001). Although knowledge is embodied, it can be spread when it is verbalized, articulated, and structured and thus transformed into information and saved as decontextualized data with the help of database technologies in a preset structure, context, and semantics (Tuomi 1999). Independent of its source, information can be of poor or of good quality, depending on its “fitness for use” (Wang and Strong 1996), a user-centred quality concept from the quality management literature (Juran 1988). There is an agreement amongst researchers and practitioners that information quality is a multi-dimensional concept by its very 1

nature and that accuracy is only one of many other equally important dimensions of information quality like accessibility, consistency, understandability, completeness or timeliness (Batini et al. 2009). Effective and efficient information flows are imperative to allocate information where it is needed in an organisation. It is further argued that the IT business value chain is strongly connected to information resources, its quality and the related risk induced by information resources (Borek et al. 2011). Poor information quality can bring huge risks to an organisation and even lead to serious disasters as exemplified in the cases of the explosion of the space shuttle Challenger and the shooting down of an Iranian Airbus by the USS Vincennes (Fisher and Kingma 2001). Studies have shown that poor information quality can have a negative impact on operational and strategic management, which can require information rework, cause significant process inefficiencies, spoil valuable resources, and lead to poorer decision making and lost future, e.g. (Redman 1998; Slone 2006; English 1999). These information-quality related risks, in the following called information risks, need to be managed accordingly in order to ensure a high organisational performance. Information risk, i.e. the business impact of information quality, can be assessed with a handful of techniques, as for example (English 1999; Loshin 2001; McGilvray 2008), which however do not provide an overall information risk management process and are not linked to the body of knowledge in risk management. A process for managing information quality-related risks should take into account the probabilistic nature of impacts, which implies a transfer of concepts from the risk management discipline. This paper illustrates how existing approaches to improving information quality (IQ) can be selected and used as part of a Total Information Risk Management (TIRM) process. First, we briefly present the TIRM process itself. Then, as the focus of this paper, we present how we developed a Ten Step approach to improve information quality within the TIRM process and describe each step in more detail. The TIRM process extended with the Ten Step approach gives managers a powerful tool for a riskoriented approach to improving information quality for operational and strategic management. A Process for Total Information Risk Management (TIRM) The TIRM process has been developed using the following research approach (Borek 2010), (1) an extensive literature review on relevant information risk literature and related topics, (2) interviews with operational, strategic and IT managers in a variety of industries, i.e. transport, utility, energy, steel manufacturing, pharmaceutical, semiconductor, chemical, (3) and interviews with management and IT consultants in some of these industries, (4) an in-depth action research study at a global manufacturer, where the TIRM process has been applied, and (5) analysis of the data and the lessons learned and integrating them with the current literature. An international widely recognised standard, ISO 31000, is used as a basis for the TIRM process, including its terminology, as this should increase the likelihood of acceptance, but also to make sure that it is based on current risk management best practice (International Organization for Standardization 2009a). Risk is defined by the ISO standard as the “effect of uncertainty on objectives“ (p.1). ISO 31000 provides general guidelines for risk management; in particular, it consists of a risk management framework and a risk management process. The TIRM process can be used within the ISO 31000 framework and follows the ISO 31000 risk management process stages, but also refines each process step to adjust the process specifically for managing information risks, as illustrated in Figure 1. In the following, we will give an overview of the main TIRM process stages based on ISO 31000. For the stages “communication 2

and consultation” and “monitoring and review” the existing stages of the standard can be applied.

TIRM Process 2 Establish the Context

3 Information Risk Assessment

3.1 Information Risk Identification 3.2 Information Risk Analysis 3.3 Information Risk Evaluation

5 Monitoring and Review

1 Communication and Consultation

based on ISO 31000

4 Information Risk Treatment

Figure 1 – TIRM process based on the ISO 31000 risk management process (International Organization for Standardization 2009a)

Establish the Context The external and internal context of the organisation needs to be established along with the context of the TIRM process. The external context can include “social and cultural, political, legal, regulatory, financial, technological, economic, natural and competitive environment, whether international, national, regional or local” (International Organization for Standardization 2009a, 15). The internal context can include governance, organisational structure, policies and objectives of the organisation, capabilities, standards etc. In particular, the current information management capabilties and practices are of high interest and can be assessed using an information quality management capability maturity model, e.g. (Baskarada, Koronios, and Gao 2007). Furthermore, the context of risk management, i.e. the context of the TIRM processes itself, needs to be determined and adapted to the current goals and practices in the organisation. As the last step of this stage, risk criteria need to be determined. The nature of risk considered in TIRM is risk that has its roots in information resources that are used in an organisation and can have a wide range of consequences. In general, all other definitions, e.g. likelihood, level of risk and risk appetite etc., should be taken from the risk management function in a given organisation, if existent, or should otherwise be defined under consideration of the general risk management best practice, see, for instance, (Hopkin 2010). Information Risk Assessment Information risk assessment consists of three sub-steps. First, information risks need to be identified in a defined scope. This requires understanding what information is required for a given process or activity and where information quality problems do 3

appear. Information quality can be assessed using existing information quality assessment methodologies, see (Batini et al. 2009) for a good overview of methodologies. The output is a set of information quality problems for each process or activity. Second, the risk of an information quality problem needs to be analysed. Risk analysis “involves consideration of the causes and sources of risk, their positive and negative consequences, and the likelihood that those consequences can occur“ (International Organization for Standardization 2009a, 18). This can be done qualitatively using significance scales like “low”, “medium, “high”, or quantitatively by taking estimates; an overview of risk analysis techniques is given in (International Organization for Standardization 2009b). Third, a thorough risk evaluation is required, taking the wider context of information risk management into account, e.g. the level of other information risks and their inter-relationships. In particular, it needs to be decided if an information risk should be treated, by comparing the level of risk with risk criteria from stage 2, “establish the context”, and the priorities of different treatments implementations have to be determined (International Organization for Standardization 2009a). Information Risk Treatment ISO 31000 describes risk treatment as “selecting one or more options for modifying risks, and implementing those options” (International Organization for Standardization 2009a, 18). Risk treatment is a cycle, which starts with the assessment of the treatment and evaluating if the remaining risk is not tolerable, in which case a new treatment is created and assessed. Certain treatment options may not require improvements to be made to IQ, such as avoiding the risk by not starting the activity that creates the risk. In contrast, other treatment options such as, for example, better informed decision making and changing the likelihood of a risk would often necessitate some level of IQ improvement. The following sections therefore describe a Ten Step process for improving IQ. Development of the Ten Step Process for IQ Improvement within TIRM Many of the information risk treatment options for TIRM involve an improvement of information quality. For these cases, we developed a Ten Step improvement methodology within TIRM that incorporates important aspects from existing improvement techniques. To develop this methodology, we (1) selected improvement techniques from the existing literature, then (2) we extracted activities from the existing techniques and (3) categorised them into improvement process steps and improvement options to create the Ten Step approach. In the following, we describe each research phase in detail. Selection of Improvement Techniques The existing IQ Improvement Techniques (ITecs) were obtained from a literature review by selecting studies (papers, books, reports etc.) based on the following selection criteria. Studies were selected if: • the study contains an ITec and describes what activities are involved • the study describes a IQ methodology and part of the methodology is an ITec • the study contains an ITec that has been subject to a rigorous review (as required by papers in high ranking journals or ITecs described in peer reviewed books)

4

•

The study contains an ITec that has been subject to an actual implementation and successful trial of the approach

Studies were rejected if: • the study does not describe an ITec and the activities in sufficient detail to enable a IQ assessor to clearly and easily implement the activities • the study only describes IQ assessment and not an ITec Table 1: Improvement Techniques (ITecs) IQ Improvement Technique Source name (McGilvray 2008) EDQP (Executing Data Quality Projects) (Batini and CDQM (Complete Data Quality Methodology) Scannapieco 2006) (Loshin 2001) COLDQ (Cost-effect Of Low Data Quality) (Redman 2001) DQFG (Data Quality Field Guide) (Pipino, Lee, and SODQA (Subjective-Objective Data Quality assessment) Wang 2002) (Wang 1998) TDQM (Total Data Quality Management) (English 1999) TQdM (Total Quality data Management)

These criteria ensure that any general process extracted or developed from the selected ITecs is implementable and is more likely to result in a successful improvement project. The final list of ITecs is presented in Table 1 with the name of the ITec shown in the first column and the source from where the ITec was taken shown in the second column. Extraction of Activities Activities are defined as the list of stages or phases contained in the ITec that are required to complete the IQ improvement project. The activities were extracted from the ITecs systematically and a distinct list of activities, where the common activities from different ITecs have been combined into a single activity, is shown in Table 2. Table 2: Information Quality Improvement Activities Activity Description Analyse data defect types To use previous data cleaning work to discover patterns of data errors, their frequencies, and the costs and impacts on the business. The output is a list of data defects as a list. Audit and control data Assure that the right data is extracted from the right files, properly extract, transformation transformed according to the transformation specification, and and loading loaded properly into the right fields in the target data source. Build a IQ team Select people who will manage and implement the improvement activities. Calculate derivations and Optimise data warehouse (or system) performance by determining summary data and storing derived data for the most frequently asked queries requiring complex calculations. Conduct a cost/benefit Develop a cost/benefit analysis using a prioritised list of analysis of improvement improvement options as a basis. This should take into account the options cost of the IQ improvement exercise and the costs of having poor data quality. Define a metadata model Define a metadata model. That is, what metadata will be used. 5

Define IQ rules Determine the actual meaning of the data

Determine what IQ rules currently exist Develop alternative data quality improvement options

Execute the improvement Identify data sources

Identify root causes of IQ problems Manage your suppliers Match and consolidate data Select processes or problems to focus on Select the most effective way to execute the improvement options Select tools for improvement Standardise data

Transform and enhance data into target Trial simple solutions to the IQ problem Verify the effectiveness of improvement actions

Define the rules to which data must adhere. These could be existing business rules. Determine the actual meaning of the data contrary to the definition of the data in its place of storage (e.g. the meaning of the heading of a field in a database). The objective is to determine the data that needs to be redefined and what new data needs adding (either in the source system or target system [if the data is being migrated]). Determine what IQ rules currently exist and to what extent are these rules are currently being followed. Develop alternative data quality improvement options/remedies. For example, an option might be to update the company database more frequently or distribute the updates to remote sites more often. Another option could be to perform data cleansing on the database at selected time intervals. These could be data-oriented or process-oriented approaches. Implement IQ improvement actions in a controlled manner to improve IQ. This may include the actual execution of software, or the initiation of actions to change business processes. Determine all pertinent data sources, files, specific data fields etc. that will be subject to data cleansing and/or files that contain the most reliable data. The most reliable data could be merged with other reliable data to generate a single source of truth. Identify root cause(s) of IQ problems (this may include prioritising these). Determine what rules will be imposed on external data providers (for example, a set of IQ expectations and penalties for nonconformance). Create a single authoritative electronic occurrence of reference to represent a single real-world object or entity (i.e. record linkage). Identify a process or a IQ problem that IQ improvements can focus on and are most likely to yield significant benefits if the data can be improved. Develop a plan, which outlines how the IQ improvement process will be conducted (taking into consideration all constraints such as time, cost, availability of resources etc.). Select suitable tools (e.g. software, or formal methods) for improvement. An example of software includes data cleansing tools. Standardise data into atomic values and standard formats. This enables data to be sharable (for example throughout an organisation), easily cleansed, and makes it easier for duplicate records to be identified and data to be consolidated. Map the corrected and consolidated data into a data warehouse (or other system) data architecture. This requires transforming any data from the data types, domain values, and formats into the respective data types, domain values, and formats in the target system. Identify simple solutions to IQ problems as a starting point and trial these with the aim of demonstrating that the trial solutions work. Verify that the selected IQ improvements do solve the problem.

6

Categorisation of Options and Processes The list of activities extracted from the ITecs were categorised into being directly related to the process of carrying out the improvement technique (referred to as ‘improvement process steps’), or options which may be included in the process but do not form a core process of the improvement technique (referred to as ‘ improvement options’). The list of improvement process steps and improvement options were independently reviewed to check for inconsistencies in the categorisation process, and the final list of processes and options are shown in Table 3 and Table 4 respectively. Table 3: Improvement Process Steps EDQP

Select processes or problems to focus on Build a IQ team Identify root causes of IQ problems Develop alternative data quality improvement options Select tools for improvement Conduct a cost/benefit analysis of improvement options Select the most effective way to execute the improvement options Trial simple solutions to the IQ problem Execute the improvement Verify the effectiveness of improvement actions

CDQM

COLDQ

TDQM

2

2 3

1

1

4

2

2

3

3

1 2

SODQA

1

1 2

DQFG

1

TQdM

1 2 3

3

3 4 5 3

5

4

4

6

5

4 5

Table 4: Improvement Options Identify data sources Determine the actual meaning of the data Standardise data Match and consolidate data Analyse data defect types Transform and enhance data into target Calculate derivations and summary data Audit and control data extract, transformation and loading Define a metadata model Define IQ rules Determine what IQ rules currently exist Manage your suppliers

Improvement process steps are the essential parts of every information quality improvement process, which show how generally to approach information quality improvement, while improvement options are specific options that can be – but do not 7

have to be - chosen within the improvement process depending on the context. The numbers in Table 3 show the ordering of the improvement process steps within each ITec. The Ten Step Approach proposed in the next section contains all process steps in Table 3 under consideration of which step needs to come before another based on the original ordering and taking into account the interdependencies between the steps, for example, when one step delivers the input for another step, it needs to come first. The Ten Step Process for IQ Improvement within TIRM The Ten Step Process shows what needs to be done in order to complete an IQ improvement project as part of the risk treatment stage of TIRM. Each improvement step (IS) is described in the following sections. IS.1. Select processes or problems to focus on The first step involves identifying a process or a IQ problem that IQ improvements can focus on and are most likely to yield significant benefits if the data can be improved. The process or problem comes directly from the information risk assessment stage of TIRM. IS.2. Build a IQ team Based on the problem/processes identified in step IS.1, relevant people with the knowledge and preferably direct involvement with the problem/process should be assembled in an IQ improvement team. IS3. Identify root causes of IQ problems The IQ team then meets to investigate and identify root causes of the problems using their knowledge of the problem and surrounding processes. IS.4. Develop alternative data quality improvement options Once the root causes have been identified, relevant improvement options for addressing the root causes should be determined. For example, an option might be to update the company database more frequently or distribute updates to remote sites more often. Another option could be to perform data cleansing on the database at selected time intervals. Other example improvement options that have been proposed in the ITecs are shown in Table 4. IS.5. Select tools for improvement If required, relevant tools can be purchased or developed to support the improvement options identified in step IS.4; example software includes data cleansing or data profiling tools. IS.6. Conduct a cost/benefit analysis of improvement options If multiple improvement options are required, a cost/benefit analysis should be conducted to select and prioritise the improvement options. This calculation should be based on the results from the information risk assessment stage of TIRM as estimates for the costs of having poor data quality. It should also take into account the cost of the IQ improvement exercises. IS.7. Select the most effective way to execute the improvement options This step requires the development of a plan that defines how the IQ improvement process will be conducted, which should take into consideration all constraints such as time, cost, availability of resources etc. 8

IS.8. Trial simple solutions to the IQ problem This step involves identifying simple solutions to IQ problems as a starting point and trial these with the aim of demonstrating that the trial solutions work. IS.9. Execute the improvement This step requires that the improvement options are executed in a controlled manner to improve IQ, such as, for example, running the software selected in step IS.5 or the initiation of actions to change business processes. IS.10. Verify the effectiveness of improvement actions Finally, as the last step, any improvement options executed in step IS.9 should be checked for their effectiveness regarding the set goal of each option and refined where necessary. This can be done by an iterative execution of the TIRM information risk assessment stage. The results of this verification are used to refine and improve the implementation of the TIRM process in an organization as part of the monitoring and review stage of TIRM. Information Risk Treatment Applications Information risk treatment is strongly dependent on the organisational context. In some of our case study companies, as for example in two manufacturing companies that we investigated, most of the information is communicated orally between humans. In these cases, improvement activities focus on documenting the knowledge and information, but also improving the communication processes between different departments. In the case of a vertically integrated energy company, a significant amount of information about the distribution network has been stored in databases, which makes it key to improve the current data, but also data entry, and to integrate and standardize different data sources. Conclusion Information risk arises from poor (or good in the case of opportunity risk) information quality that has a direct business impact on an organisation. To mitigate problems related to information risk, we have extended a process for Total Information Risk Management with a Ten Step approach for information quality improvement. This has been done using a (1) review of existing improvement techniques, (2) by extraction of the activities that are contained in the techniques and (3) categorisation of the activities. Our results can help managers to effectively navigate their path through information quality improvement in order to mitigate connected information risks and enhance organisational performance. Acknowledgments This research has been funded by EPSRC project “Information Quality in Asset Management”, reference number EP/G038171/1. References Baskarada, S., Koronios, A. and Gao, J. 2007. IQM-CMM: A Framework for Assessing Organizational Information Quality Management Capability Maturity. In Proceedings of the 12th International Conference on Information Quality, 317–332. Batini, C., Cappiello, C., Francalanci, C. and Maurino, A. 2009. “Methodologies for data quality assessment and improvement.” ACM Computing Surveys (CSUR) 41 (3): 16. Batini, C., and Scannapieco, M. 2006. Data quality: Concepts, methodologies and techniques. SpringerVerlag New York Inc.

9

Bontis, N. 2001. “Assessing knowledge assets: a review of the models used to measure intellectual capital.” International Journal of Management Reviews 3 (1): 41–60. Borek, A. 2010. The Business Impact of Information Quality in Asset Management. First Year Report, Cambridge, UK: University of Cambridge, August. Borek, A., Helfert, M., Ge, M. and Parlikad, A.K.N. 2011. An information oriented framework for relating IS/IT resources and business value. In Proceedings of the International Conference on Enterprise Information Systems (ICEIS). Beijing, China. English, L. P. 1999. Improving data warehouse and business information quality: methods for reducing costs and increasing profits. John Wiley & Sons. Fisher, C. W, and Kingma, B.R. 2001. “Criticality of data quality as exemplified in two disasters.” Information & Management 39 (2): 109–116. Hopkin, P. 2010. Fundamentals of Risk Management: Understanding Evaluating and Implementing Effective Risk Management. Kogan page. International Organization for Standardization. 2009a. ISO 31000:2009 Risk Management – Principles and Guidelines on Implementation. International Organization for Standardization. 2009b. ISO/IEC 31010:2009 - Risk management -- Risk assessment techniques. Juran, J.M. 1988. Quality Control Handbook. 4th ed. McGraw-Hill Inc.,US, September. Klein, D.A. 1998. The strategic management of intellectual capital. Butterworth-Heinemann. Loshin, D. 2001. Enterprise knowledge management: The data quality approach. Morgan Kaufmann. McGilvray, D. 2008. Executing Data Quality Projects: Ten Steps to Quality Data and Trusted Information. Morgan Kaufmann, August 22. Pipino, L.L, Lee, Y.W. and Wang, R.Y. 2002. “Data quality assessment.” Communications of the ACM 45 (4): 211-218. Redman, T.C. 1998. “The impact of poor data quality on the typical enterprise.” Communications of the ACM 41 (2): 79-82. Redman, T.C. 2001. Data quality: the field guide. Digital Pr. Slone, J.P. 2006. Information quality strategy: An empirical investigation of the relationship between information quality improvements and organizational outcomes. Doctoral Thesis, Minneapolis, MN, USA: Capella University, October. Tuomi, I. 1999. “Data Is More Than Knowledge: Implications of the Reversed Knowledge Hierarchy for Knowledge Management and Organizational Memory.” Journal of Management Information Systems 16 (3): 103-117. Wang, R.Y. 1998. “A product perspective on total data quality management.” Communications of the ACM 41 (2): 58-65. Wang, R.Y., and Strong, D.M. 1996. “Beyond accuracy: What data quality means to data consumers.” Journal of management information systems 12 (4): 33.

10