QUALITY INITIATIVE TASK FORCE SYSTEMS COMMITTEE INTERIM STATUS REPORT

March 24, 2016 Last Revised: May 11, 2016

EXECUTIVE SUMMARY The Quality Initiative Systems Committee (“QI Systems”) met regularly during the 2015/16 academic year to plan, execute, and document progress towards its objectives. Major achievements include: ●

Adoption of high-level enterprise system architecture diagram that is compliant with the phase one, UALR Interim Decision Support Structure, data warehouse implementation; ● Staff skill-set assessment for Information Technology Services (“IT Services”) and the Office of Institutional Research (“OIR”); ● Development of preliminary data classification framework, data quality findings, and data quality reports; ● Automation of common administrative and regulatory data reports; ● As an extension of the interim decision support system, development of a preliminary operational data store (ODS) that maintains daily enrollment data along with certified snapshots of historical regulatory data; ● Continued development of Degree Works, with an estimated completion of Spring 2017 for use on or before Fall 2017; and ● Research and findings from two onsite vendor data warehouse and business intelligence visits that may be used, in part, to develop a request for proposals (RFP) in compliance with Act 557 of 2015. In collaboration with the QI Analytics Committee (“QI Analytics”), the following milestones were accomplished: ● ● ●

Development of a preliminary data dictionary; Proposal of a data governance framework; and Implementation and “go-live” of Civitas Illume; including two online trainings and two on-site trainings in March 2016. In addition to its milestones, QI Systems provides the following recommendations as it continues implementation of a decision support system at UALR: 1. Upon approval by university leadership, collaborate with the UALR Office of Procurement to develop a RFP for a phase two data warehouse and business intelligence implementation to support the UALR Ideal Decision Support System Structure. 2. As an extension of the interim decision support system, continue development of the ODS that maintains daily enrollment data along with certified snapshots of historic regulatory data. 3. Expand the data dictionary to include a business glossary and data elements beyond those required for state and federal reporting. This effort should solicit input from the campus community to ensure consensus on key definitions. 4. Formalize a data governance subcommittee, consisting of members from the QI Task Force, charged with enhancing the data governance framework through development of data policies. 5. Consider the use of Civitas Illume, focusing on adoption at the program, department, and college levels. 1

TASK FORCE COMMITTEE MEMBERSHIP ANALYTICS COMMITTEE Anindya Gosh Ann Bain (Chair) Ashekul Huq Belinda Blevins-Knabe Brad Patterson Robert Corwyn Robert Mitchell Russel Bruhn Stephanie Farewell Theresa Beiner SYSTEMS COMMITTEE Ashekul Huq Cody Decker (Chair) Eduardo Garcia John Rathje (Chair) John Talburt Stephanie Farewell Sung-kwan Kim





2

TABLE OF CONTENTS Executive Summary ...................................................................................................................................... 1 Task Force Committee Membership ............................................................................................................ 2 Introduction ................................................................................................................................................. 5 Data Warehouse Industry Overview ............................................................................................................ 6 Data Quality ................................................................................................................................................. 7 Data Quality Assessment & Root Cause Analysis ..................................................................................... 7 Completeness ....................................................................................................................................... 7 Conformity ........................................................................................................................................... 7 Consistency .......................................................................................................................................... 8 Operational Process Considerations ........................................................................................................ 8 Data Dictionary and Business Glossary .................................................................................................... 9 Staff Skill-Set Analysis ................................................................................................................................. 10 IT Services Skill-Set Analysis ................................................................................................................... 10 OIR Skill-Set Analysis .............................................................................................................................. 10 Information Technology ............................................................................................................................. 11 Operational Data Store & Automated Reporting ................................................................................... 11 Degree Audit Implementation ............................................................................................................... 12 Civitas Illume Implementation ............................................................................................................... 12 Infrastructure Assessment ......................................................................................................................... 13 Solution Cost .......................................................................................................................................... 15 Conclusion .................................................................................................................................................. 16 Appendix I: High Level Data Warehouse Architecture Diagram ................................................................. 17 Appendix II: On-site Data Warehouse Vendor Invitation ........................................................................... 18 Appendix III: QI Systems Vendor Questions ............................................................................................... 20 Appendix IV: Recent Data Warehouse Industry Advances ......................................................................... 22 Industry Advances and Changes ............................................................................................................. 22 Shift to the Cloud ............................................................................................................................... 22 Dell Purchase of Statistica .................................................................................................................. 24 Microsoft Cortana Analytics Suite & Azure Machine Learning ........................................................... 25 Machine to Machine Learning: (Commentary from SAS ) .................................................................. 25 Deep Learning .................................................................................................................................... 25

3

Perceptual Intelligence ....................................................................................................................... 25 UALR Updates ......................................................................................................................................... 25 Blackboard Analytics Addition to UALR Portfolio. .............................................................................. 25 New Perspectives ............................................................................................................................... 25 Appendix V: Vendor Analysis from Gartner ............................................................................................... 27 Gartner Magic Quadrant for Advanced Analytics Platforms .................................................................. 27 Magic Quadrant for Business Intelligence and Analytics Platforms ....................................................... 28 Appendix VI: OIR & IT Services Skill-Set Analysis ........................................................................................ 32 Appendix VII: Data Warehouse Deployment Options ................................................................................ 36 Appendix VIII: Data Warehouse TCO Model .............................................................................................. 37 Appendix IX: Total Cost of Ownership Survey ........................................................................................... 41



4

INTRODUCTION QI Systems collaborates with QI Analytics as the two primary committees of the QI Task Force. QI Systems focuses on the data, information technology and the personnel components of a decision support system whereas QI Analytics works towards outcome measures, dashboarding, and reporting needed for operational and strategic planning at UALR. QI Systems made significant progress towards its objectives during the academic year. Many of these achievements, such as automated reporting built on the interim reporting system, Argos, and the preliminary data dictionary, have immediate benefits to the campus community. Other achievements, such as the two on-site vendor demonstrations, are necessary milestones towards the launch of phase two, Ideal Decision Support System, at UALR. The status report is divided into five sections: readiness, data quality, information technology, and system implementation. The area of governance was collaborated on with QI Analytics and will be represented in its report. Each section of this report contains one or more of the fourteen charges assigned to QI Systems along with a status update and recommendation for progressing the remaining deliverables. Due to the technical nature of the QI Systems, a substantial amount of the committee's findings are provided in supplementary appendices to the interim status report. This report summarizes the planning, execution, and key findings for the QI Systems from July 2015 through March 2016. It is intended to serve as a comprehensive status update for the majority of the committee’s efforts for the academic year. The report will be updated at the conclusion of the academic year.





5

DATA WAREHOUSE INDUSTRY OVERVIEW An objective of QI Systems was to evaluate the gaps between the current systems and an ideal decision support system at UALR. An essential component of this analysis includes an overview of the data warehouse industry. QI Systems approved a high-level architecture diagram for discussion with stakeholders and industry leaders. The diagram displayed in Appendix I contains the core components of the interim data warehouse solution, including the extract, transform, and load (ETL) processes by which data is moved from source systems (e.g. Banner) to a data warehouse environment. The data warehouse environment is represented as a collection of systems (operational data store and certified data store) and processes (validation and certification). The diagram also represents sample data outputs, such as administrative, ad hoc, and regulatory reports. In support of the charge to assess data warehouse vendors and analyze industry solutions, QI Systems invited vendors to demonstrate and discuss data warehouse approaches for UALR. The formal invitation is included in Appendix II. Specifically, the actions related to the on-site visits by the committee included: 1. Formally requesting an on-site demonstration and discussion with IBM and SAS; 2. Providing a list of questions that should be addressed during the presentation and discussion; and 3. Sharing with the vendors artifacts produced by QI Systems intended to facilitate discussion and inform stakeholders regarding the data warehouse implementation and scope. Participants were invited from the campus community, including the Deans Council (and their designees), QI Analytics, Enrollment Management, Finance and Administration, and the Alumni Association, to participate in the on-site vendor visits. A survey administered by the College of Business (COB) was used to collect feedback from the campus community on each vendor’s presentation, ETL and data warehouse tools, data governance, reporting/analytic capabilities, and overall solution. QI Systems found that the data warehouse industry has continued to progress over the last decade, and in particular, experienced significant disruption in the last three years in data warehouse technologies. The initial charge of QI Systems, to identify the top two vendors which address UALR’s data reporting and warehouse needs, are respected throughout this document. Additional information and insight are offered for consideration to compliment initial findings. These comments are offered not to diminish the work of the committee, but extend options in light of industry changes, new UALR developments, and recently changed procurement statutes. Appendix IV offers summarization on recent industry advances and relevant changes at UALR while Appendix V provides analysis of leading vendors based on detailed findings from Gartner, an established research and advisory firm providing information technology-related insight. Act 557 requires sub-committee review of any purchase of $50,000 for professional and consultant services, legal review and $100,000 for technical services. State contracts for services required may be used in lieu of a RFP, however, review is still required. This change in procurement legislation is an

6

important component of this report and will influence the path chosen by UALR to implement its data warehouse solution.

DATA QUALITY DATA QUALITY ASSESSMENT & ROOT CAUSE ANALYSIS As part of QI Systems’ on-going work to improve the quality of data, a list of data quality issues were compiled. This will be an iterative process; however, to date, seventy-nine issues have been identified. The majority of issues relate to data conformity and completeness. As an example, a field that commonly presents data quality problems is the social security number (SSN). For some records, the SSN is found to be different for the same individual in different tables. The following four issues have been shortlisted, each in one data quality dimension for further analysis:

COMPLETENESS Completeness means that certain attributes should be assigned values in a data set. Completeness rules can be assigned to a data-set in various levels of constraints: ● ● ● ●

Mandatory fields that require a value (i.e., Student Name) Optional attribute that may have a value based on some condition (i.e., SSN should be nine digits in length) Inapplicable attributes which may not have a value (i.e., Spouse name for an unmarried person) Null values that may indicate some information

As an example of the completeness problem, consider the following data quality challenge. In the Banner table titled “SGBSTDN”, the fields sgbstdn_major_code_1 and sgbstdn_major_code_2 identify the first major and second major of a student. It is challenging to identify the major of a student as the major_code_1 and major_code_2 are absent for some records. It is expected that a student will have a second major only if he/she has a first major. For some records, the first major is missing and second major is present.

CONFORMITY This dimension refers to whether data values conform to a specific format and column metadata attributes. A column has various metadata attributes associated with it: data type, precision, format pattern, a predefined enumeration of values, domain ranges, storage formats, etc. Consider the following conformity data quality issue with individuals considered international students. The Banner tabled “GORVISA_VISA_START_DATE” represents the visa start date of a student. This field helps to determine the student's residency status. There are nulls and impossible dates that cause incorrect reporting. One data quality challenge is determining the status of a foreign national. There is no direct approach or a single attribute that determines whether a student is an international student. Though there is a field titled “SPBPERS_CITZ_CODE” that determines whether a student is a citizen. The attribute SPBPERS_CITZ_CODE is a two-character text code identifying a person’s citizenship. Examination of the 7

structure of the database would imply that this field is intended to record a country code identifying a person’s country of citizenship – based on the Banner “STVCITZ” validation table. However, due to current configured use of the “STVCITZ” attribute, this field is in effect as a Boolean Yes/No to indicate citizenship.

CONSISTENCY Consistency of data means that data across the system is appropriately synchronized. There is no conflicting information for a given instance. The most notable inconsistency challenge is with the SSN. In Banner, SSN field is found in various tables. It is noted that SSN is missing in a field while for the same record it is available in some other field (i.e. SSN field is not consistent across the database). The following fields represent SSN information in the Banner database: ● ● ● ●

SWCTEMPS_ID SPBPERS_SSN SPRIDEN_ID SAVADMT_SSN

On analyzing the above fields, it is noted that the column SPRIDEN_ID is inconsistent. Within the SPRIDEN_ID field, the SSN is often transposed by one digit. In other instances, Banner houses the SSN in fields labeled SWBADTS_SSN, GWBFDUP_SSN, SHBHEAD_SID_SSNUM, SRTPERS_SSN. A missing SSN is the most frequent data quality issue seen in these fields.

OPERATIONAL PROCESS CONSIDERATIONS An important element of QI Systems is its work related to data quality. In collaboration with OIR, the common perceptions of data quality were evaluated. A fundamental challenge was discrepancies on Argos reports used by faculty and staff. To address this challenge, a preliminary data classification framework is proposed. Administrative and academic units at UALR produce a number of data reports. As an example, the OIR produces an estimated 2.2 reports per business day. Reports may have similar names but represent different data. For example, enrollment reports may be based on data sourced directly from the student information system, Banner; from regulatory data previously submitted to the Arkansas Department of Higher Education (ADHE); or from a combination of transactional and regulatory data. A novel classification framework was reviewed by QI Systems and presented to the UALR Chancellor’s Leadership Group (CLG) to help address this problem. There are a number of dimensions of information quality. These dimensions have been categorized as intrinsic, contextual, representational, and accessibility dimensions of information quality (Lee, Strong, Kahn, & Wang, 2002). The proposed data classification framework seeks to improve the actual and perceived quality of reports by addressing the representational dimensions, including understandability, interpretability, concise representation, and consistent representation. Data reports often contain similar data elements, definitions, and formatting; however, the data values may differ due to the business rules or source data used to develop the reports. These classification

8

standards do not address privacy nor access controls, as these considerations will be addressed in other governance policies. The first data classification type proposed is live data. Reports commonly consist of data sourced directly from an information system. The most prominent information system at UALR is Banner. Live data is the most likely data set to contain errors, as the data may not have been reviewed, cleaned, validated, or certified. Live data is commonly known as preliminary, transactional, uncertified, realtime, Banner, and unofficial data. OIR proposes to classify data sourced directly from an information system without a validation or certification process as “live data.” Certain “snapshots” of live data are considered particularly useful. As an example, consider census day data pulled live from Banner as of the eleventh day after the start of the fall semester. In these special instances, the data will be classified as live data and a designation shall be appended to the label to indicate the reason for the snapshot (i.e., Live Data-Census Day Snapshot). Furthermore, reports containing live data shall make use of a date or date and time stamp to indicate the date/time data was pulled from an information system. The second data classification type proposed is official data. A data validation and certification process is usually part of preparing official data, such as data loaded into a data warehouse. Official data is used to comply with regulatory reporting, such as certified reports to the ADHE or the Integrated Postsecondary Data System (IPEDS). A key characteristic of official data is its pre-processing tasks that include review, cleaning, and validating the data as complete and correct. OIR proposes that official data submitted to a regulatory body, such as ADHE or IPEDS, is further appended with a “regulatory” designation. The third data classification type proposed is blended data. Blended data represents the most common reports and represent a hybrid of data obtained directly from an information system (live data) as well as official data. Live, official, and blended data designations may be used to classify the three primary types of data used on reports. These designations should be used as an interim solution until formal data governance policies are implemented at UALR. However, by proposing a simple framework for the classification of data used on reports, OIR will bring awareness to the different classifications of data used for institutional decision-making and reporting.

DATA DICTIONARY AND BUSINESS GLOSSARY An objective of the Quality Initiative Task Force and a prerequisite of the data warehouse implementation, is to document key data definitions and metrics at UALR. This work is particularly important because it will determine the data available in the data warehouse, which will be used to produce analytics and reports for administrators, faculty, and staff. According to IBM’s Dictionary of Computing, a data dictionary is “a centralized repository of information about data such as meaning, relationships to other data, origin, usage, and format.” A data dictionary defines low-level data definitions (e.g., Student Name, Ethnicity, Birthdate, etc.), whereas a business glossary defines high-level definitions (e.g., Enrollment, Headcount, StudentSemester Credit Hours, etc.) derived from data elements in the data dictionary. A business glossary

9

differs from a data dictionary in that it stores definitions derived from more granular data found in the data dictionary. In order to expedite delivery of the data dictionary and business glossary, OIR documented the data elements needed to accommodate state and federal reporting. It is organized into categories, such as a course, financial aid, instructor, student, and registration. The data dictionary should be considered substantially complete for regulatory reporting. QI Systems, in collaboration with QI Analytics, anticipates an iterative, transparent approach that ensures feedback from the campus community is collected and synthesized as the data dictionary and business glossary is developed. The dictionary should now be expanded to include definitions required for administrative, ad hoc, and institutional reporting. QI Analytics and QI Systems plan to release a joint invitation for administrative units to review and contribute to the data dictionary and business glossary during Spring 2016. A similar process is proposed for academic units.

STAFF SKILL-SET ANALYSIS QI Systems recognizes that personnel are the most critical aspect of a data warehouse implementation. In acknowledgment of this component, a review of skill-sets in both IT Services and OIR was conducted. In order to assess potential strengths, weaknesses, and opportunities regarding skill-sets, the two offices conducted an extensive self-assessment to ascertain 1) skill level, 2) ability to apply relevant skills, and 3) interest in specific skill areas. Staff were assessed on skill-sets related to business intelligence, data analysis, data modeling, including ETL technologies, data governance, programming, and project management. The preliminary findings from the skill-set assessment indicate that professional development will be necessary to support a decision support system at UALR. It is unlikely that existing personnel could independently implement a solution; however, the analysis revealed a number of existing skill-sets that could be supplemented with external expertise to maintain the solution once properly implemented. A separate analysis is provided for IT Services and OIR.

IT SERVICES SKILL-SET ANALYSIS The IT Services skill-set analysis indicated staff will require training to administer and develop integrations required in a new decision support system. IT Services staff have not engaged in professional development in recent years. IT Services staff have the aptitude and certain relevant skills which provide confidence that UALR will not need long term staff augmentation. However, implementation will require professional services and should be retained to support initial solution configuration and performance tuning. Current IT Services staff loads will be adjusted to gain necessary support insight and the IT Services reorganization plan identifies critical skill-sets for solution sustainability.

OIR SKILL-SET ANALYSIS Similarly, OIR has not recently engaged in professional development opportunities. OIR staff reported the highest skill-sets in data analysis and interpretation with Microsoft Access, Microsoft Excel and 10

Structured Query Language (SQL) and data reporting with Microsoft SQL and Argos. Staff reported limited skill-sets with HTML, Microsoft.NET and Java technologies. OIR staff also indicated familiarity with relational database technologies; however, skill-sets were reported low regarding ETL technologies (irrespective of technology vendor). OIR staff were also asked to assess their level of interest in different areas, irrespective of current skillset. The most interest was expressed in areas related to business intelligence, data analysis and project management. There was also interest in data governance, programming, and data modeling. In order to address skill-set weaknesses, and provide staff an opportunity to further develop their skillsets, an online training subscription was purchased for OIR staff. Additionally, the analysis from the skill-set assessment will be used to inform hiring procedures and utilization of professional development opportunities for existing staff. The summarized results from the IT Services and OIR skill-set analysis are provided in Appendix VI.

INFORMATION TECHNOLOGY OPERATIONAL DATA STORE & AUTOMATED REPORTING QI Systems approved a high-level enterprise system architecture diagram that is compliant with the phase one, UALR Interim Decision Support Structure, data warehouse implementation. This visual is displayed in Appendix I. An update was later drafted to incorporate the preliminary data classification framework. The update is provided in Figure 1. Figure 1: High-Level Enterprise System Architecture Diagram with Data Classification



11

This visual serves as an important conversation piece with the campus community and potential implementation vendors about data systems at UALR. It highlights the data and IT governance components as well as the potential for IT-approved data reporting interfaces. In the collaboration between IT Services and OIR, a prototype ODS was implemented to support an interim decision support system at UALR. OIR populated student and course tables within the ODS as well as regulatory data based on data previously submitted to the ADHE. Using the newly implemented ODS, OIR now retains daily student headcount and student semester credit hour (SSCH) data. This architecture is being used to retain daily snapshot data as well as automate reports using Argos and Microsoft SQL. Based on its progress to date, QI Systems recommends that this work continue. A joint project between IT Services and OIR, with oversight provided by QI Analytics and QI Systems, should be used to continue populating the ODS with student, course, employee, instructor and financial aid data, based off data included in the data dictionary. It should then be expanded to include data identified by QI Analytics as well as input from the campus community on the business glossary. The work is limited by resources currently available in IT Services and OIR; however, it may be expedited and enhanced once a vendor is engaged through a RFP to implement the ideal decision support system at UALR.

DEGREE AUDIT IMPLEMENTATION Degree Audit services are essential to student success and meaningful progression toward graduation. Ellucian Degree Works is a comprehensive academic advising, transfer articulation, and degree audit solution that aligns students, advisors, and institutions to a common goal: helping students graduate on time. Ellucian Degree Works captures valuable aggregate information, providing administrators with important metrics for planning future term course offerings, streamlining processes for more efficient cost management, and keeping pace with higher education standards. In particular, following an Ellucian Action Plan study, the solution will address specific issues discovered, including: ●

Set up separate grading for each graduate program, eliminating the need to manually separate grades in Banner. ● Set up curriculum plans so students always know requirements and electives for their program to facilitate planning. The solution also includes National Collegiate Athletic Association (NCAA), financial aid, and veteran affairs modules that will assist administrators as they work to improve student retention. This aggregate information, along with other key data elements can be made available to the UALR defined reporting solution in support of student success measures. The project is projected to be completed in the Spring 2017 and available for Fall 2017.

CIVITAS ILLUME IMPLEMENTATION According to Civitas, “Illume is a powerful application, giving institutions unprecedented ability to dive deep into their data and surface insights and unseen correlations. This level of data access provides profound context for understanding the dynamics of student characteristics, policies, and institutional initiatives on student persistence and success – all in real-time.” The implementation of Illume was accelerated during Fall 2015. 12

IT Services provided a ten-year extract of de-identified data to populate Illume. Designees from the offices of admissions, financial aid, graduate school, institutional research, and records and registration participated in a data validation session to confirm the completeness and correctness of data populated in Illume. The validation sessions concluded during December 2015. Two online training sessions in December 2015 were also provided to members of the campus community. Illume became operational during March 2016 with the launch of two one-site power user sessions. Illume will be expanded during Spring 2016 to account for additional filters at the college, department, and degree levels. Additional collaboration among faculty and staff will be necessary with the continued roll-out of Illume across campus. QI Systems recommends, as part of the data governance policies, that consideration be given to the access, security, and use of data within Illume. Default functionality provides no personally identifiable data; however, the reporting tools enable data to be displayed for cohorts that contain relatively small numbers of students, potentially enabling the data to become unintentionally identifiable.

INFRASTRUCTURE ASSESSMENT Technical features determine how well a potential data warehouse solution can satisfy UALR’s requirements and how effectively it can fit in with the existing information systems and infrastructure. The technical features for the UALR data warehouse system selection include both front-end and backend utilities. The front-end services are responsible for delivering data to the user community such as browsing, querying and reporting data. The front-end utilities include display interface, access tools and query functionality. The back-end services are responsible for gathering and preparing the data and managing the data warehouse system. For the front-end utilities, there are no significant issues anticipated. For the back-end services, UALR will need servers for hosting the data warehouse storage and also for managing the data warehouse (e.g., data quality checks, ETL functionality, metadata management and data warehouse administration). The quantity and capacity of the servers are yet to be determined. Regarding the business intelligence platform, the inputs are required from QI Analytics for the assessment of the analytics components. Depending on the final solution and implementation model, infrastructure needs will be driven by the final solution and implementation model. The following table summarizes the possible models (Hosted in the Cloud, Managed Host solution, On-Premise). More specific requirements will be based on the solution selected by UALR through a formal RFP solicitation.



13

Table 1: Data Warehouse Hosting Options

Cloud

Managed Hosting

On-Premises

Network (internal)

Requires Attention1

Requires Attention1

Requires Attention1

Network (Internet)

No Issues

No Issues

No Issues

Authentication

Requires Attention2, 3

Requires Attention2, 3

Little/No Issue3

Storage

N/A

N/A

Requires Attention4

Server

N/A

N/A

Requires Attention5

Security

Requires Attention6,7

Requires Attention6,7

Requires Attention6,7

Disaster Recovery

Little/No Issue

Little/No Issue – contract and SLA dependent

Requires Attention8

Data Center

N/A

N/A

Requires Attention9

Desktop/End Point

Assessment Required10

Assessment Required10

Assessment Required10

Cost model

Significant Operational

Significant Operational, 1X investment

1X investment, small incremental operational investment

1 – Network switches and wireless access points are end-of-life. Work is underway to replace and update network infrastructure. This is a global issue, not data warehouse specific. 2 – Secure authentication integration will need to be established that leverage current UALR identity. 3 – UALR should move to a role-based identity management system to ensure access to sensitive data is respected from a transitional system (system of record) to other systems, such as a reporting solution. 4 – UALR storage is at capacity. The new Director of Technology Infrastructure is developing a mid and long-term storage strategy for UALR that will account for data repositories, reporting, rich media and other needed storage solutions on campus.

14

5 – UALR server implementation is under review to become more cost effective and better managed. The server solution (in part) is dependent on an updated storage solution. 6 – UALR is in need of a modern Firewall for proactively managing intrusion detection and intrusion prevention. UALR should also consider DLP (data loss protection) as an enterprise solution to manage movement of sensitive data elements. 7 – UALR should address desktop management to protect data on desktops and intellectual property on workstations/workforce end points. 8 – UALR business continuity should be documented to ensure current investments meet data reporting/repository expectations. 9 – UALR’s current data center is not well positioned as a data center. Risks within the environment exist. 10 – The client software (presentation of data and reports) is a critical component that may require modern browsers and/or modern computing devices to render the display of generated reports.

SOLUTION COST Full solution cost and total cost of ownership (TCO) will not be fully known until a solution is identified and procured through a RFP. Research within the industry, and in particular with Garter, demonstrates that until a solution and method of deployment are identified, TCO will be limited to estimates. Multiple deployment models are available, including on-premise, managed hosting, infrastructure as a service (IaaS), and data warehouse platform as a service (dwPaaS). A brief definition of each is discussed in Appendix VII. A cost model for each deployment method has been drafted to help UALR predict costs of ownership (Appendix VIII) and additional reference information, based on a survey of over 2000 organizations, is provided for further information regarding potential costs (Appendix IX). Numbers provided should be used as estimates only. Depending on the solution path UALR determines, current investments may influence final costs. QI Systems focused on two solution providers to determine feasibility and quality of industry solutions. The two solutions reviewed (IBM and SAS), provide initial estimates for licensing and implementation costs ranging from $400,000 - $600,000. Ongoing costs will be a function of implementation model, growth and required supplemental labor. A more formal TCO model is under development and can be referenced in Appendix VIII.



15

CONCLUSION In partnership with QI Analytics, QI Systems made significant progress towards its objectives during the first nine months of the academic year. These achievements include objectives that have immediate benefits to the campus community. Other achievements, such as findings from on-site vendor visits, will assist the Quality Initiative Task Force as its moves towards implementation of the Ideal Decision Support System at UALR. Notable Milestones Progressed during 2015/16 ● ●

● ● ● ● ● ● ●

Adoption of high-level enterprise system architecture diagram that is compliant with the phase one, UALR Interim Decision Support Structure, data warehouse implementation; As an extension of the interim decision support system, development of a preliminary operational data store (ODS) that maintains daily enrollment data along with certified snapshots of historic regulatory data; Automation of common administrative and regulatory data reports; Continued development of Degree Works, with an estimated completion of Spring 2017 for use on or before Fall 2017; Development of preliminary data dictionary, data classification framework, data quality findings and data quality reports; Implementation and “go-live” of Civitas Illume, including two online trainings and two on-site trainings in March 2016; Proposal of a data governance framework; Research and findings from two on-site vendor data warehouse and business intelligence visits that may be used, in part, to develop a RFP in compliance with Act 557 of 2015; and Staff skill-set assessment for IT Services and OIR.

QI Systems proposes to continue working on the charges identified by the Office of the Provost for the remainder of the academic year. However, to guide the work, the committee offers the following supplemental charges to focus its efforts through June 2016. ● Upon approval by university leadership, collaborate with the UALR Office of Procurement to develop a RFP for a phase two data warehouse and business intelligence implementation to support the UALR Ideal Decision Support System structure. ● As an extension of the interim decision support system, continue development of the ODS that maintains daily enrollment data along with certified snapshots of historic regulatory data. ● Expand the data dictionary to include a business glossary and data elements beyond those required for state and federal reporting. This effort should solicit input from the campus community to ensure consensus on key definitions. ● Formalize a data governance subcommittee, consisting of members from the QI Task Force, charged with enhancing the data governance framework through the development of data policies. ● Consider use of Civitas Illume, focusing on adoption at the program, department, and college levels. 16

APPENDIX I: HIGH LEVEL DATA WAREHOUSE ARCHITECTURE DIAGRAM





17

APPENDIX II: ON-SITE DATA WAREHOUSE VENDOR INVITATION TO: FROM: DATE: RE:

On-Site Data Warehouse Demonstration and Discussion

The University of Arkansas at Little Rock (UALR) Quality Initiative (QI) Systems Committee (Committee) invites vendor demonstrations and discussion from [Vendor] regarding a data warehouse solution for UALR. This memo is intended to serve the following purposes: 1) Formally request on behalf of the Committee an on-site demonstration and discussion with your company; 2) Provide your company with questions posed by the Committee that should be addressed during the presentation and discussion; and 3) Share artifacts produced by the Committee (or its designee) intended to facilitate discussion and inform your final proposal regarding the data warehouse implementation and scope. The Committee requests an on-site demonstration and discussion the week of November [date], 2015. Alternatively, the week of November [date], 2015 is available as a backup. The meeting will take place onsite at UALR. The Committee is also available for a virtual session (pre-meeting) during the week of November [date], 2015. A projector will be provided for the meeting. The vendor should bring a laptop for demonstration purposes as well as any necessary handouts. Please forward any requirement for Internet access you or your team may have during the day of your presentation. The Committee consists of technical and non-technical members. Committee members participated in prior demonstrations that focused on business intelligence, including analytics, dashboards, reporting, and to a lesser degree on the fundamentals of the data warehouse implementation. The Committee respectfully requests the discussion focus on the implementation details regarding the data warehouse, including the software, hardware, and technical services needed to implement the data warehouse. The proposed data warehouse implementer (integrator) should be included in the demonstration and discussion. The Committee seeks details regarding the technology and processes necessary to extract data from UALR’s primary student information system, Ellucian Banner, and populate a data warehouse with data elements identified in UALR’s data dictionary. As an interim solution, UALR intends to use its existing business intelligence tool, Ellucian Argos. Business Intelligence requirements and tools will be discussed and evaluated at later phases of this effort. 18

Three artifacts are enclosed for the vendor’s consideration and use for the on-site demonstration and discussion: ● A set of questions identified by the Committee that should be addressed during the vendor demonstration and/or discussion. ● A high-level system architecture diagram adopted by the Committee. The diagram contains a number of source systems; however, the primary source system will be Ellucian Banner for the initial data warehouse implementation. ● A draft data dictionary prepared by the UALR Office of Institutional Research. The draft data dictionary will be expanded as additional data elements are identified for administrative and regulatory reporting. A business glossary will be added at a later date. The business glossary will consist of key metrics derived off data elements included in the data dictionary. The vendor is encouraged to review the UALR QI website for additional information. Attention should be given to the charges for the QI Systems Committee. The UALR QI website is available at the following URL: http://ualr.edu/academics/qi/ Questions about the enclosed artifacts, and demonstration or discussion items should be send to Cody Decker ([email protected]) and John Rathje ([email protected]). Questions concerning logistics should be address to Vicky Walden-Wilson, Administrative Specialist III, at 501-916-5025 or [email protected]. The Committee is eager to view and discuss the data warehousing solution proposed by your company. We look forward to seeing you in person soon.



19

APPENDIX III: QI SYSTEMS VENDOR QUESTIONS 1) Based on experiences and best practices, identify post-implementation year 1, 2 and 3 effort/resource allocation to keep current and maintain data relevance. Response should include: a. Resources involved and relative FTE effort of each resource for each year. Further identify source/responsibility of effort (Vendor, Integrator/Implementer, UALR) b. Equipment/infrastructure investment and costs to ensure performance standard and capacity for expected data growth for each year c. Licensing costs for each year d. Other expected lease/implementation costs for each year 2) Based on experiences and best practices, identify skill sets and related job descriptions for a. Solution Implementation b. Solution Maintenance c. Additional Model Development d. Report Development e. For the above, identify ideal balance between contracting resources and UALR resources 3) How many stakeholder engagement sessions are included with the proposed implementation? a. What outcomes are expected from these sessions? b. Who are the key audiences for each of these sessions? 4) How would the included data dictionary and business glossary be used to guide developers during the warehouse implementation? 5) What assurance is provided that all data identified in the dictionary/business glossary will be included in the warehouse (assuming the source data is made available)? 6) What data model is proposed for implementation? Explain. 7) What considerations should be taken into account regarding timing for data updates? Should all data be refreshed at the same time? Is it possible to drill through to transactional (realtime) data in defined instances? 8) What software/solution will be used to maintain ETL builds? 9) What is the proposed implementation environment? (e.g. on-premises, cloud, hybrid) Explain rationale. a. What is the proposed configuration (technical landscape) for any on-premises or UALR hosted infrastructure? b. What are the minimum network requirements to access the system? c. Are there any limitations to remote access or technologies to access the solution? 10) Are there student opt-­out provisions? Is there a way to record FERPA disclosures? How is data retention managed? 11) How granular is the access control (data element, table, cube, report, etc.)? 12) Based on information they have gathered about us, what are the risk factors of the implementation? 13) What are the roles of integrator? How are the implementation jobs divided between the integrator and the vendor? How are the users involved during the implementation? 20

14) Explain best practices for designing security and access control to data. 15) If you are the selected vendor, how could your proposal be procured under university/state procurement regulations? 16) What assumptions are made in the proposal regarding technical skill-sets and physical infrastructure that are in place at UALR? 17) What is the proposed communication plan? 18) What is the total cost of ownership, by year, for your proposal? 19) Are you able to describe any other software security models with consideration given to confidence that a given approach will produce dependable and intended outcomes? 20) Are you able to describe any considerations given to formally control the software design process to validate the use of secure components and any other related secure-by-default designs? 21) What security best practices are recommended to a. Grant, monitor, and audit access to data elements and associated reports and report development? b. Restrict access to data models and development tools? c. Ensure only sound and certified software components are part of official design and development efforts. 22) Why is your solution the ideal solution for UALR’s data warehouse implementation? 23) Based on the scope as it is understood as well your experience with other data warehouse implementations, what is a reasonable timeframe for UALR’s data warehouse implementation? 24) What are the recommended quality assurance practices that should be executed in parallel with UALR’s existing information system processes? 25) Describe the interoperability between the solution you are proposing as well as other common business intelligence (BI) and warehouse platforms.



21

APPENDIX IV: RECENT DATA WAREHOUSE INDUSTRY ADVANCES INDUSTRY ADVANCES AND CHANGES Numbers of changes have occurred in the industry which, on their own, may not warrant attention in this report; however, the rate of change in the technology industry and the significance of individual changes in total merit our attention.

SHIFT TO THE CLOUD With the increased adoption of digital business and cloud technologies in the market, enterprise budgeting, spending and procurement decision makers are increasing their focus and spend across the areas of public, private, hybrid, and community clouds. Key findings are as follows: •

Cloud service adoption continues to reshape IT markets. The impact is being felt not only on enterprises but also on traditional technology vendors and IT services providers.

•

Parts of the cloud market are growing aggressively, with differences across various geographies.

•

Enterprises are recognizing the key benefits in cloud adoption, and these benefits are becoming differentiators for them in the ever-increasing competitive world of hybrid IT.

The following table1 is a good summary of possible benefits regarding cloud services along with organizational actions and expected benefits from these actions. Overall Benefit Sought

Action to Be Taken

Benefits Gained

Cost Savings

Agreed baseline of existing costs.

•

Known position from which to track ROI and measure effectiveness of strategy.



Ensure financial monitoring is sufficiently granular to track overall cost reductions/increases.

•

Corporate responsibility and accountability able to be discharged efficiently and accurately.



SaaS control tools to be in place to capture • activity and prospective areas of expenditure.

Risk exposure known and quantifiable.

•

Efficient discharge of freedom of information (FoI) function.

•

Known data locations for data protection requirements and

1

Summarized from Gartner: “Government Cloud Benefit Realization Starts With Business Alignment” December 2015

22

Overall Benefit Sought

Action to Be Taken

Benefits Gained requirement for exit plan development.



Agility



Establish SaaS contract register to enable • corporate knowledge base to be developed.

Informs company financial monitoring requirements.

•

Allows solution aggregation to be attempted.

•

Overall technology spend for the agency to be monitored and ensures control is exercised.

•

Informs the audit process and ensures resource allocation to priorities is possible.

Agree a metric to determine current level of • agility to avoid subjective discussions.

Ability to respond to emerging events or initiatives.

Determine an optimum level of agility.

Scalability/Elasticity Determine if workloads are predictable/static/fluctuating.

•

Provides some latitude where forecasting function may be poor.

•

Ensures optimum level of investment and focus is committed, rather than excessive amounts.

•

Static workloads may not benefit from a move to the cloud unless other benefits are present.

•

Fluctuating workloads and indeterminate storage could yield cost savings.



Determine if staff numbers are to remain static or fluctuate.

•

True SaaS if controlled well can yield savings if truly elastic.



Determine if application is suitable for an agile cloud platform.

•

Migration may be cost-effective if minimal re-architecting is required.

23

Overall Benefit Sought

Availability

Security



Action to Be Taken

Benefits Gained •

Cloud-native or cloud-ready applications may yield cost savings if moved to IaaS or PaaS.

•

Nonagile applications may cost more when moved to IaaS or PaaS.

Determine the optimum failover or disaster • recovery requirements.

Failover and DR capabilities can be dramatically improved.

Determine optimum level of security for information classification.

Consider use of public cloud.

•

Excessive and underused over capacity can be avoided reducing capital expenditure requirements.

•

Avoidance of excessive security expenditure and reduced complexity of the service.

•

Ensure appropriate tools and controls are in place.

•

Megavendors can often provide a higher security start point than many internal organizations.

•

Certified U.S.-based government cloud can provide assurance of compliance in the U.S. only.

•

Privacy issues are being addressed by some vendors within Europe. Ø These last two points may carry additional costs.

.

DELL2 PURCHASE OF STATISTICA With Statistica (acquired in 2014), Dell has more than 30 years of providing a high-quality platform to a satisfied customer base. Since the acquisition, Dell has added functionality to make it a more modern enterprise solution capable of addressing a broader range of use cases. 2

Dell is a key vendor / supplier of technology and services to UALR

24

MICROSOFT3 CORTANA ANALYTICS SUITE & AZURE MACHINE LEARNING Cortana Analytics Suite combines strong ease of use with scalability (through Azure HDInsight), and supports a wide spectrum of skills (including Power BI for data analysts, and R and Python integration via the Jupyter Notebook App for core data scientists). Customers who choose Microsoft for Advanced Analytics Platforms primarily do so for alignment with existing business intelligence investments, product roadmap and vision, as well as ease of use and availability of skills.

MACHINE TO MACHINE LEARNING: (COMMENTARY FROM SAS4 ) Machine learning is a method of data analysis that automates analytical model building. Using algorithms that iteratively learn from data, machine learning allows computers to find hidden insights without being explicitly programmed where to look. The iterative aspect of machine learning is important because as models are exposed to new data, they are able to independently adapt. They learn from previous computations to produce reliable, repeatable decisions and results. It is a science that is not new – but one that is gaining fresh momentum. Because of new computing technologies, machine learning today is not like machine learning of the past. While many machine learning algorithms have been around for a long time, the ability to automatically apply complex mathematical calculations to big data – over and over, faster and faster – is a recent development. Ex: Google Autonomous Car, Recommended offers on Netflix/Amazon, etc.

DEEP LEARNING Deep learning is a fast-growing area in machine learning research that has achieved breakthroughs in speech, text and image recognition. It is based on endowing a neural network with many hidden layers, enabling a computer to learn tasks, organize information, and find patterns on its own.

PERCEPTUAL INTELLIGENCE The ability to interact with students (or faculty/staff) in new ways and infer intent with vision, face, speech, text and sentiment analysis.

UALR UPDATES BLACKBOARD ANALYTICS ADDITION TO UALR PORTFOLIO. Blackboard Analytics for Learn combines data from Blackboard Learn™ with student and course attributes from our SIS (Banner Student) to create comprehensive reports and dashboards for students, instructors, staff, and leaders.

NEW PERSPECTIVES New leadership in both OIR and IT Services, in addition to other areas have allowed for the development of new ideas and methods to address reporting needs that were otherwise not presented. While we

3 4

Microsoft is a significant vendor in UALR strategy and has strong ties to the College of Business SAS is a leading provider of reporting solutions and has ties to Engineering and Information Technology

25

work to identify the more comprehensive and integrated solution, our teams are capable and poised to deliver a near-term reporting environment with little or no resource investment, if desired.



26

APPENDIX V: VENDOR ANALYSIS FROM GARTNER GARTNER MAGIC QUADRANT FOR ADVANCED ANALYTICS PLATFORMS Gartner defines advanced analytics as the analysis of all kinds of data using sophisticated quantitative methods (such as statistics, descriptive and predictive data mining, machine learning, simulation and optimization) to produce insights that traditional approaches to BI — such as query and reporting — are unlikely to discover. Advanced analytics platforms provide an end-to-end environment for developing and deploying models, including: Data access to a variety of data sources. The advanced analytics customer reference survey indicates that while the majority of users are analyzing transactional data, new data sources — such as text, log and sensor data, and location data — are becoming increasingly common. Data preparation, exploration, and visualization is a key area of functionality as analysis is performed by users who may lack familiarity with the data and have increasingly high expectations of tools for automating data discovery, visualization and preparation. The ability to develop and build analytic models, including clustering, classification and predictive models, forecasting models, simulation models and optimization models. Ability to deploy models and integrate them into business processes and applications. Deployment is a significant pain point for many organizations, so allowing easy adoption of models as part of a business process or application — rather than them just being exported as code or a database score — improves project success rates. Capabilities to perform platform, project and model management. The need to be able to validate the performance of models and track them once deployed is necessary; the ability to reuse models and audit their development and usage can be mandatory, rather than just desired, in certain more regulated industries and environments. High performance and scalability for both development and deployment. The ability to perform at high levels of speed and accuracy with large volumes data and streaming data is still critical for organizations, and with rising data volumes becomes even more of a differentiator. 27

Figure: Gartner Magic Quadrant for Business Intelligence and Analytics Platforms



MAGIC QUADRANT FOR BUSINESS INTELLIGENCE AND ANALYTICS PLATFORMS During the past several years, the balance of power for BI and analytics platform buying decisions has gradually shifted from IT to business as the long-standing BI requirement for centrally provisioned, highly governed and scalable system-of-record reporting has been counterbalanced by the need for analytical agility and business user. The evolution and sophistication of the self-service data preparation and data discovery capabilities available in the market has shifted the focus of buyers in the BI and analytics platform market — toward easy-to-use tools that support a full range of analytic workflow capabilities and do not require significant involvement from IT to predefine data models upfront as a prerequisite to analysis. Gartner's position is that organizations should initiate new BI and analytics projects using a modern platform that supports a Mode 2 (a nonlinear approach that involves learning through iteration, emphasizing agility and speed and, above all, the ability to manage uncertainty) delivery model, in order to take advantage of market innovation and to foster collaboration between IT and the business through 28

an agile and iterative approach to solution development. The vendors featured in this year's Magic Quadrant present modern approaches to promoting production-ready content from Mode 2 to Mode 1 (a linear approach to change, emphasizing predictability, accuracy, reliability and stability), offering far greater agility than traditional top-down, IT-led initiatives — and resulting in governed analytic content that is more widely adopted by business users that are active participants in the development process. As the ability to promote user-generated content to enterprise-ready governed content improves, so it is likely that, over time, many organizations will eventually reduce the size of their enterprise system-ofrecord reporting platforms in favor of those that offer greater agility and deeper analytical insight. Vendors are assessed for their support of five main use cases: 1.

Agile Centralized BI Provisioning. Supports an agile IT-enabled workflow — from data to centrally delivered and managed content — using the self-contained data management capabilities of the platform.

2.

Decentralized Analytics. Supports a workflow from data to self-service analytics.

3.

Governed Data Discovery. Supports a workflow from data to self-service analytics, to systems-ofrecord, IT-managed content with governance, reusability and portability.

4.

Embedded BI. Supports a workflow from data to embedded BI content in a process or application.

5.

Extranet Deployment. Supports a workflow similar to agile centralized BI provisioning for the external customer or, in the public sector, citizen access to analytic content.

Vendors are also assessed according to the following 14 critical capabilities: Infrastructure 1. BI Platform Administration. Capabilities that enable scaling the platform, optimizing performance and ensuring high availability and disaster recovery. 2. Cloud BI. Platform-as-a-service and analytic-application-as-a-service capabilities for building, deploying and managing analytics and analytic applications in the cloud, based on data both in the cloud and on-premises. 3. Security and User Administration. Capabilities that enable platform security, administering users, and auditing platform access and utilization. 4. Data Source Connectivity. Capabilities that allow users to connect to the structured and unstructured data contained within various types of storage platforms, both on-premises and in the cloud. Data Management 5. Governance and Metadata Management. Tools for enabling users to share the same systems-ofrecord semantic model and metadata. These should provide a robust and centralized way for administrators to search, capture, store, reuse, and publish metadata objects, such as dimensions, hierarchies, measures, performance metrics/key performance indicators (KPIs) and report layout

29

objects, parameters and so on. Administrators should have the ability to promote a business-userdefined data model to a system-of-record metadata object. 6. Self-Contained Extraction, Transformation and Loading (ETL) and Data Storage. Platform capabilities for accessing, integrating, transforming, and loading data into a self-contained storage layer, with the ability to index data and manage data loads and refresh scheduling. 7. Self-Service Data Preparation. The drag-and-drop, user-driven data combination of different sources, and the creation of analytic models such as user-defined measures, sets, groups and hierarchies. Advanced capabilities include semantic auto discovery, intelligent joins, intelligent profiling, hierarchy generation, data lineage, and data blending on varied data sources, including multi-structured data. Analysis and Content Creation 8. Embedded Advanced Analytics. Enables users to easily access advanced analytics capabilities that are self-contained within the platform itself or available through the import and integration of externally developed models. 9. Analytic Dashboards. The ability to create highly interactive dashboards and content, with visual exploration and embedded advanced and geospatial analytics, to be consumed by others. 10. Interactive Visual Exploration. Enables the exploration of data via the manipulation of chart images, with the color, brightness, size, shape and motion of visual objects representing aspects of the dataset being analyzed. This includes an array of visualization options that go beyond those of pie, bar, and line charts, to include heat and tree maps, geographic maps, scatter plots, and other special-purpose visuals. These tools enable users to analyze the data by interacting directly with a visual representation of it. 11. Mobile Exploration and Authoring. Enables organizations to develop and deliver content to mobile devices in a publishing and/or interactive mode, and takes advantage of mobile devices' native capabilities, such as touchscreen, camera, location awareness and natural-language query. Sharing of Findings 12. Embedding Analytic Content. Capabilities including a software developer's kit with APIs and support for open standards for creating and modifying analytic content, visualizations and applications, embedding them into a business process, and/or an application or portal. These capabilities can reside outside the application (reusing the analytic infrastructure), but must be easily and seamlessly accessible from inside the application without forcing users to switch between systems. The capabilities for integrating BI and analytics with the application architecture will enable users to choose where in the business process the analytics should be embedded. 13. Publishing Analytic Content. Capabilities that allow users to publish, deploy and operationalize analytic content through various output types and distribution methods, with support for content search, storytelling, scheduling and alerts.

30

14. Collaboration and Social BI. Enables users to share and discuss information, analysis, analytic content and decisions via discussion threads, chats and annotations. Figure: Gartner Magic Quadrant for Mode 2 Business Intelligence and Analytics Platforms

The evolution and sophistication of the self-service data preparation and data discovery capabilities available in the market has shifted the focus of buyers in the BI and analytics platform market — toward easy-to-use tools that support a full range of analytic workflow capabilities and do not require significant involvement from IT to predefine data models upfront as a prerequisite to analysis.



31

APPENDIX VI: OIR & IT SERVICES SKILL-SET ANALYSIS University of Arkansas at Little Rock Office of Institutional Research Staff Skill-Set Matrix 2/18/2016

1, 1 2, 2 2, 2

1 1 1

Interest (0-1)

1 1 0 0 0

Proficiency (X,Y)

3, 2 2, 2 2, 1 1, 1 2, 1

Interest (0-1)

1 1 1 1 1

Proficiency (X,Y)

1, 1 1, 1 1, 1 1, 1 1, 1

Interest (0-1)

1 1 1 1 1 1 1

Proficiency (X,Y)

2, 1 1, 1 0, 1 0, 1 0, 1 0, 1 0, 1

Interest (0-1)

1 1 1 1 1 1

Position 7

Proficiency (X,Y)

1, 1 2, 2 2, 2 3, 2 2, 2 1, 1

Position 6

Interest (0-1)

0 1 1 0 1 0 1 1

Position 5

Proficiency (X,Y)

Evisions Argos Microsoft SQL Server Business Intelligence Oracle Business Intelligence Engine Information Builders IBM Cognos SAP NetWeaver SAS Tableau Data Analysis (Overall) Data Interpretation Microsoft Excel Microsoft Access Google Spreadsheets Structured Query Language (SQL) Statistical Software Package (R, SPSS, etc.) Data Modeling & ETL (Overall) Relational Database Design (Normalization) Dimensional Database Design (Facts/Dimensions) ETL - Informatica PowerCenter ETL - IBM Data Stage ETL - Oracle Data Integrator ETL - Microsoft SQL Server Integration Services (SIS) ETL - SAP Business Objects Data Services Data Governance (Overall) Policy Review Policy Development Data Stewardship Facilitation Relational Database Systems Dimensional Database Systems Programming (Overall) HTML SQL Microsoft .NET (C#) Microsoft .NET (ASP) Java Project Management (Overall) Project Management Professional Methdologies (PMP) Written Communication (Email, Tutorial, Whitepaper) Oral Communication (Small & Large Group Facilitation)

Position 4

Interest (0-1)

0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 1, 2

Business Intelligence (Overall)

Position 3

Proficiency (X,Y)

Interest (0-1)

Position 2

Proficiency (X,Y)

Position 1

1, 2 2, 2 2, 2 0, 1 0, 2 0, 1 0, 1 0, 1 0, 1 2, 2 2, 2 2, 2 3, 2 2, 2 2, 2 1, 2 1, 1 1, 2 0, 1 1, 1 0, 1 0, 1 0, 1 0, 1 1, 1 1, 1 1, 1 1, 1 2, 2 1, 2 1, 2 0, 1 2, 3 1, 1 0, 1 0, 1 2, 1 2, 1 2, 2 2, 1

1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

2, 3 1, 2 2, 3 0 ,1 0, 2 3, 3 1, 2 1, 2 2, 3 3, 3 3, 3 3, 3 3, 3 2, 3 2, 3 1, 2 2, 3 3, 3 2, 3 1, 2 3, 3 0, 1 2, 2 1, 2 2, 3 3, 3 2, 3 2, 3 2, 3 2, 3 2, 2 2, 2 2, 3 1, 3 1, 2 2, 2 3, 3 3, 3 3, 3 2, 3

1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 0 1 1 1 0 1 1 1 1

1, 2 2, 3 2, 3 0, 2 0, 2 0, 2 0, 2 1, 2 0, 2 2, 2 2, 2 3, 2 3, 2 1, 2 3, 2 3, 3 2, 2 2, 3 1, 2 0, 1 0, 1 0, 1 1, 1 1, 1 2, 3 2, 3 2, 3 2, 2 2, 2 0, 1 2, 2 2, 2 3, 2 0, 1 0, 1 1, 2 1, 2 1, 3 2, 3 2, 3

1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

2, 2 2, 3 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 2, 3 3, 3 3, 3 3, 3 3, 3 2, 2 2, 3 3, 3 2, 2 2, 3 2, 3 0, 1 0, 1 0, 1 0, 1 0, 1 2, 3 2, 3 2, 3 2, 3 2, 3 2, 3 2, 2 1, 1 2, 3 0, 1 0, 1 1, 1 2, 2 1, 2 3, 3 2, 3

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1

1, 2 3, 3 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 2, 2 1, 2 1, 2 1, 2 1, 2 3, 3 0, 1 1, 1 2, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 2, 2 0, 1 3, 2 2, 2 3, 3 1, 2 0, 1 3, 3 1, 1 0, 1 1, 1 1, 1

1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1

0, 1 1, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 2, 2 0, 1 2, 2 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 0, 1 1, 1 0, 1 1, 1 1, 1

0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0

Notes Proficiency rating is expressed as (X,Y) where X = Person's level of skill or knolwedge and Y = Level of responsibility applying the skill of knowledge Skill or Knowledge Level (X) 0 = No capability 1 = Basic level of capability 2 = Intermediate level of capability 3 = Advanced level of capability Application of Skills/Knowledge (Y) 1 = Must work under supervision 2 = Can work independetly with little or no direct supervision 3 = Can manage others applying the skill or knowledge Interest 0 = Has no interest in applying this skill or knowledge. 1 = Is interested in applying this skill or knowledge.



32





33

34







35

APPENDIX VII: DATA WAREHOUSE DEPLOYMENT OPTIONS On-Premises: All server, software and associated administration are the responsibility of UALR. This includes the need to provision for development, test and production environments. Disaster recovery and backup are managed by UALR. Managed Hosting: Server and software are hosted in an external data center. Service level agreements are required to manage uptime and performance, and disaster recovery/backups may be additional costs. Provisioning of resources is managed through service tickets and measured through SLA terms. Solution administration, tuning, and development remain the responsibility of UALR. IaaS: Infrastructure as a Service is a standardized, highly automated offering where computer resources, complemented by storage and networking capabilities are owned and hosted by a service provider and offered to customers on-demand. Customers are able to self-provision this infrastructure, using a Webbased graphical user interface that serves as an IT operations management console for the overall environment. API access to the infrastructure may also be offered as an option. PaaS: Platform as a Service offering, usually depicted in all-cloud diagrams between the SaaS layer above it and the IaaS layer below, is a broad collection of application infrastructure (middleware) services (including application platform, integration, business process management and database services). However, the hype surrounding the PaaS concept is focused mainly on application PaaS (in our case dwPaaS).



36

APPENDIX VIII: DATA WAREHOUSE TCO MODEL Models to predict TCO based on deployment model (on-premise, managed host, IaaS and dwPaaS) are outlined below. Following the models (note – the models are DRAFT and require further research), example seven year TCO is provided. Database Expense Task Summary $ = base level usually included, upsell opportunity for cloud vendor 1 = one time expense, O = ongoing expense On Premises

IaaS

Managed Host

dwPaaS

Comments/Instructions

Data Center Expenses Data Center Floor Space

O

Power

O

Cooling

O

Estimate of cost for data center floor space. Power requirements and associated cost per year. Cooling requirements and associated cost per year.

Network Infrastructure

O

Estimated cost for network connectivity, bandwidth, infrastructure, IP addresses.

Database Hardware Expenses Storage Acquisition

1

O

Installation/Setup

1

1

Configuration Maintenance/Support

1 O

1

Estimated Annual FTE Servers

O

O

Acquisition

1

O

Installation/Setup

1

1

Configuration Maintenance/Support

1 O

1

Estimated Annual FTE

O

O

1

1

Cost of physical storage (SAN, NAS). Appliances may include this in the base acquisition price. Acquisition cost. For IaaS offerings, this is monthly cost. Estimate of setup costs. Will calculate based on estimated FTE salary and number of hours required. Estimate of configuration costs. Will calculate based on estimated FTE salary and number of hours required. Annual maintenance to vendor. Market rate for storage admin, annual salary * % of time expected. Cost of servers required to run the DW/BI Acquisition cost. For IaaS offerings, this is monthly cost. Estimate of setup costs. Will calculate based on estimated FTE salary and number of hours required. Estimate of configuration costs. Will calculate based on estimated FTE salary and number of hours required. Annual maintenance to vendor. Market rate for server admin, annual salary * % of time expected.

Operating System Expenses

37

Acquisition

1

O

1

Acquisition cost. Estimate of setup costs. Will calculate based on estimated FTE salary and number of hours required. Estimate of configuration costs. Will calculate based on estimated FTE salary and number of hours required.

Installation/Setup

1

1

Configuration

1

1

Maintenance/Support

O

O

Backups

O

O

O

Backup Retention

O

O

O

Backup Validation Backup Software Backup Hardware Backup Setup/Configuration Monitoring

O 1 1

O 1 1

O 1 1

1 O

1 O

Patching/Updates

O

O

Configuration and setup of backups. Cost of monitoring operating system. Estimated cost for patching maintenance per year.

Estimated Annual FTE

O

O

Market rate for system admin, annual salary * % of time expected.

Acquisition

1

1/O

1

Cost of DW/BI software acquisition. May be monthly for IaaS.

Installation/Setup

1

1

1

Estimate of install/setup costs, one-time hours, normalized to DBA FTE salary.

Configuration

1

1

1

Estimate of configuration costs, ongoing, normalized to DBA FTE salary.

1

1

1,$

1

1

1,$

1

1

1,$

1

1

1,$

1

1

1

O

Optional DBMS licensed features.

1

1

1

O

Optional DBMS licensed features.

1

1

1

O

Optional DBMS licensed features.

1

1

1

O

Optional DBMS licensed features.

1

1

1

O

Maintenance/Support Backups

O O

O O

O 0,$

O,$

Backup Retention

O

O

O

O

Optional DBMS licensed features. Annual maintenance for DW/BI software. May not apply to IaaS if a monthly subscription. Estimated cost of backups for DW/BI. Cost of retaining backups for required period.

O

Annual maintenance to vendor. May not apply to IaaS if a monthly charge. Estimated cost of backups for operating system. Cost of retaining backups for required period. Time required to validate backups and recovery procedures. Cost of any required backup software. Cost of any required backup hardware.

DW/BI Expenses

High Availability (HA) Licensing HA Hardware Expense Disaster Recovery (DR) Licensing DR Hardware Expense Optional Licensed Feature Optional Licensed Feature Optional Licensed Feature Optional Licensed Feature Optional Licensed Feature

O,$

O,$

Additional required software licenses (if any) to support HA operations. Additional required hardware to support HA operations. Additional required software licenses (if any) to support DR operations. Additional required hardware to support DR operations.

38

O

Time required to validate backups and recovery procedures. Cost of any required backup software. Cost of any required backup hardware.

Backup Validation Backup Software Backup Hardware Backup Setup/Configuration Monitoring

O 1 1

O 1 1

O 1,$ 1,$

1 O

1 O

1,$ 0,$

Patching/Updates

O

O

O

Tuning/Performance Estimated Annual FTE Application Development/Data Modeling

O

O

O

O

O

O

O

Estimated hours for tuning/performance, normalized to DBA FTE salary. Market rate for DBA, annual salary * % of time expected.

O

O

O

O

Cost to develop applications. Variable.

Many cloud vendors offer discounts for term commitments, but they often include an upfront cost.

O,$

Configuration and setup of backups. Cost of monitoring for DW/BI. Estimated hours for patching/updating, normalized to DBA FTE salary.

Cloud-Specific Expenses Upfront Cost for Term Commitment

1

1

Ongoing License costs (per Month)

C

O

Data Transfer from Cloud

O

O

IP Addresses

O

Monitoring VPN to Cloud Cloud Compute Cloud Storage Cloud Service AddOn 1 Cloud Service AddOn 2 Cloud Service AddOn 3 Cloud Service AddOn 4

O O

O O O

Monthly charge for DBMS if not covered above. Cost of moving data to/from cloud. Estimate based on data volume * cloud vendor charge. Monthly charge for IP addresses, if required by cloud vendor. Monthly charge for monitoring, if not included with cloud service. Cost of site to site VPN to cloud. Optional. Cost of cloud compute nodes. Cost of cloud storage for DBMS.

O

O

Optional cloud service feature.

O

O

Optional cloud service feature.

O

O

Optional cloud service feature.

O

O

Optional cloud service feature.

Above: The model has been refined to support costs associated with each model. The model is in draft form and has not been formally reviewed by the QI Committee. The charts on the following page, result from available estimates of costs and should be reviewed as examples of ownership costs over seven years. Once the model has been tested and tuned, it will provide outcomes that can be trusted for cost predictions.

39

40

APPENDIX IX: TOTAL COST OF OWNERSHIP SURVEY To help UALR consider the detail that will go into a final TCO model, information is provided from various research efforts. The following research details the business intelligence platform ownership cost (BIPOC) results from Gartner's 2015 BI and Analytics Magic Quadrant customer survey of 2,083 companies using BI platforms. Figure 1 shows the key cost drivers of the seven license components analyzed in this report. They are divided into three overall categories: 1. License/additional infrastructure (hardware)/maintenance and recurring fees: License costs Maintenance and recurring fees Additional infrastructure (hardware) 2. Initial internal implementation and ongoing IT FTEs: • • •

• •

Initial implementation (internal resources) Ongoing implementation, IT FTEs Initial implementation and ongoing implementation (external service provider):

• •

Initial implementation (external service provider) Ongoing implementation (external service provider)

3.



41

Key Findings from Gartner Research

Buying low license cost platforms does not always translate into low BIPOC over time, or into higher business benefits. •

Open-source platforms, often purchased due to low license costs, have the highest threeyear BIPOC per user due to high IT full-time-equivalent (FTE) costs per user.

•

Data discovery leaders have the lowest three-year BIPOC per user, largely driven by a reduced need for IT FTEs and external implementation services.

•

Cloud BI vendors have a below average three-year BIPOC per user with a very favorable (the lowest) hardware cost per user profile and low initial implementation costs.

•

Large independents and megavendors have higher-than-average initial licenses, maintenance, and hardware costs, as well as initial and ongoing implementation costs, but lower-than-average IT FTE costs per user — due to the efficiency of supporting large, viewer and report-centric deployments — result in a favorable overall three-year cost per user profile.

High-Level Recommendations from Gartner •

Balance any cost consideration with functional requirements, expected adoption and potential business benefits when evaluating vendors.

•

Manage the cost of incremental pricing as deployment sizes grow and manage maintenance increases over time, rather than focus solely on initial price negotiations.

•

Consider cloud BI as an option to reduce infrastructure costs and to take advantage of product innovations without the need to upgrade existing infrastructure.

Three-Year Business Intelligence Platform Ownership Cost Three Cost Components

• • • • • • •

Initial license cost per user + (Maintenance or recurring license fees per user * 3 years) + [Additional infrastructure/hardware per user + (additional infrastructure/hardware per user * 0.2 for maintenance * 3 years)] + Initial implementation cost per user (external service provider) + Initial implementation cost per user (internal resources) + (Ongoing external service provider cost per user * 3 years) + [(Technical FTEs * an annual salary of $114,121 * 3 years)/deployment size]

Given this definition, the following average costs were identified through the survey of 2000+ companies. 42

BIPOC per User by Deployment Size

BIPOC Cost Components by Deployment Size



43