DEPARTAMENTO DE INFORMÁTICA
Licenciatura em Engenharia Informática
SEIS – SPACE ENVIRONMENT INFORMATION SYSTEM REPORTING AND ANALYSIS TOOL Graduation Report September 2004
RICARDO MARTINS FERREIRA (11048)
ORIENTATION: PROF. DOUTOR JOÃO MOURA PIRES EXTERNAL COORDINATOR: PROF. DOUTORA RITA ALMEIDA RIBEIRO INSTITUTION: CA3 – SOFT COMPUTING AND AUTONOMOUS AGENTS
Acknowledgments This work is dedicated to all the SEIS team for their commitment and excitement to the work. Thanks to Professor João Moura Pires for his support to all of us in this project, it’s really nice to work with him. Greetings to all the CA3 group members: Antonio Falcão, Ivan Dorotovic, Ricardo Raminhos, Paulo Bravo and Rita Ribeiro. (Thank you all for keeping everyone happy during work hours). Thanks to Nuno Viana, our project manager. (We all love his oldies music jukebox – back from the 70’s or 80’s). Thanks to Joaquim Neto for working with us during the last months. We all wish you good luck in America! Special thanks to Marta Pantoquilho who invited me in the first place to join the SEIS team and the CA3 group.
And thanks to all my friends and colleagues, to my family and to Filipa, my love :).
-I-
Summary Analysis of space conditions is a common task for a Spacecraft operator. These conditions are available from a multitude of data sources, each one with several parameters, and each one with their own format. The existence of a centralized system would greatly help the Spacecraft operator. The Space Environment Information System project (SEIS) issues this problem, integrating Space Weather (S/W) and Spacecraft (S/C) data and providing tools to the operators for monitoring and analysing this data. This project consists on a series of windows applications for offline analysis and reporting of both S/W and S/C data. The application was developed using Microsoft Visual Basic .NET. Due to the large amounts of data available, the data is stored in a Data Warehouse (DW), implemented in Microsoft SQL Server 2000, and queried using the MDX Language. Data analysis and reporting are mostly done using specific visualizations, due to the large amount of data available. Reporting is done defining and configuring a set of visualizations over certain parameters.
For more information on the SEIS Project: http://www.uninova.pt/ca3/en/project_SEIS.htm
- III -
Index 1
2
3
4
5
Introduction________________________________________ 1 1.1
Academic Context ______________________________________________ 2
1.2
Technological and Scientific Context _______________________________ 2
1.3
Project goals __________________________________________________ 3
1.4
Document Structure ____________________________________________ 4
SEIS Overview______________________________________ 5 2.1
Introduction __________________________________________________ 6
2.2
SEIS Architecture and Applications ________________________________ 7
Developed Tasks ____________________________________ 9 3.1
Task Introduction _____________________________________________ 10
3.2
Chronogram _________________________________________________ 10
Reporting and Analysis Tool __________________________ 11 4.1
Introduction _________________________________________________ 12
4.2
Data Catalogue _______________________________________________ 12
4.3
Report Designer ______________________________________________ 12
4.4
Report Browser _______________________________________________ 12
Data Catalogue ____________________________________ 14 5.1
Introduction _________________________________________________ 15
5.2
Data Catalogue and Metadata ___________________________________ 15
5.3
Standalone mode _____________________________________________ 15 5.3.1 User Interface __________________________________________ 16 5.3.2 Data Navigation _________________________________________ 17 5.3.3 Metadata Visualization ____________________________________ 22 5.3.4 Data Visualization _______________________________________ 23
5.4
Chart Series Picker Mode _______________________________________ 30
-V-
5.4.1 Operation modes ________________________________________ 30 5.4.2 Interface_______________________________________________ 31
6
Visualizations______________________________________ 32 6.1
Introduction _________________________________________________ 33
6.2
Multiple Axis Charts ___________________________________________ 34 6.2.1 Linear Chart ____________________________________________ 36 6.2.2 Biplot Chart (MT Charts) __________________________________ 41 6.2.3 World map Chart ________________________________________ 43 6.2.4 2D Correlation Chart _____________________________________ 48
6.3
Single Axis Charts_____________________________________________ 52 6.3.1 Events Charts___________________________________________ 52 6.3.2 Categorical Chart ________________________________________ 53 6.3.3 Gradient Chart __________________________________________ 54
6.4
Multiple Chart Alignment (Multichart) _____________________________ 64
6.5
Implementation Details ________________________________________ 66 6.5.1 Dealing with Interpolations ________________________________ 67
7
Report Designer____________________________________ 69 7.1
Introduction _________________________________________________ 70
7.2
Implementation ______________________________________________ 70 7.2.1 Report Definition Language ________________________________ 70 7.2.2 RDL Type Classes ________________________________________ 89 7.2.3 RDL Type Property Editors _________________________________ 93
7.3
User Interface________________________________________________ 95 7.3.1 Report Tab _____________________________________________ 96 7.3.2 Body Tab ______________________________________________ 97 7.3.3 Code Tab ______________________________________________ 98 7.3.4 Preview Tab ____________________________________________ 99
7.4
Report Operations____________________________________________ 100 7.4.1 Adding a Textbox _______________________________________ 100 7.4.2 Adding a Linear Chart ___________________________________ 103 7.4.3 Adding a Gradient Chart _________________________________ 109
- VI -
7.4.4 Adding a 2D Correlation Chart _____________________________ 113 7.4.5 Adding a Multi Chart ____________________________________ 119 7.4.6 Previewing the Report ___________________________________ 125
8
Metadata ________________________________________ 126 8.1
Introduction ________________________________________________ 127
8.2
Metadata XSD Structure _______________________________________ 127 8.2.1 Displayable Elements ____________________________________ 129 8.2.2 Searchable Elements ____________________________________ 130 8.2.3 Sortable Elements ______________________________________ 130 8.2.4 Element Identifiers______________________________________ 131
9
Common Components ______________________________ 132 9.1
Introduction ________________________________________________ 133
9.2
Data & Data Access __________________________________________ 133 9.2.1 Sample Database _______________________________________ 133 9.2.2 Data Access Classes_____________________________________ 137
9.3
Data Visualization ____________________________________________ 146 9.3.1 Timeseries Data Visualization _____________________________ 146
9.4
Metadata Access Library _______________________________________ 147 9.4.1 Introduction ___________________________________________ 147 9.4.2 Implementation Details __________________________________ 147
10 Conclusion ______________________________________ 149 10.1 Evaluation of the Developed work _______________________________ 150 10.2 Future work ________________________________________________ 150 10.2.1
Data Catalogue ______________________________________ 150
10.2.2
Report Designer______________________________________ 150
10.2.3
Report Browser ______________________________________ 151
10.2.4
DIM integration ______________________________________ 151
10.2.5
Metadata ___________________________________________ 151
10.3 Internship Evaluation _________________________________________ 151
- VII -
11 Bibliography_____________________________________ 152 12 Annexes________________________________________ 154 12.1 Annex 1 – Report Definition Language Schema_____________________ 155 12.2 Annex 2 – Parameter Metadata Schema __________________________ 165 12.3 Annex 3 – Data Catalogue Tree Definition XML _____________________ 169 12.3.1
Schema for the Tree Definition XML file ___________________ 169
12.3.2
Tree Definition XML File________________________________ 170
12.4 Annex 4 – RDL Classes Diagram ________________________________ 171
- VIII -
12.4.1
Report Class and related classes _________________________ 171
12.4.2
TextboxType Class and related Classes____________________ 172
12.4.3
SingleLinearChartType Class and related classes ____________ 173
12.4.4
SingleGradientChartType Class and related classes __________ 174
12.4.5
SingleXYCorrelationChartType class and related classes ______ 175
12.4.6
MultiChartType class and related classes __________________ 176
Figure Index FIGURE 2.1: SEIS
ARCHITECTURE
................................................................................. 7
FIGURE 3.1 - CHRONOGRAM ...................................................................................... 10 FIGURE 5.1: DATA CATALOGUE MAIN WINDOW .................................................................. 16 FIGURE 5.2: COMPLETE TREE ..................................................................................... 18 FIGURE 5.3: SEARCH INTERFACE .................................................................................. 19 FIGURE 5.4: SAMPLE SEARCH WITH MANY FIELDS ............................................................... 20 FIGURE 5.5: SAMPLE SEARCH COMPLETED ....................................................................... 20 FIGURE 5.6: DIALOG BOX ASKING THE DESCRIPTION FOR THE SAVED SEARCH ................................ 20 FIGURE 5.7: SAVED SEARCHES ................................................................................... 21 FIGURE 5.8: SAVED SEARCH RENAMING ......................................................................... 21 FIGURE 5.9: SAMPLE DATA FOR A SINGLE PARAMETER .......................................................... 23 FIGURE 5.10: SAMPLE DATA FOR A COMPLEX PARAMETER....................................................... 24 FIGURE 5.11: TIME WINDOW INFORMATION GROUP ............................................................. 25 FIGURE 5.12: SAMPLE CHART MINUTE AGGREGATION LEVEL (PARAMETER GRAIN LEVEL) .................... 26 FIGURE 5.13: SAMPLE CHART HOUR AGGREGATION LEVEL (HOURLY AVERAGES) ............................. 26 FIGURE 5.14: SAMPLE CHART HALF-DAY AGGREGATION LEVEL (HALF-DAY / 12
HOUR AVERAGES).........
26
FIGURE 5.15: SAMPLE CHART DAY AGGREGATION LEVEL (DAILY AVERAGES).................................. 26 FIGURE 5.16: SAMPLE CHART ZOOMING IN ...................................................................... 27 FIGURE 5.17: SAMPLE DATA AFTER ZOOMING IN ................................................................ 27 FIGURE 5.18: TIME WINDOW DETAIL AFTER ZOOMING IN ....................................................... 27 FIGURE 5.19: SAMPLE COMPLEX CHART ZOOMING IN ........................................................... 28 FIGURE 5.20: SAMPLE COMPLEX CHART AFTER ZOOMING IN .................................................... 28 FIGURE 5.21: SLIDING MARKER MOVING ......................................................................... 29 FIGURE 5.22: SLIDING MARKER MOVING WITH NO INTERPOLATION ............................................ 29 FIGURE 5.23: DATA CATALOGUE WITH CHART SERIES PICKER ................................................ 30 FIGURE 5.24: CHART SERIES PICKER DETAILED INTERFACE .................................................... 31 FIGURE 5.25: CHART SERIES PICKER FILLED WITH PARAMETERS .............................................. 31 FIGURE 6.1: TWO AXIS CARTESIAN CHART....................................................................... 34 FIGURE 6.2: TWO AXIS CARTESIAN CHART WITH ONE SERIES .................................................. 34 FIGURE 6.3: TWO AXIS CARTESIAN CHART WITH TWO SERIES ................................................. 34 FIGURE 6.4: THREE AXIS CARTESIAN CHART..................................................................... 35 FIGURE 6.5: THREE AXIS CARTESIAN CHART WITH TWO SERIES REFERRING THE Y-AXIS AND OTHER REFERRING THE
Z-AXIS ........................................................................................................................... 35 FIGURE 6.6: SINGLE SERIES WITH NO INTERPOLATION ......................................................... 36 FIGURE 6.7: SERIES WITH LINEAR INTERPOLATION.............................................................. 37 FIGURE 6.8: SERIES WITH SMOOTHING INTERPOLATION ........................................................ 37 FIGURE 6.9: SERIES WITH ACTIVE OOL
SAMPLE ................................................................
37
FIGURE 6.10: LINEAR CHART WITH SERIES LEGEND AT RIGHT .................................................. 38 FIGURE 6.11: REAL VALUE BASED X-AXIS ....................................................................... 38
- IX -
FIGURE 6.12: TIME-BASED X-AXIS............................................................................... 38 FIGURE 6.13: SETTING THE AXIS SCALE ......................................................................... 39 FIGURE 6.14: SAMPLE CHART WITH LOGARITHMIC SCALE....................................................... 39 FIGURE 6.15: Y-AXIS MIRRORING SAMPLE ...................................................................... 39 FIGURE 6.16: SLIDING MARKER POINTING SERIES VALUES ..................................................... 40 FIGURE 6.17: SLIDING MARKER POINTING INTERPOLATED SERIES VALUES .................................... 40 FIGURE 6.18: SLIDING MARKER POINTING MISSING VALUES ................................................... 40 FIGURE 6.19: SAMPLE ZOOMING IN .............................................................................. 41 FIGURE 6.20: BIPLOT CHART MARKERS AND LEGEND............................................................ 42 FIGURE 6.21: BIPLOT CHART WITH BACKGROUND SHADINGS .................................................. 42 FIGURE 6.22: BIPLOT CHART WITH "SERIES 1"
HIGHLIGHTED .................................................
43
FIGURE 6.23: WORLD MAP IMAGE WITH THE LONGITUDE AND LATITUDE AXIS REPRESENTED ............... 43 FIGURE 6.24: WORLD MAP CROSSING PROBLEM ................................................................. 44 FIGURE 6.25: WORLD MAP LONGITUDE CROSSING SITUATION ................................................. 45 FIGURE 6.26: WORLD MAP LATITUDE AND LONGITUDE CROSSING SITUATION ................................ 45 FIGURE 6.27: WORLD MAP EXTREME LATITUDE AND LONGITUDE CROSSING SITUATION...................... 45 FIGURE 6.28: WORLD MAP WITH SAMPLE ORBIT WITH 3
ANNOTATIONS AND ONE DESCRIPTION VISIBLE ...
46
FIGURE 6.29: SHOW REVOLUTION POPUP MENU ................................................................. 46 FIGURE 6.30: WORLD MAP WITH HIDDEN REVOLUTION ......................................................... 46 FIGURE 6.31: WORLD MAP CHART SELECTING A ZOOM IN RANGE .............................................. 47 FIGURE 6.32: WORLD MAP CHART AFTER ZOOMING IN .......................................................... 48 FIGURE 6.33: 2D CORRELATION CHART SAMPLE ................................................................ 48 FIGURE 6.34: 2D CORRELATION CHART SELECTING A ZOOM IN RANGE ....................................... 51 FIGURE 6.35: 2D CORRELATION CHART AFTER ZOOMING IN ................................................... 51 FIGURE 6.36:
REGULAR EVENT ....................................................................................
52
FIGURE 6.37:
EVENT WITH VALUE AND MAXIMUM ................................................................
52
FIGURE 6.38: EVENTS CHART WITH SIMULTANEOUS EVENTS ................................................... 52 FIGURE 6.39: EVENTS CHART SLIDING MARKER POINTING TWO ACTIVE EVENTS .............................. 53 FIGURE 6.40: CATEGORICAL CHART .............................................................................. 53 FIGURE 6.41: CATEGORICAL CHART SLIDING MARKER .......................................................... 54 FIGURE 6.42: GRADIENT CHART .................................................................................. 57 FIGURE 6.43: GRADIENT CHART SCALE DETAILS ................................................................ 57 FIGURE 6.44: GRADIENT SCALE DIALOG ......................................................................... 58 FIGURE 6.45: GRADIENT SCALE DIALOG WITH A SELECTED COMPONENT ...................................... 58 FIGURE 6.46: DRAGGING THE GRADIENT COLOUR COMPONENT ................................................ 59 FIGURE 6.47: SETTING THE GRADIENT COLOUR COMPONENT POSITION DIRECTLY ............................ 59 FIGURE 6.48:
CHANGING A GRADIENT SCALE COMPONENT COLOUR ............................................
59
FIGURE 6.49: ADD COMPONENT CONTEXT MENU ................................................................ 60 FIGURE 6.50: GRADIENT SCALE AFTER ADDING A COMPONENT ................................................. 60 FIGURE 6.51: REMOVE COMPONENT CONTEXT MENU ............................................................ 61 FIGURE 6.52: GRADIENT SCALE AFTER REMOVING THE COMPONENT ........................................... 61 FIGURE 6.53: GRADIENT SCALE MIN AND MAX CHECKBOXES .................................................. 62 FIGURE 6.54: GRADIENT WITH MIN AND MAX SET .............................................................. 62 FIGURE 6.55: GRADIENT FIXED SCALE COMPONENT VALUE CHANGING. ....................................... 62
-X-
FIGURE 6.56: RESULTING GRADIENT CHART WITH FIXED SCALE ............................................... 63 FIGURE 6.57: GRADIENT CHAR SLIDING MARKER ................................................................ 63 FIGURE 6.58: CHART ALIGNMENT SAMPLE ....................................................................... 64 FIGURE 6.59: MULTICHART SLIDING MARKER SAMPLE .......................................................... 65 FIGURE 6.60: MULTICHART WITH BIPLOT CHARTS .............................................................. 65 FIGURE 6.61: MULTICHART X-AXIS SELECTION .................................................................. 66 FIGURE 6.62: RESULTING POINTS OF CURVE (EACH BLACK POINT CONNECTED FORM THE BLUE CURVE).... 67 FIGURE 6.63: SAMPLE CURVE WITH VIRTUAL COLUMNS ......................................................... 68 FIGURE 7.1: LAYOUTTYPE SCHEMA VIEW ......................................................................... 71 FIGURE 7.2: TOPALIGNMENTTYPE SCHEMA VIEW ................................................................ 71 FIGURE 7.3: REPORTITEMTYPE EXPANDED SCHEMA VIEW ....................................................... 72 FIGURE 7.4: FONTTYPE SCHEMA VIEW............................................................................ 72 FIGURE 7.5: COLORTYPE SCHEMA VIEW .......................................................................... 73 FIGURE 7.6: TEXTBOXTYPE EXPANDED SCHEMA VIEW ........................................................... 74 FIGURE 7.7: CHARTTYPE SCHEMA VIEW .......................................................................... 76 FIGURE 7.8: CHARTSERIESTYPE SCHEMA VIEW.................................................................. 77 FIGURE 7.9: LINEARCHARTSERIESTYPE SCHEMA VIEW ......................................................... 78 FIGURE 7.10: CHARTSERIESLISTTYPE SCHEMA VIEW ........................................................... 78 FIGURE 7.11: LINEARCHARTAXISTYPE SCHEMA VIEW .......................................................... 79 FIGURE 7.12: LINEARCHARTTYPE SCHEMA VIEW ................................................................ 79 FIGURE 7.13: TIMEWINDOWTYPE SCHEMA VIEW ................................................................ 80 FIGURE 7.14: CHARTXAXISTYPE SCHEMA VIEW ................................................................. 80 FIGURE 7.15: SINGLELINEARCHARTTYPE SCHEMA VIEW ....................................................... 81 FIGURE 7.16: XYCORRELATIONCHARTTYPE SCHEMA VIEW ..................................................... 82 FIGURE 7.17: SINGLEXYCORRELATIONCHARTTYPE SCHEMA VIEW ............................................ 83 FIGURE 7.18: GRADIENTCOLORCOMPONENTTYPE SCHEMA VIEW .............................................. 83 FIGURE 7.19: GRADIENTTYPE SCHEMA VIEW .................................................................... 84 FIGURE 7.20: GRADIENTCHARTTYPE SCHEMA VIEW ............................................................ 84 FIGURE 7.21: SINGLEGRADIENTCHARTTYPE SCHEMA VIEW .................................................... 85 FIGURE 7.22: MULTICHARTTYPE SCHEMA VIEW ................................................................. 86 FIGURE 7.23: REPORTITEMSTYPE SCHEMA VIEW ................................................................ 86 FIGURE 7.24: BODYTYPE SCHEMA VIEW ......................................................................... 87 FIGURE 7.25: HEADERFOOTERTYPE SCHEMA VIEW .............................................................. 87 FIGURE 7.26: REPORT SCHEMA VIEW............................................................................. 88 FIGURE 7.27: CHARTSERIESTYPE CLASS ........................................................................ 89 FIGURE 7.28: CHARTSERIESTYPE SCHEMA VIEW ................................................................ 89 FIGURE 7.29: REPORTITEMTYPE CLASS .......................................................................... 90 FIGURE 7.30: TEXTBOXTYPE CLASS .............................................................................. 91 FIGURE 7.31: SINGLELINEARCHARTTYPE CLASS ................................................................ 91 FIGURE 7.32: CHARTTYPE CLASS ................................................................................. 92 FIGURE 7.33: XYCORRELATIONCHART CLASS ................................................................... 92 FIGURE 7.34: MULTICHARTTYPE CLASS.......................................................................... 93 FIGURE 7.35: TEXTBOXTYPE CLASS .............................................................................. 94 FIGURE 7.36: REPORT DESIGNER MAIN WINDOW ............................................................... 95
- XI -
FIGURE 7.37: REPORT DESIGNER – REPORT TAB ............................................................... 96 FIGURE 7.38: REPORT DESIGNER - BODY TAB .................................................................. 97 FIGURE 7.39: REPORT DESIGNER - BODY TAB .................................................................. 98 FIGURE 7.40: REPORT DESIGNER - BODY TAB .................................................................. 99 FIGURE 7.41: REPORT BODY AFTER ADDING TEXTBOX.........................................................100 FIGURE 7.42: PREVIEW VISIBLE .................................................................................101 FIGURE 7.43: PREVIEW INVISIBLE ...............................................................................101 FIGURE 7.44: SAMPLE TEXTBOX FULLY FORMATTED ............................................................101 FIGURE 7.45: TEXTBOX RDL
CODE
.............................................................................102
FIGURE 7.46: REPORT BODY AFTER ADDING A LINEAR CHART ................................................103 FIGURE 7.47: SINGLELINEARCHARTPROPERTYEDITOR – X AXIS TAB .......................................104 FIGURE 7.48: SINGLELINEARCHARTPROPERTYEDITOR – LEFT AXIS TAB ....................................104 FIGURE 7.49: DATA CATALOGUE PICKER WITH THE SELECTED PARAMETERS FOR THE LEFT AXIS. ..........105 FIGURE 7.50: SINGLELINEARCHARTPROPERTYEDITOR – LEFT AXIS TAB AFTER ADDING SERIES. .........105 FIGURE 7.51: SINGLELINEARCHARTPROPERTYEDITOR – RIGHT AXIS TAB ..................................106 FIGURE 7.52: SINGLELINEARCHARTPROPERTYEDITOR – TITLE TAB .........................................107 FIGURE 7.53: SINGLELINEARCHARTPROPERTYEDITOR – OPTIONS TAB .....................................107 FIGURE 7.54: BODY SECTION AFTER ADDING AND FORMATTING A LINEAR CHART ...........................108 FIGURE 7.55: LINEAR CHART PREVIEW .........................................................................108 FIGURE 7.56: BODY AFTER ADDING A GRADIENT CHART ......................................................109 FIGURE 7.57: SINGLEGRADIENTCHARTPROPERTYEDITOR – X AXIS TAB ....................................110 FIGURE 7.58: SINGLEGRADIENTCHARTPROPERTYEDITOR – SERIES TAB ....................................110 FIGURE 7.59: SINGLEGRADIENTCHARTPROPERTYEDITOR – SERIES TAB AFTER SELECTING A PARAMETER 111 FIGURE 7.60: SINGLEGRADIENTCHARTPROPERTYEDITOR – GRADIENT SCALE TAB .........................111 FIGURE 7.61: BODY AFTER ADDING AND FORMATTING A GRADIENT CHART..................................112 FIGURE 7.62 – GRADIENT CHART PREVIEW.....................................................................112 FIGURE 7.63:BODY AFTER ADDING A 2D CORRELATION CHART ..............................................113 FIGURE 7.64: SINGLEXYCORRELATIONCHARTPROPERTYEDITOR – X AXIS TAB ............................114 FIGURE 7.65: SINGLEXYCORRELATIONCHARTPROPERTYEDITOR – X AXIS TAB AFTER SELECTING A PARAMETER ........................................................................................................................................114
FIGURE 7.66: SINGLEXYCORRELATIONCHARTPROPERTYEDITOR – Y AXIS TAB AFTER SELECTING A PARAMETER ........................................................................................................................................115
FIGURE 7.67: SINGLEXYCORRELATIONCHARTPROPERTYEDITOR – TIME WINDOW TAB....................115 FIGURE 7.68: SINGLEXYCORRELATIONCHARTPROPERTYEDITOR – OUTPUT SERIES TAB ..................116 FIGURE 7.69: BODY AFTER ADDING AND FORMATTING A 2D CORRELATION CHART .........................117 FIGURE 7.70: 2D CORRELATION CHART PREVIEW .............................................................117 FIGURE 7.71: BODY AFTER ADDING AND FORMATTING A 2D CORRELATION CHART .........................118 FIGURE 7.72:BODY AFTER ADDING A MULTI CHART ............................................................119 FIGURE 7.73: MULTICHARTPROPERTYEDITOR - X AXIS TAB .................................................120 FIGURE 7.74: MULTICHARTPROPERTYEDITOR - CHARTS TAB .................................................120 FIGURE 7.75: MULTICHARTPROPERTYEDITOR - CHARTS TAB AFTER ADDING A LINEAR CHART ............121 FIGURE 7.76: MULTICHARTPROPERTYEDITOR - FIRST LINEAR CHART LEFT AXIS ............................121 FIGURE 7.77: MULTICHARTPROPERTYEDITOR - FIRST LINEAR CHART TITLE .................................122
- XII -
FIGURE 7.78: MULTICHARTPROPERTYEDITOR - CHARTS TAB AFTER ADDING THE SECOND LINEAR CHART .....................................................................................................................................................122 FIGURE 7.79: MULTICHARTPROPERTYEDITOR - SECOND LINEAR CHART TITLE ..............................122 FIGURE 7.80 - MULTICHARTPROPERTYEDITOR - CHARTS TAB AFTER ADDING A GRADIENT CHART .......122 FIGURE 7.81: MULTICHARTPROPERTYEDITOR - GRADIENT CHART SERIES ..................................123 FIGURE 7.82: BODY AFTER ADDING MULTI CHART .............................................................124 FIGURE 7.83: MULTI CHART PREVIEW...........................................................................124 FIGURE 7.84: FIRST PART OF THE REPORT PREVIEW ...........................................................125 FIGURE 7.85: SECOND PART OF THE REPORT PREVIEW ........................................................125 FIGURE 8.1: PARAMETERS SCHEMA VIEW........................................................................128 FIGURE 9.1: SPACE WEATHER SIMPLE TIMESERIES STAR-SCHEMA ...........................................134 FIGURE 9.2: OCCURRENCE_DATE DIMENSION HIERARCHY AND SAMPLE MEMBERS. .........................135 FIGURE 9.3: OCCURRENCE_TIME DIMENSION HIERARCHY AND SAMPLE MEMBERS. ..........................135 FIGURE 9.4: SPACEWEATHER_PARAMETER DIMENSION HIERARCHY AND SAMPLE MEMBERS. ...............135 FIGURE 9.5: SPACECRAFT TIMESERIES STAR-SCHEMA .........................................................136 FIGURE 9.6: SPACECRAFT_PARAMETER DIMENSION HIERARCHY AND SAMPLE MEMBERS. ...................137 FIGURE 9.7: DIMENSIONMEMBER CLASS ........................................................................137 FIGURE 9.8: TIMEDIMENSIONMEMBER CLASS ..................................................................138 FIGURE 9.9: DATEDIMENSIONMEMBER CLASS ..................................................................139 FIGURE 9.10: SPACEWEATHERPARAMETERDIMENSIONMEMBER CLASS ......................................140 FIGURE 9.11: SPACECRAFTPARAMETERDIMENSIONMEMBER CLASS...........................................140 FIGURE 9.12: MDXQUERY CLASS ...............................................................................141 FIGURE 9.13: RESULTING TABLE FROM THE MDX
QUERIES ...................................................141
FIGURE 9.14: SPACEWEATHERSIMPLETS_MDXQUERY CLASS ...............................................142 FIGURE 9.15: SPACECRAFTTIMESERIES_MDXQUERY CLASS .................................................143 FIGURE 9.16: SPACEWEATHERSIMPLETS_CUBEDATA CLASS ................................................144 FIGURE 9.17: SPACECRAFTTIMESERIES_CUBEDATA CLASS ...................................................145 FIGURE 9.18: TSDATAVISUALIZATION CLASS ..................................................................146 FIGURE 9.19: PARAMETERSMETADATA CLASS ..................................................................147 FIGURE 10.1: TIME WINDOW HELPER MOCK-UP FORM ........................................................151 FIGURE 12.1: DATA CATALOGUE TREE DEFINITION SCHEMA VIEW ............................................169 FIGURE 12.2: REPORT CLASS AND RELATED CLASSES DIAGRAM ...............................................171 FIGURE 12.3: TEXTBOXTYPE CLASS AND RELATED CLASSES DIAGRAM ........................................172 FIGURE 12.4: SINGLELINEARCHARTTYPE CLASS AND RELATED CLASSES DIAGRAM..........................173 FIGURE 12.5: SINGLEGRADIENTCHARTTYPE CLASS AND RELATED CLASSES DIAGRAM ......................174 FIGURE 12.6: SINGLEXYCORRELATIONCHARTYPE CLASS AND RELATED CLASSES DIAGRAM................175 FIGURE 12.7: MULTICHARTTYPE CLASS AND RELATED CLASSES DIAGRAM ...................................176
- XIII -
Table Index TABLE 1:
SAME SAMPLE RATE AND NOT SHIFTED PARAMETERS CORRELATION SAMPLE .........................
49
TABLE 2:
SHIFTED PARAMETERS CORRELATION SAMPLE ..........................................................
49
TABLE 3: PARAMETER X [DOMINANT] TABLE 4: PARAMETER X
VS.
VS.
PARAMETER Y....................................................... 50
PARAMETER Y [DOMINANT]....................................................... 50
TABLE 5: SAMPLE COLOURS AND RGB
COMPONENT INTENSITIES ..............................................
TABLE 6:
SAMPLE FIXED GRADIENT SCALE COLOURS AND THEIR COMPONENTS
TABLE 7:
SAMPLE RELATIVE GRADIENT SCALE COLOURS AND THEIR COMPONENTS
54
................................ 55 ............................ 55
TABLE 8: SAMPLE FIXED GRADIENT SCALE ........................................................................ 56 TABLE 9: COLOUR FOR AN INTERMEDIATE SERIES VALUE ........................................................ 56 TABLE 10: GETDATA METHOD RESULTING DATA TABLE FOR N GLOBALIDS AND M ROWS ...................145
- XV -
1 Introduction 1.1
Academic Context _____________________________________________ 2
1.2
Technological and Scientific Context _______________________________ 2
1.3
Project goals __________________________________________________ 3
1.4
Document Structure ____________________________________________ 4
Introduction
1.1 ACADEMIC CONTEXT This report was written in the context of the final course project discipline of the Computer
engineering
course
of
Faculdade
de
Ciências
e
Tecnologia
/
Universidade Nova de Lisboa (FCT/UNL). This is a 12-credit discipline, in one semester, having a weekly occupation of 25 hours. In this project the weekly occupation was of 36 hours. The project was developed from January 2004 till September 2004. Professor João Moura Pires of the Computer Science department of FCT/UNL gave the academic orientation and advisory to this project. The project was developed in UNINOVA, in the Soft Computing And Autonomous Agents Group (CA3). The UNINOVA institution is a Research and Development center for new technologies like informatics, robotics and electronics. The CA3 group is a group with ~15 elements (including students, research engineers, coordinators, scientific advisors, and collaborators). The group is involved with the European Space Agency (ESA) in many projects. Most of these projects are prototype systems for the mission control room. Due to the dimension of the SEIS project, a total of 5 investigators and developers make up the SEIS team: •
Prof. João Moura Pires (scientific advisor)
•
Eng. Nuno Viana (project managment and development of the ETL block).
•
Eng. Marta Pantoquilho (quality manager and development of the DIM block)
•
Ricardo Ferreira (development of the Reporting and Analysis Tool (RAT))
•
Joaquim Neto (development of the Monitoring Tool (MT).
1.2 TECHNOLOGICAL AND SCIENTIFIC CONTEXT The project consists on a windows application for offline analysis and reporting of S/C and S/W data. Due to the large amounts of data available, the data must be stored in a Data Warehouse (DW). The DW is implemented in SQL Server 2000, and is queried using the MDX Language. Multiple data source browsing is essential in this project. The user must easily find a specific parameter from a collection of hundreds of parameters. Therefore each parameter has associated meta-information that can be used not only for documentation purposes, but also for search and organization purposes.
-2-
Project goals Data analysis is mostly done using visualizations. These visualizations should show the operator in a simple and quick way the results of sensor reading in a certain period of time, or the position of the spacecraft around the earth globe. Data analysis can be delivered in a form of a Report. A Report is a set of objects with analysis features (e.g.: charts). The Reporting component of the project allows the user to define custom reports, using a specialized designer. Therefore time visual correlation, and direct correlation between parameters can be done, enhancing the tool power in terms of data analysis.
1.3 PROJECT GOALS The main goal of this project was to develop the Reporting and Analysis Tool, continuing the work previously done in the Bases de Dados e Data Warehousing discipline. The predicted development goals of the project were: •
Enhancement of the Data Catalogue;
•
Integration of the Data Catalogue with the Data Integration Module (DIM);
•
Research and development of new visualizations;
•
Development of the Report Designer;
•
o
Research and development of a report XML representation;
o
Integration with Data Catalogue;
Development of the Report Browser.
Goals not achieved: •
The integration with the Data Integration Module (DIM) was not fully completed, because the final version of the DIM is not yet implemented. o
For test and presentation purposes a subset of the final DIM was implemented and loaded. It’s currently fully integrated with the Data Catalogue.
•
The Report Browser was not developed. o
Currently a report can be previewed in the report designer, but not browsed by a separated application like the Report Browser will do.
-3-
Introduction Additional development goals: •
Development of a centralized metadata repository o
Due to the large amount of metadata available, and the need share it through all the SEIS applications, a central metadata repository was designed and implemented.
1.4 DOCUMENT STRUCTURE To introduce the scope of the SEIS project the next chapter will provide a global overview of the SEIS project. The structure of the body of the document is similar to the structure of the application development. There are three distinct fronts of development and each one corresponds to one chapter: •
Data Catalogue;
•
Visualizations;
•
Report Designer.
And there are common components to all of these: •
Metadata;
•
Common Components.
Under each of these chapters, there will be sub-chapters with each relevant issue. The document ends with the conclusion, including future work, and some annexes.
-4-
2 SEIS Overview 2.1
Introduction __________________________________________________ 6
2.2
SEIS Applications ______________________________________________ 7
SEIS Overview
2.1 INTRODUCTION Space Weather (S/W) is a phenomenon caused by radiation and atomic particles emitted by the Sun and stars. The sun, solar wind and solar and galactic solar rays largely determine the Space Weather and its effects, while the state of Earth’s magnetosphere, ionosphere and thermosphere also bear an influence. Space Weather can influence the performance and reliability of space-borne and ground-based technological systems as well as endanger human life or health. Having so, the importance of the space weather data is crucial to the mission control operators. The Space weather data is extracted from scientific research institutes such as: NOAA/SEC, Lominincky’s Peak and WDC in Kyoto University. Space weather data is classified in parameters and events. Space weather parameters comprise but are not limited to: proton, electron and neutron fluxes, ionosonde data, radio bursts, solar activity indexes, solar wind and magnetic field disturbances values. Space weather events include geomagnetic storms, solar flares and coronal mass ejections, among others. For the mission control operators, it’s important to correlate the Space Weather data with the Spacecraft data. So SEIS also includes Spacecraft (S/C) data like telemetry, sensor readings, state, etc. The spacecraft data is extracted from existing data of three distinct on-going spacecraft missions, hosted by the European Space Agency. The three addressed missions are INTEGRAL (INTErnational Gamma-Ray Astrophysics Laboratory), ENVISAT (ENVIronmental SATellite) and XMM (X-Ray Multi-Mission). The spacecraft data consists of: spacecraft’s orbital positions for all addressed missions; ENVISAT mission’s SEUs historical database of occurrences; and the XMM satellite’s radiation sensors telemetry data. For the INTEGRAL mission, the SEIS system will extract real telemetry (from several instruments such as the INTEGRAL Radiation Environment Monitor and the Imager on-board INTEGRAL), operational data (e.g. current ground-station coverage), and environmental data (e.g. eclipse passage times) in near real-time. The SEIS system will integrate all this data in its database and provide it to the supporting Applications.
-6-
SEIS Architecture and Applications
2.2 SEIS ARCHITECTURE AND APPLICATIONS The SEIS system is composed of several components. The following diagram shows the architecture of the system:
Figure 2.1: SEIS architecture
Back-end / Infrastructure Components: •
BL1 (ETL) – functional block that is responsible for extracting data from the data sources, transforming it and load it into the Data Integration Module (BL2);
•
BL2 (Data Integration Module) – this block contains the databases for all the applications in the project. It contains a relational database for the Monitoring Tool (BL5);
•
BL3 (Forecasting Model) – provides forecasting for parameters in the system. Contains an Artificial Neural Network Importer (ANN) and a 3M Block developed by Deimos Engenharia.
-7-
SEIS Overview Front-end / Application components: •
BL4 (Reporting & Analysis Tool) - The goal of Reporting and Analysis tool (RAT) is to provide to the operators, an offline tool for data analysis (S/C Data, S/W Data, Events, Alarms). It allows data browsing, visualization, and correlation via reporting.
•
BL5 (Monitoring Tool) - The goal of the Monitoring Tool is to provide near real-time data to the spacecraft mission controllers. This data is delivered in a graphical way using time-based charts. It also features a configurable alarm system for alerting the operators about S/W or S/C conditions.
Both the applications include multi-user capabilities and configuration.
-8-
SEIS Architecture and Applications
3 Developed Tasks 3.1
Task Introduction _____________________________________________ 10
3.2
Chronogram _________________________________________________ 10
-9-
Developed Tasks
3.1 TASK INTRODUCTION For the development of the Reporting and Analysis Tool the following tasks were assigned: •
Data Catalogue – development of the data catalogue component.
•
Report Designer – development of the Report Designer component.
•
Sample Database Loading (ETL) – development of a tool that parses source data text files, extracting the information, and loading it to the sample database (DW) via stored procedures.
•
RAT Data Access for Sample Database – development of classes in the Reporting and Analysis tool for data access (using the MDX Language)
•
Metadata Repository – research and development of a global repository of metadata for the SEIS project.
•
Visualizations – development and enhancement of visualizations for the RAT tool and the MT tool.
From 28-Jun-2004 to 29-Jun-2004 it was held a progress meeting with the users of the SEIS system for presentation of the current state of the project, discussion over several open issues, and user acknowledgments.
3.2 CHRONOGRAM
Figure 3.1 - Chronogram
- 10 -
4 Reporting and Analysis Tool 4.1
Introduction _________________________________________________ 12
4.2
Data Catalogue _______________________________________________ 12
4.3
Report Designer ______________________________________________ 12
4.4
Report Browser _______________________________________________ 12
Reporting and Analysis Tool
4.1 INTRODUCTION The Reporting and Analysis Tool is a tool for offline analysis of data. All the data present in the SEIS database is available in this tool, for previewing and analysis purposes. The tool is divided in the following components: •
Data Catalogue
•
Report Designer
•
Report Browser
4.2 DATA CATALOGUE The Data Catalogue application was designed to provide access to all the available data, with special concern about multiple data source navigation. Each parameter has meta-information associated with it, and the Data Catalogue uses that information to organize the parameters in a tree, and to provide search capabilities. Once a parameter is selected, the application retrieves a sample of the data from the database, and displays it in a chart with interactive capabilities. These interactive capabilities include: moving the time window back and forth, zooming in and out, or retrieving interpolated values from the series. This way the user can have a first glimpse of the data. Additionally the application also displays data quality and statistics information about the sample retrieved.
4.3 REPORT DESIGNER The Report Designer allows the user to build his own reports. In the reports the user will be able to add several elements, like textboxes, charts or multi-charts. For the static elements (like textboxes) the user just has to specify their properties, for chart elements the user will have to select the desired parameter(s) to display, along with a time window definition. The report designer also features a preview mode, for the user to see how the report will look like, with all the charts filled with data. This preview mode is limited and doesn’t allow further interactions with the data. (For more interaction the user shall use the Report Browser)
4.4 REPORT BROWSER The Report Browser allows the user to interact with previously created reports or with pre-defined reports. This interaction is similar to the interaction allowed in - 12 -
Report Browser the data catalogue (zoom in and out, move the time window or obtain interpolations). Report data exportation is possible: the user can print the results of a report, save the report as a JPEG image, save the data (values) of a chart to a text file, or copy portions of the report to include in word documents. The pre-defined reports include: •
World Map Report – report including a plot of the world map with the orbits of one of the three spacecrafts (INTEGRAL, ENVISAT and XMM), with highlighting of Single Event Upsets and Alarms occurred, along with altitude information and other S/W Events occurrence.
•
Single Event Upset (SEU) Report – report for single event upsets in the reference INTEGRAL mission. The contents of this report are still yet to be defined.
- 13 -
Data Catalogue
5 Data Catalogue 5.1
Introduction _________________________________________________ 15
5.2
Standalone mode _____________________________________________ 15 5.2.1 User Interface __________________________________________ 16 5.2.2 Data Navigation _________________________________________ 17 5.2.3 Metadata Visualization ____________________________________ 22 5.2.4 Data Visualization _______________________________________ 23
5.3
Chart Series Picker Mode _______________________________________ 30 5.3.1 Operation modes ________________________________________ 30 5.3.2 Interface_______________________________________________ 31
- 14 -
Introduction
5.1 INTRODUCTION The data catalogue application was designed to provide access to all the available data, with special concern about multiple data source navigation, metadata displaying, data quality and statistics, as well as a first graphical visualization and interaction with the data. The data catalogue has two working modes: standalone and in picker mode, let’s start with standalone.
5.2 DATA CATALOGUE AND METADATA The behaviour of the data catalogue application is highly dependent on metadata information. The application is fully configurable using the annotations in the Metadata XSD Structure: see Displayable Elements, Searchable Elements and Sortable Elements. In this chapter, every configurable component of the Data Catalogue will be presented and a reference will be made to the metadata annotation that configures it.
5.3 STANDALONE MODE In this mode the data catalogue is run as a distinct application, not integrated with any other application component.
- 15 -
Data Catalogue
5.3.1 USER INTERFACE The Data Catalogue main window is presented below:
2
1
4 3 Figure 5.1: Data Catalogue main window
This window is divided into 4 main sections: 1. The left tabs – groups all the possible ways of getting finding data for analysis. It has a complete tree with all available parameters in the system and a search engine. 2. The right tabs – contains all the details about the currently selected parameter on the left side, and allows the user to see and interact with sample data (in the “Sample Data” tab). 3. The history panel – contains the previously visited parameters and sources, for an easy and fast track of the user’s path. This panel is a resizable sliding panel, so it can be closed if the user wants or resized when opened. By default, this panel is closed. 4. Default options panel – contains the default options for the selected parameter in the tree or in the search results.
- 16 -
Standalone mode
5.3.2 DATA NAVIGATION The Data Catalogue application was designed to provide access to all the available data, with special concern about multiple data source navigation. Each parameter has meta-information associated with it, and the Data Catalogue uses that information to organize the parameters in a tree, and to provide search capabilities. In the left tab (presented above), the user has several ways of getting into a specific parameter. The user can browse through all parameters using the “Complete Tree”, or search for a specific parameter or a group of parameters, specifying a search criteria. In the next sub-sections, each navigation mode will be fully explained.
5.3.2.1
COMPLETE TREE BROWSING
Selecting the “Complete Tree” tab in the left tab control, the user will have access to a tree containing all the available parameters, arranged in a hierarchy, based on all the parameters metadata. The building of the tree is done looking at all the distinct values of a configurable sequence of metadata elements. To obtain these values the Data Catalogue consults the metadata using the Metadata Access Library. (For more information on metadata refer to 8 - Metadata). The last level of the tree contains the parameters (leafs). For the case of a complex parameter, the complex parameter is shown as an intermediate node and the components are shown as child nodes. The configuration of the levels of the tree is based on a simple XML file, which includes a sequence of elements in the parameters metadata XML file. Refer to Annex 3 – Data Catalogue Tree Definition XML for detail on this file.
- 17 -
Data Catalogue
1
2
3 4
Figure 5.2: Complete Tree
Explanation of the meaning of each element: 1. Complete Tree treeview – displays the complete tree of parameters. 2. Simple parameter – simple parameters are shown as leaf nodes of the tree. 3. Complex parameter – the parameter “Magnetometer components” is a complex parameter, and the child nodes (Bx, By and Bz) are the components of the complex parameter. 4. Unavailable parameters – the parameters that don’t have data are shown with a red square icon, instead of a green one for those who have data.
- 18 -
Standalone mode
5.3.2.2
SEARCHING
Due to the high number of parameters in the system, a search engine was implemented in the Data Catalogue. In this engine the user can select a set of metadata fields for searching. Selecting the “Search” Tab on the left tab control:
1
4 6
7
2
8
3 5
9
Figure 5.3: Search interface
Explanation of the meaning of each element: 1. Search field combo-box – allows the user to select a field for searching. The available fields for searching are the Searchable Elements defined in 8.2 - Metadata XSD Structure. 2. Search criteria box - allows the user to specify a search criteria for the field chosen in the search field combo-box. 3. More button – allows adding another search line (search field + search criteria) to this search, specifying like this an AND condition to the search. 4. Delete button – removes this search line. 5. Sort combo-box – Specifies sorting for the search results. 6. Search button – clicking on this button, the search with the specified criteria will be done. 7. Reset button – resets the current search criteria.
- 19 -
Data Catalogue 8. Save Search button – after completing the search, the user will be able to save this search. A dialog box will show up asking the user a description for this search (shown below), and it will be listed in the Saved Searches. 9. Search results list – this list will display the search results (list of parameters).
Figure 5.4: Sample search with many fields
Figure 5.5: Sample search completed
Figure 5.6: Dialog box asking the description for the saved search
- 20 -
Standalone mode
5.3.2.3
SAVED SEARCHES
Selecting the “Saved Searches” Tab on the left tab control, the user will access a list of all the saved searches, and their results.
1
2
3
4
Figure 5.7: Saved Searches
Explanation of the meaning of each element: 1. List of saved searches – contains all the saved searches. This list is ordered alphabetically, and allows the user to rename any search, as shown on the figure below (single-clicking a search). The list of saved searches is maintained between sessions: the application saves them in a XML file. 2. Remove button – removes the selected saved search in the list. 3. Sort combo-box – sort combo-box for specifying the sort order in the search results for this saved search. 4. Search results – displays the search results for the selected saved search.
Figure 5.8: Saved Search renaming
- 21 -
Data Catalogue
5.3.3 METADATA VISUALIZATION The Data Catalogue provides access to metadata information regarding a certain parameter. Selecting a parameter using one of the methods presented in Data Navigation, metadata information will be shown on the “Details” tab of the right tab control.
1
2 3
The details tab is composed of three sliding panels, which are opened by default. Each panel contains the following items: 1. Data Info Panel – contains metadata information about the selected parameter. The metadata elements displayed in this panel are the Displayable Elements defined in 8.2 - Metadata XSD Structure. 2. Statistics Panel – In this panel available statistics for the parameter are shown. 3. Quality Panel – contains information about quality of the available data. Currently the Statistics and Quality information is not available. This information will be maintained by the DIM (Data Integration Model) and will be included in the final physical design of the data warehouse, as a data mart.
- 22 -
Standalone mode
5.3.4 DATA VISUALIZATION Once a parameter is selected, the application retrieves a sample of the data from the database, and displays it in a chart with interactive capabilities. These interactive capabilities include: moving the time window back and forth, zooming in and out, or retrieving interpolated values from the series. This way the user can have a first glimpse of the data.
5.3.4.1
SAMPLE DATA FOR A SINGLE PARAMETER
Selecting a single parameter in the tree (e.g.: P1 – Protons from 0.8-4 MeV), and clicking on the “Sample Data” tab in the right tab control:
1
4
2
3
5 6
Figure 5.9: Sample data for a single parameter
In this tab all the interactivity with the data occurs. This tab contains the following items: 1. Data Info – A resumed version of the Data Info panel presented above. The items displayed in this panel are the Displayable Elements having the smallDisplayable attribute (Boolean) set to true, defined in 8.2 - Metadata XSD Structure.
- 23 -
Data Catalogue 2. Time Format options – specifies the time format that the user will see the Sample Data. 3. Time Window – Information about the current time window, as well as zoom level, and aggregation level for the data. 4. Sample Data – Tabular display of the data within the current time window. 5. Sample Chart options – Set of options for the sample chart. 6. Sample Chart – Graphical display of the data within the current time window.
5.3.4.2
SAMPLE DATA FOR A COMPLEX PARAMETER
Selecting a complex parameter in the tree (e.g.: “Magnetometer components” containing 3 components: “Bx”, “By” and “Bz”). Clicking on the “Sample Data” tab in the right tab control:
Figure 5.10: Sample data for a complex parameter
Each component is displayed as a different column in the sample data tabular display. Instead of displaying a single chart in the Sample Chart area, a Multi - 24 -
Standalone mode Chart is displayed with distinct charts for each component. In this case we a chart for the “Bx” component, another for the “By” and other for the “Bz”. The colours chosen to display each chart are based on the default options for each of the component parameters.
5.3.4.3
TIME WINDOW DEFINITION
In the Sample Data tab, after selecting a specific parameter, a sample of the data will be shown. The time window considered by default is the whole time window when data is available, and the aggregation level will be set to the minimum possible, displaying the data in it’s normal granularity. In the DP progress meeting it was purposed that the sample data should only display the last 3 days of available data. This feature will be implemented later, when the DIM physical implementation is ready.
5.3.4.3.1 CHANGING AGGREGATION LEVEL The time window information group is in the Sample data for a single parameter tab.
Figure 5.11: Time window information group
In the aggregation level combo-box, the user will be able to select a different aggregation level, which will allow displaying of summarized data. (This is possible to do within the current time window). The summary function being used is the average function (for more information refer to 9.2.1 – Sample Database). The aggregation levels available for a given parameter is part of the Metadata information. The levels displayed here are obtained from the Metadata Access Library, calling the GetAggregationLevels function. The levels are saved in the XML file containing each parameter, and the available aggregation level element is identified in the Metadata XSD Structure using the Element Identifiers annotations – AggregationLevels.
- 25 -
Data Catalogue Changing the aggregation level has the following effect:
Figure 5.12: Sample chart minute aggregation level (parameter grain level)
Figure 5.13: Sample chart hour aggregation level (hourly averages)
Figure 5.14: Sample chart half-day aggregation level (half-day / 12 hour averages)
Figure 5.15: Sample chart day aggregation level (daily averages)
- 26 -
Standalone mode
5.3.4.3.2 CHANGING THE TIME WINDOW Selecting a new time window consists on “zooming in” in the chart. The user just has to make a selection in the sample chart with the left mouse button pressed as show on the figure below:
Figure 5.16: Sample chart zooming in
After releasing the left mouse button, a new time window will be considered (in this case from 2004.124 00:00 to 2004.126 11:04).
Figure 5.18: Time window detail after zooming in
And we are in our first zoom level. A zoom level consists on a time window definition.
It’s possible to slide the current 1 Figure 5.17: Sample Data after zooming in
time
window
backward or forward in time using
the
sample
chart
scroll bar (1).
The user can zoom-in how many times he wants and he can always return to the previous level zooming out. Zooming out is done right clicking in the chart. The whole history of zoom levels is kept in memory, so it is possible to return to any of the previous levels.
- 27 -
Data Catalogue The same behaviour applies when a complex parameter is selected:
Figure 5.19: Sample complex chart zooming in
Figure 5.20: Sample complex chart after zooming in
- 28 -
Standalone mode
5.3.4.4
PIN POINTING SPECIFIC AND INTERPOLATED VALUES
The use of the sliding marker in the sample chart allows the user to pinpoint specific and interpolated values within the current time window. Moving the mouse above the sample chart will cause the sliding marker to move. A legend is displayed near the mouse position informing the user about the interpolated value in that specific time, and that point is highlighted with a square in the sample chart. These features allow the user to spot what is the exact value of a peak or some interesting part of the chart. The user can also turn off the sliding marker using the check box above the chart.
Figure 5.21: Sliding marker moving
Using no interpolation, the sliding marker only points to real series points, instead of interpolated ones, like show on the image
Figure 5.22: Sliding marker moving with no interpolation
For more information and details on the sliding marker, refer to 6.2.1.3.1 – Sliding Marker. The available interpolation methods for a given parameter are also part of the parameters metadata. The available interpolation method element is identified in the Metadata XSD Structure using the Element Identifiers annotations – InterpolationMethods.
- 29 -
Data Catalogue
5.4 CHART SERIES PICKER MODE Due to the advanced features of the Data Catalogue in terms of data navigation and visualization, it would be nice to use it for selecting parameters for a chart in a report. The Data Catalogue provides a special mode called Chart Series Picker. The Chart Series Picker shows up at the bottom of the data catalogue window with a cool blue colour, as seen on the figure below:
1
Figure 5.23: Data Catalogue with Chart Series Picker
This picker is simply a list of parameters. The user can add parameters to the list, dragging them from the Complete Tree or from the Search / Saved Search results to the Chart Series Picker area (1).
5.4.1 OPERATION MODES The picker can be initialized with an empty list of parameters, or a filled list of parameters. Its behaviour is also defined by the caller application in the initialization. There are three distinct modes of operation: •
Single parameter mode – The user can put only one parameter in the list.
- 30 -
Chart Series Picker Mode •
Single unit mode – The user can only put parameters of the same unit in the list. The number of parameters is unlimited as long as they have all the same unit.
•
Multiple unit mode – The user can put parameters of different units in the list. The number of parameters is unlimited.
Every time the user tries to add a parameter that enters in conflict with the current mode of the series picker, an error message will be displayed and that parameter won’t be added to the list.
5.4.2 INTERFACE 2 3
1 4
5
Figure 5.24: Chart Series Picker detailed interface
Explanation of the meaning of each element: 1. List of parameters – list that contains the selected parameters. 2. Remove button – button to remove the selected parameter in the list. 3. Remove All button – button to remove all the parameters in the list 4. Cancel button – button to cancel the selection of parameters and return to the Report Designer. 5. Accept button – button to accept the current selection of parameters, returning the list to the caller application.
After adding a few parameters to the picker, it will look like this:
Figure 5.25: Chart Series Picker filled with parameters
- 31 -
Visualizations
6 Visualizations 6.1
Introduction _________________________________________________ 33
6.2
Multiple Axis Charts ___________________________________________ 34 6.2.1 Linear Chart ____________________________________________ 36 6.2.2 Biplot Chart (MT Charts) __________________________________ 41 6.2.3 World map Chart ________________________________________ 43 6.2.4 2D Correlation Chart _____________________________________ 48
6.3
Single Axis Charts_____________________________________________ 52 6.3.1 Events Charts___________________________________________ 52 6.3.2 Categorical Chart ________________________________________ 53 6.3.3 Gradient Chart __________________________________________ 54
6.4
Multiple Chart Alignment (Multichart) _____________________________ 64
6.5
Implementation Details ________________________________________ 66 6.5.1 Dealing with Interpolations ________________________________ 67
- 32 -
Introduction
6.1 INTRODUCTION When we’re talking about large amounts of data, the simplest way to visualize the data is using a chart. In the SEIS project, most of the data visualization and analysis is done using charts. These charts should show the user in a simple and quick way the trend or peaks of a sensor reading during a certain period of time. The charts shouldn’t be static. Giving interactivity to the charts enhances the user experience with the data, and allows him to perform more specialized analysis. Interactivity was one of the main goals when creating these visualizations. There are standard charts, just like the plain scatter chart, with 2 axis, X and Y, but this can be enhanced, showing a third axis, adding textual annotations in the chart or even adding backgrounds to certain areas. Colour based charts are a very interesting challenge Due to the high specificity of the subject in the SEIS project, the Space Environment, specific visualizations can were developed like the World Map Chart, where satellite orbits are drawn on a world map background image and annotations can be displayed in certain parts of the orbit.
- 33 -
Visualizations
6.2 MULTIPLE AXIS CHARTS Cartesian charts are the most usual charts. In their simplest form, these charts have 2 axes: X and Y. In the [X*Y] area a single point refers to an X coordinate and a Y coordinate.
Figure 6.1: Two axis Cartesian chart
Adding to this chart one series of (X, Y) values… (Y-series)
Figure 6.2: Two axis Cartesian chart with one series
Adding another series of (X, Y) values… (Y-series)
Figure 6.3: Two axis Cartesian chart with two series
- 34 -
Multiple Axis Charts It’s still possible to define an overlapping area over the [X*Y] area. Defining a Zaxis in the right side of the chart, and with the proper description of series it’s possible to draw series simultaneously in the [X*Y] and [X*Z] areas. The X-axis is common to both of these areas therefore visual correlation is possible Y and Z series.
Figure 6.4: Three axis Cartesian chart
Adding 2 Y-series (Red and Blue lines) and one Z-series (Green tick line) the result will be the following:
Figure 6.5: Three axis Cartesian chart with two series referring the Y-axis and other referring the Z-axis
Notice that the domain values of the Z-axis are much more smaller than the ones in the Y-axis, so visual correlation must take that in account.
- 35 -
Visualizations
6.2.1 LINEAR CHART Linear charts are the simple Cartesian charts. A linear chart can have up to 2 vertical axis (Y and Z) and one X-Axis.
6.2.1.1
SERIES
The series of the chart contains the points in that make up the chart. Each series has a description and can be drawn with a specific pen. (A pen specifies a color and width)
6.2.1.1.1 INTERPOLATIONS The points of a series can be left unconnected, or connected. Connecting two points is called interpolation. The linear chart supports three modes of interpolation: •
No interpolation – no points of the series are connected
•
Linear interpolation - Each two points are connected to each other using a straight line;
•
Smoothing interpolation - smoothing mode all the points in the series are connected by a curve that crosses all of them.
In linear and smoothing interpolation modes, the points are connected by the order that they were added to the chart. Sometimes, for a given time the value for a parameter is not available. So it’s necessary to represent that in the chart series. Missing values are represented by a gap in the series line. Interpolation samples: (All these samples refer to the same series values)
Figure 6.6: Single series with no interpolation
- 36 -
Multiple Axis Charts
Figure 6.7: Series with linear interpolation
Figure 6.8: Series with smoothing interpolation
Notice that there’s a gap in the series at x=5, at x=5 the value (y) of the series was set to missing.
6.2.1.1.2 OUT-OF-LIMIT DEFINITIONS Each series can have distinct Out-Of-Limit (OOL) definitions if necessary. An OOL definition contains lower and upper limits to the series. These limits are drawn in the chart as horizontal lines, so that the user can see clearly if any of the series values are Out-Of-Limit. (A linear chart can only show OOL lines for one series at the time) It’s possible to define 4 limits in each OOL. First there are Hard and Soft OOLs: the Hard OOL is representing with a tick line, and the Soft OOL with a dashed line. Both of them there are High and Low limits.
Hard OOL High Soft OOL High Soft OOL Low Hard OOL Low
Figure 6.9: Series with active OOL sample
- 37 -
Visualizations
6.2.1.1.3 LEGEND A series legend can be shown next to the chart in 4 positions: (top, bottom, left or right). The series legend contains all the series descriptions in the chart, as well as the series color, for the user to easily identify the series line and/or points.
Figure 6.10: Linear chart with series legend at right
6.2.1.2
AXIS
The linear chart has one X-Axis and up to two vertical axis (Y and Z). The behavior and scale of each of these axes influence the way the series points and interpolations are drawn.
6.2.1.2.1 X-AXIS The X-axis allows continuous real values or time values. When using time values, the chart becomes time-based. These charts are very important in data analysis, for correlation of measures referring to the same time, or just watch the trend of a measure through the time.
Figure 6.11: Real value based X-axis Figure 6.12: Time-based X-axis
6.2.1.2.2 Y-AXIS AND Z-AXIS Both Y-Axis and Z-axis have a specific scale. Linear charts allow the user to redefine this scale, and to select logarithmic scale if necessary.
- 38 -
Multiple Axis Charts By default the scale in a chart is linear and it’s kept automatically, based on the maximum and minimum values present in all the series. Right clicking above the chart axis or double click over it the user will have access to the Axis Scale window:
Figure 6.13: Setting the axis scale
In this window the user can set the axis scale, toggle logarithmic scale and also toggle and select the displaying of a certain series OOLs. Note that the user can select only one OOL definition at the time, as said before. The logarithmic scale is using base 10, and contains a specific feature asked by the users: It assumes that log(0) = 0, for graphical purposes.
Figure 6.14: Sample chart with logarithmic scale
6.2.1.2.3 AXIS MIRRORING It’s possible to mirror vertical axis. If a chart has only a Y-axis, it’s possible to display it also on the place of the Z-axis, with exactly the same scale. The same applies to the Z-axis. This feature highly enhances readiness of the charts, featuring a double scale.
Figure 6.15: Y-Axis mirroring sample
- 39 -
Visualizations
6.2.1.3
INTERACTIVE FEATURES
Linear charts feature interactive features. These features allow the user to interact with the data in the series, enhancing and easing the analysis of data.
6.2.1.3.1 SLIDING MARKER The sliding marker is a marker that moves with the mouse along the X-Axis. At a given position the sliding marker shows and highlights the real or interpolated values of all the series. The marker is displayed as a red vertical line and its current position is shown on the top of the chart. Samples:
Figure 6.16: Sliding marker pointing series values
Figure 6.17: Sliding marker pointing interpolated series values
Figure 6.18: Sliding marker pointing missing values
- 40 -
Multiple Axis Charts
6.2.1.3.2 SELECTING RANGES IN THE X-AXIS Linear charts allow the selection of a range in the X-axis. These selections are used by the applications for zooming in. Zooming in is to redefine the time window, with a smaller width than the current, in order to display more details on the data. Zooming out can be done right clicking in the chart area. Currently the selection of ranges is completely zoom oriented, but can be in the future used for other features. (Sample: select a range, popup a context menu, and click “Save data in range”)
Figure 6.19: Sample zooming in
After selecting a range the control will raise an event, and then the containing application is responsible for completing the zoom operation. Right clicking on the chart, will cause the control to raise a zoom-out event.
6.2.2 BIPLOT CHART (MT CHARTS) Biplot charts are a derivation of the Linear Charts presented above. These charts were done specifically for the Monitoring Tool. Biplot charts are linear charts with only one axis (Y) with a maximum of two series (That why they are called Biplot charts). Limiting the number of series in these charts was done just for easing the visualization.
6.2.2.1
BIPLOT MARKERS
Biplot charts provide two special markers for getting real and interpolated values: The nowcast marker representing the current time, and a “fixed“ sliding marker that can be placed everywhere in the chart. The behaviour of this sliding marker is slightly different than the linear chart’s sliding marker: for moving this marker the user must pick it and drag it to the desired position. The interpolated or real values pointed by both markers are shown in the marker legends area, in the right part of the chart.
- 41 -
Visualizations “Fixed” sliding marker
Nowcast marker
Marker legends
Figure 6.20: Biplot chart markers and legend
6.2.2.2
BACKGROUND SHADINGS
For a given parameter being monitored in the MT, the chart of that parameter is always composed of two distinct areas: •
The past – containing real values of the parameter;
•
The future - containing predictions of the parameter.
The nowcast marker separates these areas, and each one of them should be easily
identified.
So,
distinct
background
shadings
are
applied
to
the
backgrounds of both areas.
Figure 6.21: Biplot chart with background shadings
6.2.2.3
OTHER FEATURES
It’s possible to highlight one of the series in the Biplot, left clicking in the series area in the markers legend area. The highlighting consists on marking the series points in the chart and displaying the interpolated values in the legend with a bold font.
- 42 -
Multiple Axis Charts
Figure 6.22: Biplot chart with "Series 1" highlighted
In terms of other features, it’s has the same capabilities as the Linear Chart, like zooming or setting the y-axis scale.
6.2.3 WORLD MAP CHART The world map chart is a chart where satellite orbits can be drawn over an image containing the mercator projection of the earth globe. For this map the image used was taken from the Satellite Toolkit software. It is a linear chart, where the x axis represents longitude, and goes from –180º (West) to +180º (East) and the y axis represents latitude, going from –90º (South) to +90º (North).
Figure 6.23: World Map image with the longitude and latitude axis represented
6.2.3.1
WORLD MAP CROSSING PROBLEM
Although this chart is implemented as linear chart, it has specific issues. The earth is a globe, and this map is plain. Therefore is necessary to take at account that a satellite orbit can pass from one side of the globe to the other, or from the top of the globe to the bottom. How to manage these situations? How to interpolate two points that are on “opposite” sides of the world?
- 43 -
Visualizations The following diagram explains the problem:
Figure 6.24: World map crossing problem
One possible solution to this problem was to consider edge zones in the world map, where the next series value “might” appear on the other side… it had the disadvantage of being a fixed edge zone, that might work for some orbits, but not for others (depending on the sampling rate of the orbits). The adopted solution to the problem was considering the direction of the series points. When adding a new point to the series, looking at the last two added points it’s possible to determine a direction of the orbit obtaining 2 differentials in longitude and latitude. Comparing these differentials with the new differentials that would be created when adding the current point, it’s possible to detect crossing situations: if the signal of the previous longitude differential is opposite to the signal of the new longitude differential, then we’re on a longitude crossing situation. The direction of the longitude crossing (WestÆEast or EastÆWest) then is obtained by comparing the longitude values of the previous and the current point. The same is applied to detect latitude-crossing situations (NorthÆSouth our SouthÆNorth) Using this technique, the control adds the points correctly to the series to create the crossing effect correctly.
- 44 -
Multiple Axis Charts
Figure 6.25: World map longitude crossing situation
Figure 6.26: World map latitude and longitude crossing situation
Figure 6.27: World map extreme latitude and longitude crossing situation
- 45 -
Visualizations
6.2.3.2
SINGLE EVENT UPSET AND ALARM ANNOTATIONS
In the world map chart it’s possible to annotate certain points of the orbits. The annotation is required for marking situations of Single Event Upsets (SEUs) and Alarms triggered by the Monitoring Tool Alarm Engine. A SEU or an Alarm in an orbit can be drawn in the world map as a red round cross image over the orbit line (annotation marker). Each annotation can have a description, which will contain additional information for the SEU or Alarm. Every time the user passes the mouse over an annotation marker, its description will show up like a tool tip.
Figure 6.28: World map with sample orbit with 3 annotations and one description visible
Right clicking over an annotation marker in the world map popups a context menu where the user can toggle the visibility of the corresponding orbit. This was done in request of the users, which would like to have a World Map Report with all the Single Event Upsets that occurred in a large interval of revolutions. Drawing all the revolutions would saturate the World Map chart, so the solution was to hide all the lines, showing only the SEUs occurrences using the annotation markers. At any time the user can see or hide the corresponding orbit, toggling it with this menu.
Figure 6.29: Show revolution popup menu
- 46 -
Figure 6.30: World map with hidden revolution
Multiple Axis Charts
6.2.3.3
INTERACTIVE FEATURES
The world map has interactive features to help the user.
6.2.3.3.1 SLIDING MARKER It contains a sliding marker that is similar to the one in the linear charts, informing for a given longitude, the latitude of all the visible orbits.
6.2.3.3.2 ZOOMING IN AND ZOOMING OUT From the users experience, most of the Single Event Upsets occur near the North Pole, where the earth magnetic field can’t avoid the entry of electrically charged particles at the satellites altitude. So it was given the possibility of zooming specific areas of the world map, in order to see orbits more detailed and the SEUs markers more spaced. Again, zooming in is done selecting a range in the chart. This time the range is two-dimensional:
Figure 6.31: World map chart selecting a zoom in range
- 47 -
Visualizations And the result will be:
Figure 6.32: World map chart after zooming in
With the zoomed world map the user now can clearly see the orbits and the problems more clearly than before. The sliding marker works in the same way in the zoomed chart, but now it has more resolution. The user may zoom-in further, or zoom-out right clicking in the chart.
6.2.4 2D CORRELATION CHART These charts allow direct correlation between two measures. One measure is represented in the X-Axis, and the other is represented in the Y-Axis. This way the coordinates of a given point (x, y) represent a relation between both measures.
6.2.4.1
FEATURES
Implemented as a regular linear chart, it features two real value based axis (X and Y) and might have several series with any interpolation. It’s still possible to add a Z-axis, but for simplicity let’s just work with one vertical axis.
Figure 6.33: 2D Correlation chart sample
- 48 -
Multiple Axis Charts As with linear charts, it’s possible to define the scaling for both axes, doubleclicking on the axis area or right clicking and selecting the set scale menu. The connection of two points is being done using the order that they were inserted in the chart: 1st point connects to the 2nd point, 2nd point to the 3rd, etc.
6.2.4.2
PARAMETER CORRELATION ISSUES
In the scope of the SEIS project, these charts will be useful for direct correlation of parameters: each axis will represent the values of the corresponding parameter. This direct correlation must be done taking the time of the measures into account. For a given time the system must determine the values of both parameters. This won’t be a problem if these parameters have the same sampling rate and aren’t shifted, like shown below: Table 1: same sample rate and not shifted parameters correlation sample
Parameter X Time
Value
Parameter Y Time
Value
Parameter X vs. Parameter Y Param. X
Param. Y
1
10
1
100
10
100
2
15
2
150
15
150
3
5
3
50
5
50
4
30
4
300
30
300
If the values don’t have the same sampling rate, or are time shifted, when value is available for the parameter X, the value of parameter Y might not be available in that instant. The solution pointed by the users is to use the last available value. Table 2: shifted parameters correlation sample
Parameter X Time
Value
Parameter Y Time
Value
1
-
1
45
2
20
2
-
3
-
3
55
4
35
4
-
5
-
5
30
6
15
6
-
7
-
7
25
8
10
8
-
Parameter X vs. Parameter Y Param. X
Param. Y
20
45
35
55
15
30
10
25
- 49 -
Visualizations When the values of the series are time-shifted, there’s also the need to define a dominant parameter: The dominant parameter is the reference parameter and if the value for the other parameter is not available, we shall use the last one. Changing the dominant parameter might have different results as shown in the following example: Table 3: Parameter X [Dominant] vs. Parameter Y
Parameter X
Parameter Y
Parameter X vs. Parameter Y
[Dominant] Time
Value
Time
Value
1
-
1
45
2
20
2
30
3
-
3
55
4
-
4
20
5
60
5
-
6
-
6
40
7
-
7
25
8
40
8
-
Param. X
Param. Y
20
30
60
20
40
25
Table 4: Parameter X vs. Parameter Y [Dominant]
Parameter X
Parameter Y
Parameter X vs. Parameter Y
[Dominant] Time
Value
Time
Value
Param. X
Param. Y
1
-
1
45
2
20
2
30
20
30
3
-
3
55
20
55
4
-
4
20
20
20
5
60
5
-
6
-
6
40
60
40
7
-
7
25
60
25
8
40
8
-
6.2.4.3
INTERACTIVE FEATURES
The 2D Correlation chart has interactive features to help the user.
- 50 -
Multiple Axis Charts
6.2.4.3.1 ZOOMING IN AND ZOOMING OUT 2D Correlation charts support zooming in and zooming out, in a similar way as the world map chart. Zooming in is done selecting a two dimensional area in the chart.
Figure 6.34: 2D Correlation chart selecting a zoom in range
And the result will be:
Figure 6.35: 2D Correlation chart after zooming in
The user may zoom-in further, or zoom-out right clicking in the chart.
6.2.4.3.2 TIME ITERATOR (NOT IMPLEMENTED) It would be nice to have a feature to point values in a 2D Correlation. The resulting lines of these charts might not define a function, because for a single x value there can exists multiple y values. So the x-axis based sliding marker presented in the linear charts is not useful for that purpose. What would be interesting was to create a sort of marker iterator, where the user could cycle all the points ordered by the time. At the current iterator position the user would have all the information over that point: the coordinates of the point, and the corresponding time. To accomplish this, the way of adding points to the chart must be altered: Currently we’re just adding (x, y) pairs, but we would have to add (x, y, time) triples.
- 51 -
Visualizations This iterator would have controls like: “next point”, “previous point”, “last point” or “first point” and its current position (point) would be highlighted in the chart.
6.3 SINGLE AXIS CHARTS These charts feature only a single x-axis. Generally these charts are time-based and are suitable for represent ranges or states using intervals. Additionally using coloured bars, its also possible to represent values.
6.3.1 EVENTS CHARTS In the SEIS system, an event has always a start time and an end time, specifying a time interval. It also may contain a value of intensity (provided by the measuring sensor), and a time when the measure reached the maximum. Through the time there can be many events, and at the same time it’s possible to have several events occurring.
6.3.1.1
DISPLAYING EVENTS
An event is drawn as a rectangular box in the chart (which represents the time interval), and may include a description inside of it (which can be used for writing a small description of the event and/or it’s value). If an event has a maximum time this will be drawn inside the box as a tick red line.
Figure 6.36: regular event
Figure 6.37: event with value and maximum
It’s possible to have several events occurring at the same time. To avoid overlapping of event boxes, simultaneously events are drawn in different levels or lines.
Figure 6.38: Events chart with simultaneous events
- 52 -
Single Axis Charts
6.3.1.2
INTERACTIVE FEATURES
In terms of interactivity these charts offer the same capabilities as the linear charts.
6.3.1.2.1 SLIDING MARKER The sliding marker works similarly to the one in the linear chart, but the display of the marker label is different: the active events are displayed.
Figure 6.39: Events chart sliding marker pointing two active events
6.3.1.2.2 SELECTING RANGES IN THE X-AXIS Selecting ranges in the x-axis for zooming is similar to the procedure in the linear charts.
6.3.2 CATEGORICAL CHART Categorical charts are similar to the events charts. There are plenty of categorical values in the SEIS system. For instance a parameter that determines the state of an instrument in a spacecraft can have a restricted set of values like: (“On”, “Off”) or (“Active”, “Inactive”).
6.3.2.1
DISPLAYING
CATEGORIES
Each category could have a specific colour. Because at the same time there’s only one state, the chart could be a coloured bar that changes from one colour to the other every time there is a change in the category or state. Additionally a small description of each different category is drawn over the bar.
Figure 6.40: Categorical chart
6.3.2.2
INTERACTIVE FEATURES
In terms of interactivity these charts offer the same capabilities as the linear charts.
- 53 -
Visualizations
6.3.2.2.1 SLIDING MARKER The sliding marker works similarly to the one in the linear chart, but the display of the marker label is different: the pointed category is displayed.
Figure 6.41: Categorical chart sliding marker
6.3.2.2.2 SELECTING RANGES IN THE X-AXIS Selecting ranges in the x-axis for zooming is similar to the procedure in the linear charts.
6.3.3 GRADIENT CHART Gradient charts are single axis chart with a single series. They contain a continuously coloured bar. The colours present in this bar reflect the values of the series, using a linear colour interpolation method. The scale of a gradient chart is a set of colours, for a set of series values.
6.3.3.1
COLOUR REPRESENTATION
All the colours available are composed the three-color components (Red, Green, Blue) or RGB. Each component can have intensities ranging from 0 to 255 (from darker to lighter). The compositions of the three components with these intensities make up a colour. Table 5: Sample colours and RGB component intensities
Colour
R
G
B
Black
0
0
0
White
255
255
255
Red
255
0
0
Green
0
255
0
Blue
0
0
255
255
255
0
0
255
255
255
128
0
Yellow Cyan Orange
- 54 -
Single Axis Charts
6.3.3.2
GRADIENT SCALE REPRESENTATIONS
The idea in gradient charts is to assign to a series value a certain colour. These values can be fixed values of the series, or can be relative values.
6.3.3.2.1 FIXED GRADIENT SCALE This kind of gradient scale assigns colours to fixed values of the series. Table 6: sample fixed gradient scale colours and their components
Series Value
Colour
R
G
B
Red
255
0
0
25
Yellow
255
255
0
50
Green
0
255
0
0
So, in the gradient chart when the series contains the value 0, the displayed colour should be Red, when the series contains the value 25, it should be yellow, and when the series contains the value 50, it should be green. If a value in the series is smaller than 0, it will be displayed as Red and if a value in the series is greater than 50, it will be displayed as Green. The colours just saturate for out of range values.
6.3.3.2.2 RELATIVE GRADIENT SCALE Relative gradient scales assigns colours to relative values. These relative values refer the Min and Max of a series in a chart. The idea is the same as in fixed gradient scales, but the scale is not fixed. Table 7: sample relative gradient scale colours and their components
Series Value
Colour
R
G
B
Red
255
0
0
Half (50%)
Yellow
255
255
0
Max (100%)
Green
0
255
0
Min (0%)
So, in the gradient chart, when the series is at the minimum, the displayed colour should be Red, when the series is at half ((minimum + maximum) / 2), it should be yellow, and when the series is at the maximum it should be green. The series value matching depends on the series.
- 55 -
Visualizations
6.3.3.3
GRADIENT COLOUR INTERPOLATION
How to determine the colour of an intermediate series value? This is done obtaining an interpolated value for each of the components, and then joining again all components in just one colour. This interpolation is done using the nearest border colours corresponding to the nearest series values. Table 8: Sample fixed gradient scale
Series Value
Colour
R
G
B
Red
255
0
0
25
Yellow
255
255
0
50
Green
0
255
0
0
Imagine we want to interpolate the colour for the series value y = 10. The most near components are the first ones (0 and 25, corresponding to the colours Red and Yellow). So, in this case we can say that y ∈ ]0, 25[ component group. Assuming that the Red component intensity at a certain series value is represented by R[y] where y is the series value. Using the same notation for Blue and Green components, B[y] and G[y]… The formulas for obtaining the Red, Green and Blue component intensities for a series value of y ∈ ]Min, Max[ component group are: R[y] = (y – Min) / (Max – Min) * (R[Max] – R[Min]) + R[Min]
(6.1)
G[y] = (y – Min) / (Max – Min) * (G[Max] – G[Min]) + G[Min]
(6.2)
B[y] = (y – Min) / (Max – Min) * (B[Max] – B[Min]) + B[Min]
(6.3)
Using the above equations, for y = 10 ∈ ]0, 25[ component group, the obtained colour would be: R[10] = (10 – 0) / 25 * (255 – 255) + 255 = 255 G[10] = (10 – 0) / 25 * (255 – 0) + 0 = 102 B[10] = (10 – 0) / 25 * (0 – 0) + 0 = 0 These components joined together will produce the orange colour, which would be the correct colour to use: Table 9: Colour for an intermediate series value
- 56 -
Colour
R
G
B
Orange
255
102
0
Single Axis Charts So this would be the colour to use when the series value was 10. The same process would be applied if the gradient scale was relative.
6.3.3.4
DISPLAYING GRADIENTS
The gradients are displayed in a bar above the x-axis. The bar may be continuous or may present discontinuity in case of the existence of missing values in the series.
Figure 6.42: Gradient chart
The initial gap corresponds to missing values in the series. The scale being used in the gradient is being shown at the right of the chart. This is a relative scale with 5 colours: •
Max with the red colour (which in this chart corresponds to the 9,00E+03 value of the series)
• Figure 6.43: Gradient chart
70% with the white colour (which in this chart corresponds to the 6,30E+03 value of the
scale details
series) •
…
•
Min with the blue colour (which in this chart corresponds to the 0,00E+00 value of the series)
- 57 -
Visualizations
6.3.3.5
SETTING GRADIENT SCALE
It’s possible to set the gradient scale double clicking in the gradient bar, or right clicking and selecting the “Set scale…” menu. The following window will show up:
1
2
Figure 6.44: Gradient scale dialog
In this window, a preview of the gradient scale is shown vertically (1) and the components are placed on the right of this scale (2).
6.3.3.5.1 SELECTING A COMPONENT To select an existing component is done by clicking on the component coloured triangle or in the label next to it. The selected component is then highlighted and properties for that component show up in the upper right part of the window (3). Selecting the Green (50%) component…
3
Figure 6.45: Gradient scale dialog with a selected component
6.3.3.5.2 POSITIONING A COMPONENT Positioning a component is easy. It may be changed directly in the position textbox (3) or done by dragging the coloured component triangle up and down.
- 58 -
Single Axis Charts
Drag to 40% Æ
Figure 6.46: Dragging the gradient colour component
Figure 6.47: Setting the gradient colour component position directly
Notice that the gradient preview is automatically updated to reflect these changes.
6.3.3.5.3 CHANGING THE COMPONENT COLOUR The component colour can be changed clicking on the set colour button (4) or double clicking on the component triangle (5).
4 5
Figure 6.48: changing a gradient scale component colour
- 59 -
Visualizations
6.3.3.5.4 ADDING A COMPONENT To add a component, just left clicking in the gradient components area the following menu will appear.
Figure 6.49: Add component context menu
By selecting the add component menu item, the user will be asked for a colour for the component in a regular windows colour selection dialog, and choosing a light cyan colour, the gradient will look like this:
Figure 6.50: Gradient scale after adding a component
The new component is in the gradient. Notice that the position of the component is the position where the user left-clicked in the previous step. Now the gradient has this new component, and the user can do all the operations presented previously for this component.
- 60 -
Single Axis Charts
6.3.3.5.5 REMOVING A COMPONENT Removing a component is done by dragging it away from the gradient component scale or right clicking in it and selecting the “Remove component” menu item. Selecting the Green (40%) component and right clicking in it the following context menu will appear:
Figure 6.51: Remove component context menu
And selecting the “Remove component” menu item this component will be deleted from the gradient, and the resulting gradient will be the following:
Figure 6.52: Gradient scale after removing the component
- 61 -
Visualizations
6.3.3.5.6 SETTING SCALE TYPE (FIXED OR RELATIVE) The previously presented gradient was using a relative scale, based on the series Min and Max. To change it to a fixed scale, the user just have to set exact values for the Max and Min. Clicking on one of the Max or Min checkboxes (6) or (7):
6
Clicking on the Max or Min checkbox Æ
7 Figure 6.53: Gradient scale Min
Figure 6.54: Gradient with Min
and Max checkboxes
and Max set
The gradient is now a fixed gradient with the Max at 9,00E+03 and the Min at 0,00E+00. When the user changes the scale from relative to fixed, the Max and Min limits are automatically obtained from the limits of the series in the chart. Now instead of working with relative positions, the user can specify true fixed values. For instance, we would like that the series value 7,50E+03 (7500) would have the yellow colour. Looking at the markers, the second marker counting from the bottom is placed at 1,80E+03. All we have to do is to select this marker and change it’s value to 7,50E+03.
Figure 6.55: Gradient fixed scale component value changing.
Working with fixed gradients is similar to working with relative gradients. All the presented operations of component adding, removing, moving or changing colours are done in the same way as for relative gradients. - 62 -
Single Axis Charts The resulting chart with this scale will be:
Figure 6.56: Resulting gradient chart with fixed scale
The legend now displays the components colours and values that we’ve just finished setting.
6.3.3.6
INTERACTIVE FEATURES
The gradient charts provide interactive features to help the user.
6.3.3.6.1 SLIDING MARKER The sliding marker is similar to the one used in linear charts. Now it displays the exact value of the series in that point, and the colour associated with it.
Figure 6.57: Gradient char sliding marker
6.3.3.6.2 SELECTING RANGES IN THE X-AXIS Selecting ranges in the x-axis for zooming is similar to the procedure in the linear charts.
- 63 -
Visualizations
6.4 MULTIPLE CHART ALIGNMENT (MULTICHART) It’s possible to align multiple charts together to form a Multichart. This alignment is done adjusting the left and right margins of each chart, keeping the chart area aligned. This is useful when all the charts have the same x-axis. If the x-axis is common to all the charts and is time based, this alignment helps the user comparing different measures at the same time. All kinds of charts can be aligned, and each chart contained on the Multichart maintains its functionality as a single separated chart. (For instance it’s possible to set the scale of a specific chart axis, or set the gradient scale component colours).
Figure 6.58: Chart alignment sample
Notice that the X-Axis is aligned with all the charts and is only present in the last chart.
6.4.1.1
INTERACTIVE FEATURES
The Multichart provides also interactive features. Basic interactive features of each chart presented before are available in the Multichart.
6.4.1.1.1 SLIDING MARKER The Multichart sliding marker acts similarly as a single chart’s sliding marker. The main difference is that since all the charts are aligned and have the same xaxis, the marker is drawn from top to bottom crossing all the charts, and the interpolated values for each chart are shown next to the marker. - 64 -
Multiple Chart Alignment (Multichart)
Figure 6.59: Multichart sliding marker sample
6.4.1.1.2 BIPLOT MARKERS The Multichart can house Biplot (MT) charts. This is the way the Monitoring Tool uses to display all the charts in a panel properly aligned.
Figure 6.60: Multichart with Biplot charts
Notice that the first Biplot has an OOL definition active, the second is set with logarithmic scale in its Y-axis and the third has another OOL definition. These options were set defining the Y-axis scale of each chart.
- 65 -
Visualizations
6.4.1.1.3 SELECTING RANGES IN THE X-AXIS Selecting ranges in the x-axis for zooming is similar to the procedure as for the single charts. The main difference is that now the selection is not restricted to a single chart, but it is displayed from top to bottom of the Multichart.
Figure 6.61: Multichart x-axis selection
Zoom-out is done in the same way as in each single chart: right clicking with the mouse in the chart area.
6.5 IMPLEMENTATION DETAILS For display single charts, a chart control was implemented. This control allows the user to create all the single charts presented in this chapter. For displaying multi charts, a Multichart control was implemented. This Multichart is nothing but a container which houses single charts (chart controls). It has methods for adding charts, that this control will display, size and align automatically. Both controls are part of a project that when built produce a class library DLL. This DLL is exportable to other projects, like the Monitoring Tool.
- 66 -
Implementation Details
6.5.1 DEALING WITH INTERPOLATIONS A line is defined by two points. It’s easy to get any the interpolated point in the middle of a line, given the coordinates of both points. If a certain series interpolation is set to linear, then all the points in the series will be connected in sequence with straight lines. A curve is initially defined by at least 4 points. If a certain series interpolation is set to smoothing, a curve will be drawn in the series points. So the chart control draws a cardinal spline that passes through all of the series points. The drawing of cardinal spline will be done creating small intermediate lines, approximating the curve to a linear representation with no visual degradation. So to obtain an intermediate point inside a curve, the proceeding is the same as for lines, because a curve is just a set of lines. The main difference is that now we have much larger number of intermediate points defining lines. Since all we have is points, the performance of the interpolation is determined by the speed at which we can reach the nearest boundary points of a given coordinate.
Figure 6.62: Resulting points of curve (each black point connected form the blue curve)
Using a linear structure, like an array it would be necessary to cycle through all the points, what in certain cases can take too long.
For solving that situation, a class called SegmentInterpolator was constructed. This class has a simple constructor, which receives a linear array of points. These points are arranged in such way that the complexity of searching for points is smaller.
- 67 -
Visualizations Divide and conquer… The idea is to split all the points in several sets. The splitting of these points depends on their position in the chart. Creating virtual columns in the chart…
Figure 6.63: Sample curve with virtual columns
The columns are equally spaced and have a fixed width that can be the minimum possible. So, for a given x-coordinate it’s possible to determine in which column this point would be, with an arithmetical operation O(1). All the points contained in the columns are distributed through a bi-dimensional array where the first dimension is the column index and the second is the point index. This way when it’s necessary to obtain the interpolation (y) in the series for a specific x-coordinate the following steps are done: 1. Determine the column index containing this x-coordinate. O(1) 2. Find inside the column the boundary points for the desired x coordinate. O(n) – where n is the number of points in the column. 3. Interpolate x between these boundary points and obtaining the y. O(1)
Reducing the column width to the minimum possible, the complexity of the step 2 will greatly reduced. This method greatly increased the performance of the sliding marker and the Biplot markers.
- 68 -
7 Report Designer 7.1
Introduction _________________________________________________ 70
7.2
Implementation ______________________________________________ 70 7.2.1 Report Definition Language ________________________________ 70 7.2.2 RDL Type Classes ________________________________________ 89 7.2.3 RDL Type Property Editors _________________________________ 93
7.3
User Interface________________________________________________ 95 7.3.1 Report Tab _____________________________________________ 96 7.3.2 Body Tab ______________________________________________ 97 7.3.3 Code Tab ______________________________________________ 98 7.3.4 Preview Tab ____________________________________________ 99
7.4
Report Operations____________________________________________ 100 7.4.1 Adding a Textbox _______________________________________ 100 7.4.2 Adding a Linear Chart ___________________________________ 103 7.4.3 Adding a Gradient Chart _________________________________ 109 7.4.4 Adding a 2D Correlation Chart _____________________________ 113 7.4.5 Adding a Multi Chart ____________________________________ 119 7.4.6 Previewing the Report ___________________________________ 125
- 69 -
Report Designer
7.1 INTRODUCTION The Report Designer allows the user to build his own reports. In the reports the user can add several elements, like textboxes, charts or multi-charts. For the static elements (like textboxes) the user just has to specify their properties, for chart elements the user will have to select the desired parameter(s) to display, along with a time window definition, and optionally a set of formatting properties. The report designer also features a preview mode, for the user to see how the report will look like, with all the charts filled with data. This preview mode is yet limited and doesn’t allow further interactions with the data. The reports are saved as XML files, containing the report structure (the definition of each element and their properties). Setting a report option, the output files can also contain data for all the charts in the report. This is useful for off-line browsing in a machine access to the database. This off-line browsing will be offered in the Report Browser application.
7.2 IMPLEMENTATION The basis of this application is the Report Definition Language. This language will be presented next and the application has a straight connection with it.
7.2.1 REPORT DEFINITION LANGUAGE The Report Definition Language (RDL) is a language that defines all the elements and sub-elements in a report. This RDL is defined as a XSD Schema describing all the report elements and their structure. All the generated reports are compliant with this language. (The schema of the resulting XML files is the RDL Schema). Each RDL element type is defined as a complex type in the schema. The inspiration for this approach came from the SQL Server 2000 Reporting Services RDL. Next each type defined in the schema will be presented and explained. Starting with the atomic elements, and following the more complex elements. The complete XSD is presented in Annex 1 – Report Definition Language Schema.
- 70 -
Implementation
7.2.1.1
LAYOUTTYPE
This atomic type represents the layout of an element in a report. A layout of an element must have a left and top coordinates, and a specific width and height of the element.
Figure 7.1: LayoutType schema view
7.2.1.2
TOPALIGNMENTTYPE
This atomic type was created for elements that are top aligned in the report. These elements don’t have a free layout, and must have additional information to provide ordering between top aligned elements.
Figure 7.2: TopAlignmentType schema view
7.2.1.3
REPORTITEMTYPE (ABSTRACT TYPE)
All the elements in a report, will have either a Layout or a TopAlignment. Additionally all the elements must have a name to identify them. So this type contains an element to save the name and one LayoutType element, or a TopAlignmentType element, to save the Layout or the TopAlignment details of this element. The report elements (Textboxes, and Charts) will extend this abstract type, as we’ll see later.
- 71 -
Report Designer
Figure 7.3: ReportItemType expanded schema view
7.2.1.4
FONTTYPE
This atomic type represents a font with all it’s available formatting: Font name, size, and style: bold, italic, underline and strikeout.
Figure 7.4: FontType schema view
7.2.1.5
TEXTALIGNMENTTYPE
This atomic type represents a text alignment: it can be left aligned, center aligned or right aligned.
Text alignment
- 72 -
Implementation
7.2.1.6
COLORTYPE
This type represents a colour. A colour has 3 basic components: Red, Green and Blue, and can have transparency. This transparency index works like a 4th component named Alpha.
Figure 7.5: ColorType schema view
7.2.1.7
TEXTBOXTYPE
This type represents a textbox in the report. As a report type element it must extend the previously presented ReportItemType (Abstract Type). A textbox contains a text, drawn with a font (FontType element – Font), with a text alignment (TextAlignmentType element – TextAlignment) with a specific colour (ColorType element – Color).
- 73 -
Report Designer
Figure 7.6: TextboxType expanded schema view
7.2.1.8
CHARTELEMENTPOSITIONTYPE
Certain elements inside a chart have a specific position. This position has 4 possible values: Top, Bottom, Left and Right. This type represents these possible positions.
- 74 -
Implementation
7.2.1.9
CHARTELEMENTORIENTATIONTYPE
Certain elements inside a chart have a specific orientation. This orientation can be either horizontal or vertical. This type represents these possible orientations.
7.2.1.10 CHARTTYPE (ABSTRACT TYPE) This type represents the base configurations for a chart. All the chart types will extend this type. It contains a set of common properties to all the charts, like: chart title, the title font, the position of the title (ChartElementPositionType element
–
TitlePosition),
legend
visibility,
legend
position
(ChartElementPositionType element – LegendPosition), point markers visibility, X-axis guidelines visibility, font for the values in the axis, font for the description,
and
axis
description
orientation
(ChartElementOrientationType
element – AxisDescriptionOrientation).
- 75 -
Report Designer
Figure 7.7: ChartType schema view
7.2.1.11 AGGREGATIONLEVELTYPE This type represents the possible time aggregation levels available in the system: Second, Minute, Hour, Half Day, Day, Month or Year.
7.2.1.12 INTERPOLATIONMETHODTYPE This type represents the possible interpolation methods available to connect points of the series in a chart.
- 76 -
Implementation
7.2.1.13 CHARTSERIESTYPE This type represents a series in a chart. This series refers to a single global id of a parameter in the database, contains the name and unit of the parameter, along with a description and an aggregation level to use for retrieving data (AggregationLevelType element – AggregationLevel). This type also houses parameter data, as we’ll see later in the next section
Figure 7.8: ChartSeriesType schema view
7.2.1.14 LINEARCHARTSERIESTYPE This
type
represents
series
in
a
linear
chart
type.
It
extends
the
ChartSeriesType, adding to it an interpolation method (InterpolationMethodType element – InterpolationMethod) and a colour to draw the series line and/or points.
- 77 -
Report Designer
Figure 7.9: LinearChartSeriesType schema view
7.2.1.15 CHARTSERIESLISTTYPE This type represents a list of chart series (ChartSeriesType element – Series), or a list of linear chart series (LinearChartSeriesType element – LinearSeries).
Figure 7.10: ChartSeriesListType schema view
- 78 -
Implementation
7.2.1.16 LINEARCHARTAXISTYPE This type represents a linear axis type (left or right). A linear chart axis can be a mirror of other linear chart Axis, can have a logarithmic scale or not, and contains a series list (ChartSeriesListType element – SeriesList).
Figure 7.11: LinearChartAxisType schema view
7.2.1.17 LINEARCHARTTYPE This type represents a liner chart type. A linear chart contains both left and right (X and Y) axis (LinearChartAxisType elements – YAxis and ZAxis), and also contains all the basic formatting for any chart. So this type extends the ChartType (Abstract Type) presented before.
Figure 7.12: LinearChartType schema view
- 79 -
Report Designer
7.2.1.18 TIMEFORMATTYPE This type represents the available time formats. There are two time formats that can be used in charts: Standard and Day Of Year (DOY).
7.2.1.19 TIMEWINDOWTYPE This type represents a time window. A time window is defined by a start time and an end time.
Figure 7.13: TimeWindowType schema view
7.2.1.20 CHARTXAXISTYPE This type represents a time-based chart X Axis. This definition contains a time window
(TimeWindowType
element
–
TimeWindow)
and
(TimeFormatType element – TimeFormat).
Figure 7.14: ChartXAxisType schema view
- 80 -
a
time
format
Implementation
7.2.1.21 SINGLELINEARCHARTTYPE This type represents a single chart type. This is a report element, so it extends the ReportItemType (Abstract Type) presented earlier. It contains a definition of an X-axis (ChartXAxisType element – XAxis) and a linear chart (LinearChartType element – LinearChart).
Figure 7.15: SingleLinearChartType schema view
- 81 -
Report Designer
7.2.1.22 XYCORRELATIONCHARTTYPE This type represents a XY Correlation Chart. As any chart type, it extends the base ChartType (Abstract Type). A XY Correlation Chart contains 2 series and each one is represented in each axis (ChartSeriesType elements – SeriesX and SeriesY), that are obtained in a specific time window (TimeWindowType element – TimeWindow), where one series dominates the other and having or not x and y axis with log scales. The resulting series can have a specific Interpolation Method (InterpolationMethodType element – InterpolationMethod) a colour, and a description.
Figure 7.16: XYCorrelationChartType schema view
- 82 -
Implementation
7.2.1.23 SINGLEXYCORRELATIONCHARTTYPE This type represents a single XY Correlation Chart. This is a report element, so it extends the ReportItemType (Abstract Type) presented earlier. It contains a XY Correlation Chart (XYCorrelationChartType element – XYCorrelationChart)
Figure 7.17: SingleXYCorrelationChartType schema view
7.2.1.24 GRADIENTCOLORCOMPONENTTYPE This type represents a color component in a gradient. In a gradient a linear mapping is done between a series value and a color. A component always refers to one value and one colour (ColorType element – Color).
Figure 7.18: GradientColorComponentType schema view
- 83 -
Report Designer
7.2.1.25 GRADIENTTYPE This type represents a gradient. A gradient can have fixed or relative limits, and it’s composed of various color components (GradientColorComponentType element – GradientColorComponents). For more information on gradient charts, refer to 6.3.3 – Gradient Chart.
Figure 7.19: GradientType schema view
7.2.1.26 GRADIENTCHARTTYPE This type represents a gradient chart. As any chart type, it extends the base ChartType
(Abstract
Type).
These
charts
contain
a
series
definition
(ChartSeriesType element – Series) and a gradient definition (GradientType element – Gradient).
Figure 7.20: GradientChartType schema view
- 84 -
Implementation
7.2.1.27 SINGLEGRADIENTCHARTTYPE This type represents a single gradient Chart. This is a report element, so it extends the ReportItemType (Abstract Type) presented earlier. A single gradient chart contains an x-axis definition (ChartXAxisType element – XAxis) and a gradient chart definition (GradientChartType element – GradientChart).
Figure 7.21: SingleGradientChartType schema view
7.2.1.28 MULTICHARTTYPE This type represents a multi chart. This is a report element, so it extends the ReportItemType (Abstract Type) presented earlier. A multi chart can contain linear and gradient charts (LinearChartType and GradientChartType elements – LinearChart and GradientChart). Each chart can have it’s own options and series. All chart share the same x-axis (ChartXAxisType element – XAxis), which can be shown in every chart or in a specific frequency. For more information on multi charts refer to 6.4 – Multiple Chart Alignment (Multichart)
- 85 -
Report Designer
Figure 7.22: MultiChartType schema view
7.2.1.29 REPORTITEMSTYPE This type represents a list of report items. All the extending classes of the ReportItemType (Abstract Type) are report items. It contains a TextboxType element – Textbox, a SingleLinearChartType element – SingleLinearChart, a SingleXYCorrelationChart
element
–
SingleXYCorrelationChart,
a
SingleGradientChartType element – SingleGradientChart and a MultiChartType element – MultiChart.
Figure 7.23: ReportItemsType schema view
- 86 -
Implementation
7.2.1.30 BODYTYPE This type represents the body of the report. The body is composed of a list of items (ReportItemsType element – Items).
Figure 7.24: BodyType schema view
7.2.1.31 HEADERFOOTERTYPE This type represents a header or footer in the report. Like in the previous type these sections may contain report items
(ReportItemsType element – Items),
but have a limited size in pixels.
Figure 7.25: HeaderFooterType schema view
- 87 -
Report Designer
7.2.1.32 REPORT ELEMENT This element represents a report. A report contains meta-information, like the author of the report, description, creation and modification dates, header / footer sections of the report with all their items (HeaderFooterType elements – ReportHeader, ReportFooter, PageHeader, PageFooter), and the body section (BodyType element – Body).
Figure 7.26: Report schema view
- 88 -
Implementation
7.2.2 RDL TYPE CLASSES Each of the types presented in the above section have a direct mapping to classes in the application. Each class represents one type, and knows how to read and write it self in RDL. For types that extend others (for instance LinearChartSeriesType extends ChartSeriesType) class inherence is used. The names of each RDL class are the same as the ones given to RDL types. Most of the classes offer only selection and modification functionalities like setting or getting a certain element properties, and reading or writing from or to an RDL XML file. Classes that need to have extra functionality have additional methods. Next the more complex and important classes will be presented. Full class diagram is shown on Annex 4 – RDL Classes Diagram.
7.2.2.1
CHARTSERIESTYPE CLASS
This class represents the RDL ChartSeriesType. It represents the chart series of one parameter. This class has the ability of retrieving data from the DW using the Data Access Classes. The data is retrieved and kept in the class private member (g_DataTable) in the form of a DataTable, allowing the parameter data to be cached in the class. ChartSeriesType -g_ParameterGlobalId : String -g_ParameterName : String -g_ParameterUnits : String -g_Description : String -g_Aggregation : AggregationLevelType -g_DataTable : DataTable +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String, in saveData : Boolean) +GetSeriesData(in fromDate : Date, in toDate : Date) : DataTable
Figure 7.27: ChartSeriesType class Figure 7.28: ChartSeriesType schema view
As any RDL class, it contains a method for outputting the type to an XML file (WriteXML). The data retrieved can also be saved, to the ParameterData element. Using the native .NET binary serialization features and a compression library public available done in C#.NET
- 89 -
Report Designer Saving the DataTable to the XML file is done using the following steps: 1. Binary serialize the Datatable (using the native .NET binary serialization features) 2. Compress the serialized bytes from (1) (using a public compression library written in C# – NzipLib1) 3. Encode the compressed bytes from (2) in base64 encoding (using .NET text encoding features). Loading data from the XML file to the DataTable is done inversing the steps: 1. Decode the base64-encoded data. 2. Decompress the bytes from (1). 3. Deserialize the bytes from (2), and reconstruct the DataTable.
7.2.2.2
REPORTITEMTYPE CLASS
This class represents the RDL ReportItemType (Abstract Type). All the report item classes will inherit this class. ReportItemType -g_Name : String -g_Layout : LayoutType -g_TopAlignment : TopAlignmentType +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String) +CreateControl() : Object +RenderLayoutPreview(in g : Graphics)
Figure 7.29: ReportItemType class
Besides the common constructor, xml reader and writer, this class also contains two methods: CreateControl and RenderLayoutPreview. These methods will be redefined in the classes, which inherit this one. (The classes representing the items themselves). The functionality of each of these methods is explained next: •
CreateControl – This method creates the runtime control corresponding to the report item fully formatted and with data. This control will be added to the report browsing or previewing panel.
1
- 90 -
http://www.icsharpcode.net/OpenSource/SharpZipLib/Default.aspx
Implementation •
RenderLayoutPreview – This method renders a layout preview of the item to a specific graphics output to be used during the editing in the report designer.
7.2.2.3
TEXTBOXTYPE CLASS
This class represents the RDL TextboxType. As a report item, it inherits the ReportItemType Class, extending it and redefining the CreateControl and RenderLayoutPreview methods. TextboxType -g_Text : String -g_Font : FontType -g_Color : ColorType -g_TextAlignment : TextAlignmentType +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String) +CreateControl() : Object +RenderLayoutPreview(in g : Graphics)
Figure 7.30: TextboxType class
The
CreateControl
method
creates
a
label
fully
formatted.
The
RenderLayoutPreview method renders a preview of the label fully formatted.
7.2.2.4
SINGLELINEARCHARTTYPE CLASS
This class represents the RDL SingleLinearChartType. As a report item, it inherits the ReportItemType Class, extending it and redefining the CreateControl and RenderLayoutPreview methods. ReportItemType -g_Name : String -g_Layout : LayoutType -g_TopAlignment : TopAlignmentType +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String) +CreateControl() : Object +RenderLayoutPreview(in g : Graphics)
SingleLinearChartType -g_XAxis : ChartXAxisType -g_LinearChart : LinearChartType -g_ChartControl : ChartControl +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String, in saveData : Boolean) +CreateControl() : Object +RenderLayoutPreview(in g : Graphics)
Figure 7.31: SingleLinearChartType class
The CreateControl method creates a ChartControl initialized as a linear chart, fully formatted and with data in all the series. The RenderLayoutPreview method - 91 -
Report Designer renders a preview of the resulting chart without data. Just layout and formatting properties are shown.
7.2.2.5
CHARTTYPE CLASS
This class represents the RDL ChartType (Abstract Type). It contains the common formatting options for a chart. ChartType -g_Title : String -g_TitleFont : FontType -g_TitlePosition : ChartElementPositionType -g_ShowLegend : Boolean -g_LegendPosition : ChartElementPositionType -g_MarkPoints : Boolean -g_XGuideLines : Boolean -g_AxisFont : FontType -g_DescriptionFont : FontType -g_AxisDescriptionOrientation : ChartElementOrientationType +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String)
Figure 7.32: ChartType class
7.2.2.6
XYCORRELATIONCHARTTYPE CLASS
This class represents the RDL XYCorrelationChartType. It’s a chart class, so it inherits the ChartType Class. XYCorrelationChart -g_TimeWindow : TimeWindowType -g_SeriesX : ChartSeriesType -g_SeriesY : ChartSeriesType -g_DominantX : Boolean -g_DominantY : Boolean -g_XAxisLogScale : Boolean -g_YAxisLogScale : Boolean -g_InterpolationMethod : InterpolationMethodType -g_Color : ColorType -g_SeriesDescription : String +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String, in saveData : Boolean) +FormatChartControl(in chart : ChartControl) +AddDataSeries(in chart : ChartControl, in fromDate : Date, in toDate : Date)
Figure 7.33: XYCorrelationChart class
This class contains the regular elements, constructor, reading and writing to XML file, and two additional methods: •
FormatChartControl(in chart: ChartControl) – formats a chart control with the basic formatting (given from the ChartType Class) and with the additional options of this class;
•
AddDataSeries(in chart: ChartControl, in fromDate: Date, in toDate: Date) – adds data series to a formatted chart.
- 92 -
Implementation
7.2.2.6.1 DEALING WITH PARAMETER CORRELATION This chart creates a correlation between two parameters. These parameters are represented in this class with the private members g_SeriesX and g_SeriesY. The dominant parameter in the correlation is determined by the state of the g_DominantX and g_DominantY private members. The data for each series is obtained using the GetSeriesData method of each ChartSeriesType Class – SeriesX and SeriesY. The (x, y) pairs are created and added to the chart using the method presented in 6.2.4.2 – Parameter Correlation .
7.2.2.7
MULTICHARTTYPE CLASS
This class represents the RDL MultiChartType. As a report item, it inherits the ReportItemType Class, extending it and redefining the CreateControl and RenderLayoutPreview methods. MultiChartType -g_XAxis : ChartXAxisType -g_XAxisRepeatFrequency : Integer -g_Charts : ChartType -g_MultiChartControl : MultiChartControl +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String, in saveData : Boolean) +CreateControl() : Object +RenderLayoutPreview(in g : Graphics)
Figure 7.34: MultiChartType class
The CreateControl method creates a MultiChartControl containing all the charts fully formatted and with data in all the series. The RenderLayoutPreview method renders a preview of the resulting Multichart with all the charts without data. Just layout and formatting properties are shown.
7.2.3 RDL TYPE PROPERTY EDITORS Each RDL type class has a specific property editor. A property editor is a control for editing the properties for a RDL type class. All the property editors share the same behavior: •
The initialization is done passing a reference to the RDL class object to edit properties for.
•
Each property of the object passed is shown on the property editor.
•
Every time the user changes a PropertyChanged event is raised.
- 93 -
Report Designer A Property editor for a complex RDL type (which uses other RDL types) includes in itself property editors for each of the used RDL types. This is done easily thanks to the behavior of property editors presented above (it’s possible to chain several property editors). This technique simplifies the development of new property editors, since used RDL type property editors can be reused, and all the property editors become simpler. A property editor can include some processing capabilities but just for property handling, since the RDL type classes should do all report visible and functional processing.
7.2.3.1
SAMPLE PROPERTY EDITOR - TEXTBOXPROPERTYEDITOR
This editor allows the user to set properties for a TextboxType Class instance. Looking at the Textbox type class: TextboxType -g_Text : String -g_Font : FontType -g_Color : ColorType -g_TextAlignment : TextAlignmentType +New() +ReadXML(in node) +WriteXML(in writer, in rootElementName : String) +CreateControl() : Object +RenderLayoutPreview(in g : Graphics)
Figure 7.35: TextboxType class
This class has private members that refer to other RDL elements. •
g_Font: FontType – Refers to a FontType class (for the RDL FontType)
•
g_Color: ColorType – Refers to a ColorType class (for the RDL ColorType)
•
g_TextAlignment: TextAlignmentType – Refers to a TextAlignmentType class (for the RDL TextAlignmentType)
Each of these classes has their own property editors, so there’s a property editor for a FontType class (FontPropertyEditor), a property editor for a ColorType class (ColorPropertyEditor), and a property editor for a TextAlignmentType class (TextAlignmentPropertyEditor). Therefore the TextboxPropertyEditor will use all these editors, initializing them with the private TextboxType class instance members. For each of these internal editors, the PropertyChanged event will be caught, and will cause the TextboxPropertyEditor to raise its own PropertyChanged event.
- 94 -
User Interface
7.3 USER INTERFACE The report designer user interface is a tab-like windows user interface, with a simple toolbar for managing the report.
1 2
Figure 7.36: Report Designer main window
This window contains the following items: 1. Main toolbar – provides basic functionality, for creating a new report, load a report or save this report. 2. Main tab – allows the user to switch to other sections of the report. In the following sections the contents of each tab will be explained.
- 95 -
Report Designer
7.3.1 REPORT TAB
1 2
3
4 Figure 7.37: Report Designer – Report tab
1. Add sections toolbar - provides the possibility to add various sections to this report. 2. Report Info group – displays metadata for this current report, allowing the user to set the Author and the Description fields. The Creation Date, Modification Date and Refresh Date are automatically maintained by the application, so there's no need to edit them. 3. Options group – displays options for this report. Currently the only option available is to save series data to the report. 4. Properties sliding panel – allows the user to set properties for a specific item (it will be detailed later)
- 96 -
User Interface
7.3.2 BODY TAB
1 2
Figure 7.38: Report Designer - Body tab
In this tab the body of the report is shown. This tab contains the following elements: 1. Body Section Toolbar – used to add elements to the body section. Currently there are 5 elements available: Textbox, Chart, Multi Chart, 2D Correlation Chart and Gradient Chart. 2. Body Section Contents – the contents of the body will show up in this area of the form.
- 97 -
Report Designer
7.3.3 CODE TAB
2
1
Figure 7.39: Report Designer - Body tab
In this tab the current XML code of the report is shown. This code is compliant with the RDL Language presented above. This tab is present in the Report Designer at this stage for debugging only, and will be removed in the final release of the project. This tab contains the following elements: 1. Report Sections and Items tree – this tree contains all the report sections (body, headers and footer) and all the items inside each section. When adding new sections and elements, they will show up in the tree. 2. XML Code textbox – in this box the XML Code is shown for the selected section/item in the Report Sections and Items tree.
- 98 -
User Interface
7.3.4 PREVIEW TAB
1
Figure 7.40: Report Designer - Body tab
In this tab the preview of the report is shown. This preview includes data retrieved from the DW or from the report file itself. 1. Report Preview Panel – the report elements will show up in this panel.
- 99 -
Report Designer
7.4 REPORT OPERATIONS In this section the user interaction with the report designer will be explained, adding new elements to the report or setting properties for existing ones. These elements will be added in the Body section of the report.
7.4.1 ADDING A TEXTBOX The simplest element in a report is the textbox. A textbox displays a text in the report according to a certain formatting. This text can be either single line or multi line, and the formatting applies to the font being used, the color of the text and its horizontal alignment. Clicking on the “Textbox” button in the body section toolbar presented above one textbox will be added to the report:
1
2
3
4
5 7 6
8 9
10
Figure 7.41: Report Body after adding Textbox
Explanation of the meaning of each element: 6. Element header – all the element headers look like this in the report. An informative icon is shown indicating the type of the element added and next to it the element’s name. 7. Header toolbar – in this toolbar the user can delete the element from the report section (X button) or using the right most button to hide the - 100 -
Report Operations preview of this item. Once the preview is hidden, this button changes it’s behaviour, showing the preview once clicked again:
Figure 7.42: Preview visible
Figure 7.43: Preview invisible
8. Element Preview – shows a graphical preview of the element. 9. Textbox Property Editor Control area – in this area the Textbox Property Editor Control presented above is shown. Textbox Properties Items: 10. Name – allows the user to specify a name for this textbox. This is optional, and the default value is “Textbox” 11. Alignment Properties – controls to size the height of the textbox element and to set its order among the others. 12. Text – the text to be displayed in the textbox. 13. Font – the font to be used for the entire textbox. 14. Colour Controls – allows the user to choose a colour for the textbox text, and optionally a transparency level (slider). 15. Text Alignment toggle buttons – to specify the text alignment for this textbox After setting some properties, the sample textbox looks like this:
Figure 7.44: Sample textbox fully formatted
- 101 -
Report Designer Now this item belongs to the Body section of the report, we can check this looking at the code output of this report clicking on the Code Tab:
Figure 7.45: Textbox RDL code
This code represents our sample textbox. Note that on the Report Sections and Items tree we have our textbox under the body section. For more details on the RDL textbox type refer to TextboxType in the Report Definition Language section.
- 102 -
Report Operations
7.4.2 ADDING A LINEAR CHART Clicking on the “Linear Chart” button in the body section toolbar presented above one single linear chart will be added to the report:
1
3 2
Figure 7.46: Report Body after adding a Linear Chart
Explanation of the meaning of each element: 1. Element header and preview – same behaviour as the textbox element header but this time the preview shown is for a single chart. 2. Element name and Alignment – same behaviour as said previously. 3. Single Linear Chart Property Editor – property editor for the SingleLinearChartType.
7.4.2.1
SINGLE LINEAR CHART PROPERTY EDITOR
In the following sections each tab of the Single Linear Chart Property Editor will be explained.
- 103 -
Report Designer
7.4.2.1.1 X AXIS TAB This tab includes options for the X-axis of the chart. The X-axis of these types of charts is always time based, so setting the X-axis is done defining a time window and a time format.
1
2
3
Figure 7.47: SingleLinearChartPropertyEditor – X Axis tab
Explanation of the meaning of each element: 1. Time Format – specifies the time format that should be used in this single chart. The default value for this is set on the global options for the application. 2. Time Window definition – to define a time window that the user would like to analyze in the chart. Setting no time window will cause the chart to display all the available data for all the chart parameters. 3. Reset button – resets the current time window.
7.4.2.1.2 LEFT AXIS TAB This tab includes options for the Left Axis of the chart. In each chart axis the user can add parameters, and set options for the series of the parameters. 1
2
3 4
Figure 7.48: SingleLinearChartPropertyEditor – Left Axis tab
Explanation of the meaning of each element: 1. Axis Units – read-only textbox informing the user about the axis units. The units depend on the selected parameters for the axis. 2. Log Scale checkbox – for the user to set logarithmic scale for this axis. 3. Axis Series List – the series in this axis will be listed here. - 104 -
Report Operations 4. Add / Remove Series button – for adding or removing series to this axis. Clicking on the Add/Remove Series button the user will be able to specify what parameters he wants to have in this single chart. The Data Catalogue will be called in Chart Series Picker Mode for the Single Unit mode. Single Unit mode was used since we are editing the series for only one axis, therefore, an axis can only support a single unit.
Figure 7.49: Data Catalogue picker with the selected parameters for the Left axis.
Clicking on the accept button, the user will return to the Report Designer main window, and the Left Axis Tab will look like the following:
1
2
2 3
4 5
6
Figure 7.50: SingleLinearChartPropertyEditor – Left Axis tab after adding series.
- 105 -
Report Designer As we can see now the series are listed in the series list. These series were added here with their default options set in the data catalogue (colour for the series, aggregation level and interpolation method). Explanation of the meaning of each element: 1. Axis Units – now the axis units textbox is filled according to the units of the selected parameters 2. Parameter Name and Units – informative fields about the parameter selected in the series list. 3. Description – allows the user to specify a description for this series. The description will show up in the chart’s legend. 4. Aggregation Level – to specify an aggregation level for viewing data of this series. 5. Interpolation Method – interpolation method to use for this series. 6. Colour box – to specify the colour for this series, and optionally a transparency level (slider)
7.4.2.1.3 RIGHT AXIS TAB This tab is similar to the Left Axis Tab. In each chart axis the user can add parameters, and set options for the series of the parameters. If the user doesn’t want to add more series to the chart, he can leave the Right Axis empty or make it to mirror the Left Axis. Refer to 6.2.1.2.3 – Axis Mirroring.
1
Figure 7.51: SingleLinearChartPropertyEditor – Right Axis tab
Explanation of the meaning of each element: 1. Mirror Left Axis – check this check box to make the Right axis a mirror of the Left Axis.
- 106 -
Report Operations
7.4.2.1.4 TITLE TAB This tab allows the user to set a title for the chart.
1 2 3
Figure 7.52: SingleLinearChartPropertyEditor – Title tab
Explanation of the meaning of each element: 1. Title – allows the user to set a chart title (empty for no title chart) 2. Font – defines the font to use for the chart title 3. Position – determines the position of the title in the chart. Possible values: Top, Bottom, Left or Right.
7.4.2.1.5 OPTIONS TAB This tab allows to user to set other options for the chart.
4
1
5
2 3
6 7
Figure 7.53: SingleLinearChartPropertyEditor – Options tab
Explanation of the meaning of each element: 1. Axis Font – allows the user to set a font to use for the displaying of the axis values. 2. Description Font – allows the user to set a font to use for the descriptions in this chart (axis units descriptions and legend) 3. Vertical Axis description Orientation – determines the orientation of the axis units descriptions in the vertical axis (left & right). 4. Show legend check box – defines if the legend is shown or not 5. Legend Position combo box – defines the position of the legend. Possible values: Right, Left, Top or Down. - 107 -
Report Designer 6. Mark points check box – determines if the series points should be marked or not in the chart. 7. Show X Guidelines – The X Guidelines are thin lines that run across the chart height aligned with the ticks on the X Axis, to ease reading of the chart. This checkbox determines if they are on or off.
7.4.2.2
RESULTS
After adding the Linear Chart and setting these options:
7.4.2.2.1 LAYOUT The layout on the Report Designer should be the following:
Figure 7.54: Body section after adding and formatting a Linear Chart
7.4.2.2.2 PREVIEW The preview of this item will look like this:
Figure 7.55: Linear Chart Preview
- 108 -
Report Operations
7.4.3 ADDING A GRADIENT CHART Clicking on the “Gradient Chart” button in the body section toolbar presented above one single gradient chart will be added to the report:
1
Figure 7.56: Body after adding a Gradient Chart
Explanation of the meaning of each element: 1. Single Gradient Chart Property Editor – property editor for the SingleGradientChartType.
7.4.3.1
SINGLE GRADIENT CHART PROPERTY EDITOR
In the following sections each tab of the Single Linear Chart Property Editor will be explained.
- 109 -
Report Designer
7.4.3.1.1 X AXIS TAB The look and behavior of this Tab is the same as the X Axis Tab of the Single Linear Chart Property Editor.
Figure 7.57: SingleGradientChartPropertyEditor – X Axis tab
Leaving the time window definition empty so that the time window considered for previewing, will be the data availability window of the Series selected in the next tab.
7.4.3.1.2 SERIES TAB In this tab the user can select one parameter to display in the gradient. Gradient charts support only a single series. (Refer to 6.3.3 – Gradient Chart for more info).
1
Figure 7.58: SingleGradientChartPropertyEditor – Series tab
Explanation of the meaning of each element: 1. Select Parameter button – click this button to select the series for this gradient chart. Clicking on the Add/Remove Series button the user will be able to specify what parameter he wants for this single gradient chart. The Data Catalogue will be called in Chart Series Picker Mode for the Single parameter mode.
- 110 -
Report Operations After selecting one parameter in the Data Catalogue window, this tab will look like this: 1
1 2
3
Figure 7.59: SingleGradientChartPropertyEditor – Series tab after selecting a parameter
Explanation of the meaning of each element: 1. Parameter Name and Units – informative fields about the selected parameter. 2. Description – allows the user to specify a description for this parameter. 3. Aggregation Level – to specify an aggregation level for viewing data of the parameter.
7.4.3.1.3 GRADIENT SCALE TAB In this tab the user can set the gradient scale to use for displaying the parameter.
1
Figure 7.60: SingleGradientChartPropertyEditor – Gradient Scale tab
Explanation of the meaning of each element: 1. Gradient Scale editor – using this editor, the user can set the gradient scale. The presented scale presented is the default scale (relative gradient with 3 colour components). Setting and changing the scale with this editor is the same as in 6.3.3.5 – Setting Gradient Scale. The only difference is that this editor is horizontal.
- 111 -
Report Designer
7.4.3.1.4 TITLE TAB The look and behavior of this Tab is the same as the Title Tab of the Single Linear Chart Property Editor.
7.4.3.1.5 OPTIONS TAB The look and behavior of this Tab is the same as the Options Tab of the Single Linear Chart Property Editor.
7.4.3.2
RESULTS
After adding the Gradient Chart and setting these options:
7.4.3.2.1 LAYOUT The layout on the Report Designer should be the following:
Figure 7.61: Body after adding and formatting a Gradient Chart
7.4.3.2.2 PREVIEW The preview of this item will look like this:
Figure 7.62 – Gradient Chart preview
- 112 -
Report Operations
7.4.4 ADDING A 2D CORRELATION CHART Clicking on the “2D Corr. Chart” button in the body section toolbar presented above one 2d correlation chart will be added to the report:
1
Figure 7.63:Body after adding a 2D Correlation Chart
Explanation of the meaning of each element: 1. Single XY Correlation Chart Property Editor – property editor for the SingleXYCorrelationChartType.
7.4.4.1
SINGLE XY CORRELATION CHART PROPERTY EDITOR
In the following sections each tab of the Single XY Correlation Chart Property Editor will be explained.
- 113 -
Report Designer
7.4.4.1.1 X AXIS TAB In this tab the user selects the parameter to represent in the X-Axis, and can also set options for the X-Axis.
1
2
3
Figure 7.64: SingleXYCorrelationChartPropertyEditor – X Axis tab
Explanation of the meaning of each element: 1. Log Scale checkbox – checkbox for setting the x-axis logarithmic scale usage. 2. Select Parameter button – click this button to select the parameter for the x-axis. 3. Dominant parameter checkbox – to set the selected parameter as dominant
in
the
correlation.
For
more
information
on
dominant
parameters refer to 6.2.4.2 – Parameter Correlation Issues. Clicking on the Add/Remove Series button the user will be able to specify what parameter he wants for the x-axis. The Data Catalogue will be called in Chart Series Picker Mode for the Single parameter mode. After selecting one parameter in the Data Catalogue window, this tab will look like this:
1
1 2
3
Figure 7.65: SingleXYCorrelationChartPropertyEditor – X Axis tab after selecting a parameter
Explanation of the meaning of each element: 1. Parameter Name and Units – informative fields about the selected parameter. 2. Description – allows the user to specify a description for this parameter. - 114 -
Report Operations 3. Aggregation Level – to specify an aggregation level for viewing data of the parameter.
7.4.4.1.2 Y AXIS TAB In this tab the user selects the parameter to represent in the Y-Axis, and can also set options for the Y-Axis. The look and behavior of this tab is the same as the X Axis Tab. After selecting a parameter the result would be:
Figure 7.66: SingleXYCorrelationChartPropertyEditor – Y Axis tab after selecting a parameter
7.4.4.1.3 TIME WINDOW TAB In these correlation charts, there’s still the need to define a time window, as the number of points in the chart depend on it.
1
2
Figure 7.67: SingleXYCorrelationChartPropertyEditor – Time Window tab
Explanation of the meaning of each element: 1. Time Window definition – to define a time window for correlating data of both parameters. Setting no time window will cause a correlation of all the available data. 2. Reset button – resets the current time window.
- 115 -
Report Designer
7.4.4.1.4 OUTPUT SERIES TAB This tab configures the resulting correlation series in the chart.
1 2 3 Figure 7.68: SingleXYCorrelationChartPropertyEditor – Output Series tab
Explanation of the meaning of each element: 1. Series Description – description for the resulting series. The user may change this description, but it is automatically built based on the series descriptions of each select parameter. 2. Interpolation Method – interpolation method to use to connect points of the resulting series. 3. Colour box – to specify the colour for this series, and optionally a transparency level (slider).
7.4.4.1.5 TITLE TAB The look and behavior of this Tab is the same as the Title Tab of the Single Linear Chart Property Editor.
7.4.4.1.6 OPTIONS TAB The look and behavior of this Tab is the same as the Options Tab of the Single Linear Chart Property Editor.
- 116 -
Report Operations
7.4.4.2
RESULTS
After adding the 2D Correlation Chart and setting these options:
7.4.4.2.1 LAYOUT The layout on the Report Designer should be the following:
Figure 7.69: Body after adding and formatting a 2D Correlation Chart
7.4.4.2.2 PREVIEW The preview of this item will look like this:
Figure 7.70: 2D Correlation Chart preview
- 117 -
Report Designer Adding another textbox and hiding the properties panel the result will be the following:
Figure 7.71: Body after adding and formatting a 2D Correlation Chart
- 118 -
Report Operations
7.4.5 ADDING A MULTI CHART Clicking on the “Multi Chart” button in the body section toolbar presented above one multi chart will be added to the report:
1
Figure 7.72:Body after adding a Multi Chart
Explanation of the meaning of each element: 1. Multi Chart Property Editor – property editor for the MultiChartType.
7.4.5.1
MULTI CHART PROPERTY EDITOR
In the following sections each tab of the Multi Chart Property Editor will be explained.
- 119 -
Report Designer
7.4.5.1.1 X AXIS TAB In this tab the user set’s the X-Axis for the multi chart. All the charts in the multi chart will share this x-axis.
1
4 3
2
Figure 7.73: MultiChartPropertyEditor - X Axis tab
Explanation of the meaning of each element: 1. Time Format – specifies the time format that should be used in this multi chart. The default value for this is set on the global options for the application. 2. Time Window definition – to define a time window that the user would like to analyze in the multi chart. Setting no time window will cause the chart to display all the available data for each parameter in the charts. 3. Reset button – resets the current time window. 4. X Axis Display Frequency – Determines how many times the x-axis is displayed in the multi chart. The user may repeat the x-axis every chart (first option), or show only the x-axis in each “n” charts (second options).
7.4.5.1.2 CHARTS TAB In this tab the user can add and configure the charts present in the multi chart.
2
1
4
3 5
Figure 7.74: MultiChartPropertyEditor - Charts tab
Explanation of the meaning of each element: 1. Add Linear Chart button – adds a new linear chart to this Multi Chart.
- 120 -
Report Operations 2. Add Gradient Chart button – adds a new gradient chart to this Multi Chart. 3. Remove Chart button – removes the selected chart from this Multi Chart. 4. Charts list – List containing all the charts in this multi chart. 5. Move up/Move down buttons – allow the user to move up or move down the selected chart in the charts list, changing the order of the charts in the multi chart. 7.4.5.1.2.1
ADDING A LINEAR CHART
Clicking on the “Linear” button presented in Charts Tab, a linear chart will be added to the multi chart. The charts tab now looks like this:
2 1
Figure 7.75: MultiChartPropertyEditor - Charts tab after adding a Linear Chart
Explanation of the meaning of each element: 1. Charts List – now contains the chart we’ve added. 2. Linear Chart Property Editor – the property editor for the selected linear chart. This editor is similar to the one presented in Single Linear Chart Property Editor. (Left Axis Tab, Right Axis Tab, Title Tab, Options Tab). Setting the series in the Left Axis and the Chart title
Figure 7.76: MultiChartPropertyEditor - First linear chart left axis
- 121 -
Report Designer
Figure 7.77: MultiChartPropertyEditor - First linear chart title
7.4.5.1.2.2
ADDING ANOTHER LINEAR CHART
Clicking on the “Linear” button presented in Charts Tab, another linear chart will be added to the multi chart. After configuring and formatting this new chart, the charts tab looks like this:
Figure 7.78: MultiChartPropertyEditor - Charts tab after adding the second Linear Chart
Figure 7.79: MultiChartPropertyEditor - Second linear chart title
7.4.5.1.2.3
ADDING A GRADIENT CHART
Clicking on the “Gradient” button presented in Charts Tab, a gradient chart will be added to the multi chart.
2
1
Figure 7.80 - MultiChartPropertyEditor - Charts tab after adding a Gradient Chart
- 122 -
Report Operations Explanation of the meaning of each element: 1. Charts List – now contains the gradient chart we’ve added. 2. Gradient Chart Property Editor – the property editor for the selected gradient chart. This editor is similar to the one presented in Single Gradient Chart Property Editor (Series Tab, Gradient Scale Tab, Title Tab, and Options Tab). Setting the series in the gradient chart property editor, and using the default scale:
Figure 7.81: MultiChartPropertyEditor - Gradient chart series
- 123 -
Report Designer
7.4.5.2
RESULTS
After adding the Multi chart with these charts and their options:
7.4.5.2.1 LAYOUT The layout on the Report Designer should be the following:
Figure 7.82: Body after adding Multi Chart
7.4.5.2.2 PREVIEW The preview of this item will look like this:
Figure 7.83: Multi Chart preview
- 124 -
Report Operations
7.4.6 PREVIEWING THE REPORT For previewing the current report is done changing to the Preview Tab in the Main tab of the Report Designer. All the items in the body section are shown with data in their order. In this report preview mode it’s possible to interact with the data, using the Linear Chart Sliding Marker the Gradient Chart Sliding Marker or the Multi Chart Sliding Marker.
Figure 7.84: First part of the Report Preview
Figure 7.85: Second part of the Report Preview
- 125 -
Metadata
8 Metadata 8.1
Introduction ________________________________________________ 127
8.2
Metadata XSD Structure _______________________________________ 127 8.2.1 Displayable Elements ____________________________________ 129 8.2.2 Searchable Elements ____________________________________ 130 8.2.3 Sortable Elements ______________________________________ 130 8.2.4 Element Identifiers______________________________________ 131
- 126 -
Introduction
8.1 INTRODUCTION Metadata is a vital part of this project. The definition of each parameter, data source or spacecraft is part of the metadata. So, in each application component, there’s the need to access metadata information. This metadata information is currently saved on an XML file, with a predefined structure defined in a XSD file. The contents of the file are still static, but in the future these will downloaded from the Metadata Repository. This leads to the fact that the application is not using the same metadata as other SEIS applications like the Monitoring Tool. The metadata being used came from the development of the Bases de Dados e Data Warehousing work back in 2003.
8.2 METADATA XSD STRUCTURE In this chapter the current structure of the XML metadata will be presented. Currently in this metadata we’re only saving meta-information about the parameters in the system: S/W parameters and some of the S/C parameters. This structure also contains information for how to deal with metadata information in the RAT tool components (essential for both Data Catalogue and Report Designer behaviour). The following schema defines the necessary elements to use for identifying and documenting a S/W or S/C parameter: (The full schema definition is presented in Annex 2 – Parameter Metadata Schema)
The presented structure will be discussed and extended in the future, since the metadata information will be downloaded from the central repository. Having this, and to ease the development of repository views and transforms, the structure should be similar to the concept structuring in the repository. Regarding the additional information for the RAT tool components on how to deal with metadata, it will be studied if this information will continue to be present in the XSD structure, or if it will be part of a global fixed configuration XML file, kept in the repository. The last solution would be more correct, allowing us to detach configuration details from the document definition, but on the other side the some components would need to be redesigned, specially the Metadata Access Library.
- 127 -
Metadata
Figure 8.1: Parameters schema view
- 128 -
Metadata XSD Structure The “Parameter” element represents a parameter in the system. Each parameter has a unique GlobalId, so this element includes a “globalId” attribute.
In the case of component parameters of a certain complex parameter, each component parameter should have the same ParentGlobalId, which refers to the GlobalId of the complex parameter. For this effect, the “Parameter” element can also have a “parentGlobalId” attribute:
Each of the child elements of the “Parameter” can contain annotations. These annotations will control the way the application leads with metadata.
Looking at the definition of the source element:
Data source name: GOES/NOOA
The highlighted region of the XSD code can include the following application behaviour options:
8.2.1 DISPLAYABLE ELEMENTS The displayable elements are shown in the Data Catalogue Data Info region. Syntax:
8.2.1.1
HEADERTEXT ATTRIBUTE
(STRING)
This attribute defines a text to display before the element value. If it is set to “” then the header text used will be the name of the element.
- 129 -
Metadata
8.2.1.2
SMALLDISPLAYABLE ATTRIBUTE
(BOOLEAN)
When this attribute is set to “true”, this element will also show up in the Data Catalogue resumed version of the Data Info.
8.2.1.3
TOOLTIP ATTRIBUTE
(STRING)
Tool tip to show when the user passes the mouse above this element on the Data Info region, or on the resumed Data Info Region.
8.2.1.4
ORDER ATTRIBUTE
(INTEGER)
This attribute configures the order of display of this element, among other displayable elements. The idea is to create a sequence within all the displayable elements using this attribute.
8.2.1.5
HEADINGSPACE ATTRIBUTE
(INTEGER)
This attribute defines the spacing in pixels to leave before the element, in the Data Info region.
8.2.2 SEARCHABLE ELEMENTS The searchable elements are available in the Data Catalogue search tab for being selected as search fields. Syntax:
8.2.2.1
DISPLAYLOV ATTRIBUTE
(BOOLEAN)
When this attribute is set to “true” a list of values (LOV) will show up in the field criteria box, in a form of a combo-box.
8.2.3 SORTABLE ELEMENTS The sortable elements are displayed in the sort combo-box for search results. Syntax:
8.2.3.1
DEFAULTSORT ATTRIBUTE
(BOOLEAN)
When this attribute is set to “true” this element will be the default element for sorting the search results in the Data Catalogue.
- 130 -
Metadata XSD Structure
8.2.4 ELEMENT IDENTIFIERS There are some elements that have an important meaning to the application. Therefore these elements should be identified. Syntax:
Identifier
Where identifier can be one of the following: •
SourceCategory – for the source category element (S/W or S/C);
•
TimeCategory – for the time category element (Timeseries, Events)
•
MeasureCategory – for the measure category element (Real, Forecast)
•
ParameterName – for the parameter name element;
•
ParameterUnits – for the parameter units element.
•
Frequency – for the sampling frequency element;
•
AggregationLevels – for the aggregation levels element;
•
InterpolationMethods – for interpolation methods element;
- 131 -
Common Components
9 Common Components 9.1
Introduction ________________________________________________ 133
9.2
Data & Data Access __________________________________________ 133 9.2.1 Sample Database _______________________________________ 133 9.2.2 Data Access Classes_____________________________________ 137
9.3
Data Visualization ____________________________________________ 146 9.3.1 Timeseries Data Visualization _____________________________ 146
9.4
- 132 -
Metadata Access Library _______________________________________ 147
Introduction
9.1 INTRODUCTION All the application components share the same metadata, data access and data visualizations. This chapter will present libraries for those purposes and all their supporting structure.
9.2 DATA & DATA ACCESS 9.2.1 SAMPLE DATABASE Based on the Data Integration Model (DIM) a sample database was created. The Data Integration Model includes several star-schemas, for each data-domain of the SEIS Project. The sample database includes star-schemas for Simple S/W Timeseries parameters, and for Spacecraft Timeseries parameters. The sample database also included stored procedures for easing the insertion of data. A specific ETL tool was developed for loading data into the database using these stored procedures. Using this tool we were able to load a selection of S/W parameters and S/C parameters to the star-schemas. Since the data access in the Reporting & Analysis Tool is done using OLAP technologies, two OLAP cubes were created from the star-schemas. The cubes were created using Microsoft Analysis Services. In the next sections the star-schemas will be presented and the cube implementations discussed.
- 133 -
Common Components
9.2.1.1
SPACE WEATHER SIMPLE TIMESERIES
In this section a star-schema and a cube implementation is presented for dealing with Space Weather Simple Timeseries values.
9.2.1.1.1 STAR-SCHEMA In this simplified version of the star-schema, a record in the fact table represents the
value
(V_SW_Timeseries_Value)
(C_SpaceWeather_Parameter)
at
a
given
of
one
S/W
parameter
time
(C_Occurrence_Date
and
C_Occurrence_Time).
Figure 9.1: Space Weather Simple Timeseries star-schema
9.2.1.1.2 CUBE IMPLEMENTATION This star schema contains 3 dimension tables: •
D_Occurrence_Date
•
D_Occurrence_Time
•
D_SpaceWeather_Parameter
Each dimension has levels. These levels are used to compute pre-aggregates of the measures on the cube. The only pre-aggregates necessary and obvious to the SEIS system are time-based aggregates. The following dimensions were created as shared dimensions. 9.2.1.1.2.1 - 134 -
Data & Data Access OCCURRENCE_DATE DIMENSION This dimension was created using the table D_Occurrence_Date.
Year
(From the V_Year field)
Month – [1, 12]
(From the V_Month_In_Year field)
Day – [1, 31]
(From the V_Day_In_Month field)
Figure 9.2: Occurrence_Date dimension hierarchy and sample members.
9.2.1.1.2.2
OCCURRENCE_TIME DIMENSION
This dimension was created using the table D_Occurrence_Time.
Half Day – {“AM”, “PM”} (From the D_Half_Day field)
Hour – [0, 23]
(From the V_Hour field)
Minute – [0, 59] (From the V_Minutes_In_Hour field)
Second – [0, 59] (From the V_Seconds_In_Day field)
Figure 9.3: Occurrence_Time dimension hierarchy and sample members.
9.2.1.1.2.3
SPACEWEATHER_PARAMETER DIMENSION
This dimension was created using the table D_SpaceWeather_parameter.
GlobalId (From the D_Global_ID field)
Figure 9.4: SpaceWeather_Parameter dimension hierarchy and sample members.
- 135 -
Common Components Using these shared dimensions and the fact table F_SpaceWeather_Simple_TS the SpaceWeather_Simple_TS cube was created. The only measure defined for this cube was a calculated average based on the V_SW_Timeseries_Value field.
9.2.1.2
SPACECRAFT TIMESERIES
In this section a star-schema and a cube implementation is presented for dealing with Spacecraft Timeseries telemetry values.
9.2.1.2.1 STAR-SCHEMA In this simplified version of the star-schema, a record in the fact table represents the raw value (V_Raw_SC_Timeseries_Value) of the reading of one S/C parameter (C_Spacecraft_Parameter) at a given time (C_Occurrence_Date and C_Occurrence_Time).
Figure 9.5: Spacecraft Timeseries star-schema
9.2.1.2.2 CUBE IMPLEMENTATION This star schema contains 3 dimension tables:
- 136 -
•
D_Occurrence_Date
•
D_Occurrence_Time
•
D_Spacecraft_Parameter
Data & Data Access The Occurrence_Date Dimension and Occurrence_Time Dimension had already been defined. For this star schema there was only the need to create the following dimension: 9.2.1.2.2.1
SPACECRAFT_PARAMETER DIMENSION
This dimension was created using the D_Spacecraft_Parameter table.
GlobalId (From the D_Global_ID field)
Figure 9.6: Spacecraft_Parameter dimension hierarchy and sample members.
Using these shared dimensions and the fact table F_Spacecraft_Timeseries the Spacecraft_Timeseries cube was created. Again the only measure defined for this cube was a calculated average based on the V_Raw_SC_Timeseries_Value field.
9.2.2 DATA ACCESS CLASSES A set of classes was created to ease the task of accessing the data. Accessing the cube data is done using the MDX (MultiDimensional eXpressions) language.
9.2.2.1
DIMENSION MEMBER GENERATOR CLASSES
Due to the high specificity of the hierarchy levels of each dimension, and the need to refer those members in MDX queries, a set of classes were created for generating dimension members. There are a set of attributes and operations common to all the dimension members, so the DimensionMember class was created. The specific dimension member classes will inherit this class. DimensionMember -g_DimensionName : String +New(in dimensionName : String) +SetDimensionName(in dimensionName : String) +GetDimensionName() : String +GetDimensionMember() : String
Figure 9.7: DimensionMember class
- 137 -
Common Components
9.2.2.1.1 TIME DIMENSION MEMBER CLASS This class generates dimension members for the Occurrence_Time Dimension. DimensionMember -g_DimensionName : String +New(in dimensionName : String) +SetDimensionName(in dimensionName : String) +GetDimensionName() : String +GetDimensionMember() : String
«enumeration» TimeDimensionLevels +Second = 0 +Minute = 1 +Hour = 2 +HalfDay = 3
«uses»
TimeDimensionMember -g_Time : Date -g_Level : TimeDimensionLevels +GetDimensionMember() : String +New(in dimensionName : String, in level : TimeDimensionLevels) +New(in dimensionName : String, in time : Date, in level : TimeDimensionLevels) +ClearTime() +SetTime(in time : Date) +GetTime() : Date +SetLevel(in level : TimeDimensionLevels) +GetLevel() : TimeDimensionLevels
Figure 9.8: TimeDimensionMember class
After initializing this class for the “Occurrence_Time” dimension, it offers the following functionalities: •
Obtaining a valid dimension member string based on a certain time: .SetTime(“15:20:34”) .SetLevel(ItemDimensionLevels.Second) .GetDimensionMember() = “[Occurrence_Time].[PM].[15].[20].[34]”
•
Obtaining the full level name given a certain level: .ClearTime() .SetLevel(ItemDimensionLevels.Minute) .GetDimensionMember() = “[Occurrence_Time].[Minute]”
- 138 -
Data & Data Access
9.2.2.1.2 DATE DIMENSION MEMBER CLASS This class generates dimension members for the Occurrence_Date Dimension. DimensionMember -g_DimensionName : String +New(in dimensionName : String) +SetDimensionName(in dimensionName : String) +GetDimensionName() : String +GetDimensionMember() : String
«enumeration» DateDimensionLevels +Day = 0 +Month = 1 +Year = 2
«uses»
DateDimensionMember -g_Date : Date -g_DateFilled : Boolean -g_Level : DateDimensionLevels +New(in dimensionName : String, in level) +New(in dimensionName : String, in dt : Date, in level : DateDimensionLevels) +ClearDate() +SetDate(in dt : Date) +GetDate() : Date +SetLevel(in level : DateDimensionLevels) +GetLevel() : DateDimensionLevels
Figure 9.9: DateDimensionMember class
After initializing this class for the “Occurrence_Date” dimension, it offers the following functionalities: •
Obtaining a valid dimension member string based on a certain date: .SetDate(“03-05-2004”) .SetLevel(DateDimensionLevels.Day) .GetDimensionMember() = “[Occurrence_Date].[2004].[5].[3]”
•
Obtaining the full level name given a certain level: .ClearTime() .SetLevel(DateDimensionLevels.Month) .GetDimensionMember() = “[Occurrence_Time].[Month]”
- 139 -
Common Components
9.2.2.1.3 SPACE WEATHER PARAMETER DIMENSION MEMBER CLASS This class generates dimension members for the SpaceWeather_Parameter Dimension. DimensionMember -g_DimensionName : String +New(in dimensionName : String) +SetDimensionName(in dimensionName : String) +GetDimensionName() : String +GetDimensionMember() : String
SpaceWeatherParameterDimensionMember -g_GlobalId : String +New(in globalId : String) +SetGlobalId(in globalId : String) +GetGlobalId() : Date
Figure 9.10: SpaceWeatherParameterDimensionMember class
The functionality of this class is very simple: .SetGlobalId(“GID002303”) .GetDimensionMember() = “[SpaceWeather_Parameter].[GID002303]
9.2.2.1.4 SPACECRAFT PARAMETER DIMENSION MEMBER CLASS This
class
generates
dimension
members
for
the
Spacecraft_Parameter
dimension. DimensionMember -g_DimensionName : String +New(in dimensionName : String) +SetDimensionName(in dimensionName : String) +GetDimensionName() : String +GetDimensionMember() : String
SpacecraftParameterDimensionMember -g_GlobalId : String +New(in globalId : String) +SetGlobalId(in globalId : String) +GetGlobalId() : Date
Figure 9.11: SpacecraftParameterDimensionMember class
The functionality of this class is very simple: .SetGlobalId(“GID002304”) .GetDimensionMember() = “[Spacecraft_Parameter].[GID002304] - 140 -
Data & Data Access
9.2.2.2
MDX QUERY CLASSES
This class includes the common parts of an MDX Query. MDXQuery -g_CubeName : String -g_MeasureName : String -g_Columns() : DimensionMember +New(in cubeName : String, in measureName : String, in column : DimensionMember) +New(in cubeName : String, in measureName : String, in columns() : DimensionMember) +GetColumns() : DimensionMember +SetColumns(in columns() : DimensionMember) +BuildDateRangeSet(in setName : String, in dateDimensionName : String, in timeDimensionName : String, in fromDate : Date, in toDate : Date, in aggregation : AggregationLevel) : String +BuildDateRangeSet(in setName : String, in dateDimensionName : String, in timeDimensionName : String, in aggregation : AggregationLevel) : String
Figure 9.12: MDXQuery class
An MDX Query is always made to a certain cube, for obtaining a specific cube measure (to the cells), using a column specification. The column specification is composed by a list of dimension members for slicing the cube. (These columns will be Spacecraft or Space Weather parameters). The rows will contain the time, so for the MDX row specification it’s necessary to specify the time window. So the result of a MDX query should be the following:
Row Specification
Column Specification Parameter 1
Parameter 2
Parameter 3
Time 1
Measure
Measure
Measure
Time 2
Measure
Measure
Measure
Time 3
Measure
Measure
Measure
Figure 9.13: Resulting table from the MDX queries
Specifying a time window with a regular time dimension (with both date and time in the dimension) would be easy. Since we have the date and time have distinct dimensions in this project, the specification of the time window is done building a set.
The BuildDateRangeSet function builds a MDX named set to use in the row specification of the query. This set can be composed of up to three parts, depending
on
the
limits
of
the
time
window,
as
we’ll
see
next.
- 141 -
Common Components Considering the limits of the time window as: (StartDate, StartTime) Æ (EndDate, EndTime) •
If the time window starts and ends in the same day: (Time >= StartTime AND Time <= EndTime) AND (Date = StartDate = EndDate)
•
If the time window starts in one day and ends in other day: (Time >= StartTime) AND (Date = StartDate) + *
(Date > StartDate) AND (Date < EndDate)
+ (Time <= EndTime) AND (Date = EndDate)
The middle component (*) doesn’t exist when between StartDate and EndDate we have a difference of only one day.
9.2.2.2.1 SPACE WEATHER SIMPLE TIMESERIES MDX QUERY CLASS This class produces MDX queries for the Space weather Simple Timeseries. SpaceWeatherSimpleTS_MDXQuery +CUBE_NAME : String = "SpaceWeather_Simple_TS" +DATE_DIMENSION_NAME : String = "Occurrence_Date" +TIME_DIMENSION_NAME : String = "Occurrence_Time" -g_FromDate : Date -g_ToDate : Date -g_Aggregation : DateTimeLevels +New(in columns() : DimensionMember, in measureName : String, in fromDate : Date, in toDate : Date, in aggregation : AggregationLevel) +New(in columns() : DimensionMember, in measureName : String, in aggregation : AggregationLevel) +BuildMDXQuery() : String
Figure 9.14: SpaceWeatherSimpleTS_MDXQuery class
The BuildMDXQuery method will build the complete MDX query based on the initialization. The date range set will be included in the row specification of the query, using the time window defined by the fromDate and toDate attributes of the first constructor. It’s also possible to build a MDX query without time window - 142 -
Data & Data Access definition using the second New constructor. These types of queries are useful for retrieving all the data available, regardless of the time. 9.2.2.2.1.1
ALL DATA RETREIVAL MDX QUERY
This is an example of the output of the BuildMDXQuery method without specifying a time window: WITH set [DateRange] AS '{CROSSJOIN( {[Occurrence_Date].[Day].Members}, {[Occurrence_Time].[Minute].Members} )}' SELECT { [SpaceWeather_Parameter].[SW.2] } ON COLUMNS, NON EMPTY {[DateRange]} ON ROWS FROM SpaceWeather_Simple_TS WHERE [Measures].[Value]
9.2.2.2.1.2
SPECIFIC TIME WINDOW RETREIVAL MDX QUERY
This is an example of the output of the BuildMDXQuery method using a specific time window (3/5/2004 22:25 Æ 6/5/2004 5:01). Notice that the date range set is built with 3 segments as shown above. WITH set [DateRange] AS '{CROSSJOIN({[Occurrence_Date].[2004].[5].[3]},{[Occurrence_Time].[PM].[22].[25]:[Occ urrence_Time].[PM].[23].[59]})+ CROSSJOIN({[Occurrence_Date].[2004].[5].[4]:[Occurrence_Date].[2004].[5].[5]},{[Occur rence_Time].[Minute].Members})+ CROSSJOIN({[Occurrence_Date].[2004].[5].[6]},{[Occurrence_Time].[AM].[0].[0]:[Occurre nce_Time].[AM].[5].[1]})}' SELECT { [SpaceWeather_Parameter].[SW.2] } ON COLUMNS, NON EMPTY {[DateRange]} ON ROWS FROM SpaceWeather_Simple_TS WHERE [Measures].[Value]
9.2.2.2.2 SPACECRAFT TIMESERIES MDX QUERY CLASS This class produces MDX queries for the Spacecraft Timeseries. SpacecraftTimeseries_MDXQuery +CUBE_NAME : String = "Spacecraft_Timeseries" +DATE_DIMENSION_NAME : String = "Occurrence_Date" +TIME_DIMENSION_NAME : String = "Occurrence_Time" -g_FromDate : Date -g_ToDate : Date -g_Aggregation : DateTimeLevels +BuildMDXQuery() : String +New(in columns() : DimensionMember, in measureName : String, in fromDate : Date, in toDate : Date, in aggregation : AggregationLevel) +New(in columns() : DimensionMember, in measureName : String, in aggregation : AggregationLevel)
Figure 9.15: SpacecraftTimeseries_MDXQuery class
The behaviour of the BuildMDXQuery method is the same as on the Space Weather Simple Timeseries MDX Query Class. 9.2.2.2.2.1
- 143 -
Common Components ALL DATA RETRIEVAL MDX QUERY This is an example of the output of the BuildMDXQuery method without specifying a time window: WITH set [DateRange] AS '{CROSSJOIN( {[Occurrence_Date].[Day].Members}, {[Occurrence_Time].[Minute].Members} )}' SELECT { [Spacecraft_Parameter].[SC.6.0] } ON COLUMNS, NON EMPTY {[DateRange]} ON ROWS FROM Spacecraft_Timeseries WHERE [Measures].[Value]
9.2.2.2.2.2
SPECIFIC TIME WINDOW RETREIVAL MDX QUERY
This is an example of the output of the BuildMDXQuery method using a specific time window (14/5/2004 4:48 Æ 14/5/2004 8:08). Notice that the date range set is built with only one segment as shown above. WITH set [DateRange] AS '{CROSSJOIN({[Occurrence_Date].[2004].[5].[14]},{[Occurrence_Time].[AM].[4].[48]:[Occ urrence_Time].[AM].[8].[8]})}' SELECT { [Spacecraft_Parameter].[SC.6.0] } ON COLUMNS, NON EMPTY {[DateRange]} ON ROWS FROM Spacecraft_Timeseries WHERE [Measures].[Value]
9.2.2.3
CUBE DATA RETRIEVAL CLASSES
These classes are used for retrieving data from the OLAP cubes.
9.2.2.3.1 SPACE WEATHER SIMPLE TIMESERIES CUBE DATA CLASS This class retrieves data from the Space weather Simple Timeseries cube. The data is retrieved via an OleDbConnection, provided in the constructors. The Space Weather Simple Timeseries MDX Query Class is used for building the MDX Query. SpaceWeatherSimpleTS_MDXQuery +CUBE_NAME : String = "SpaceWeather_Simple_TS" +DATE_DIMENSION_NAME : String = "Occurrence_Date" +TIME_DIMENSION_NAME : String = "Occurrence_Time" -g_FromDate : Date -g_ToDate : Date -g_Aggregation : DateTimeLevels +New(in columns() : DimensionMember, in measureName : String, in fromDate : Date, in toDate : Date, in aggregation : AggregationLevel) +New(in columns() : DimensionMember, in measureName : String, in aggregation : AggregationLevel) +BuildMDXQuery() : String
«uses»
SpaceWeatherSimpleTS_CubeData -VALUE_MEASURE_NAME : String = "Value" +New(in connection : OleDbConnection, in globalIds() : String, in fromDate : Date, in toDate : Date, in aggregation : DateTimeLevels) +New(in connection : OleDbConnection, in globalIds() : String, in aggregation : DateTimeLevels) +GetData() : DataTable
Figure 9.16: SpaceWeatherSimpleTS_CubeData class
- 144 -
Data & Data Access There are two constructors for this class, one with a time window definition (first one), and other without time window definition (second one). In both constructors, the globalIds() parameter refers to S/W parameters to retrieve in the columns. Like this it’s possible to retrieve more than one parameter in a single call. The GetData method retrieves the data and returns a data table in the following format: Table 10: GetData method resulting data table for N globalIds and M rows
TimeStamp
GlobalId[0]
GlobalId[1]
...
GlobalId[N]
TimeStamp1
Value
Value
…
Value
TimeStamp2
Value
Value
…
Value
...
…
…
…
…
TimeStampM
Value
Value
…
Value
(DateTime type)
9.2.2.3.2 SPACECRAFT TIMESERIES CUBE DATA CLASS This class retrieves data from the Spacecraft Timeseries cube. The data is retrieved via an OleDbConnection, provided in the constructors. The Spacecraft Timeseries MDX Query Class is used for building the MDX Query. SpacecraftTimeseries_MDXQuery +CUBE_NAME : String = "Spacecraft_Timeseries" +DATE_DIMENSION_NAME : String = "Occurrence_Date" +TIME_DIMENSION_NAME : String = "Occurrence_Time" -g_FromDate : Date -g_ToDate : Date -g_Aggregation : DateTimeLevels +BuildMDXQuery() : String +New(in columns() : DimensionMember, in measureName : String, in fromDate : Date, in toDate : Date, in aggregation : AggregationLevel) +New(in columns() : DimensionMember, in measureName : String, in aggregation : AggregationLevel)
«uses»
SpacecraftTimeseries_CubeData -VALUE_MEASURE_NAME : String = "Value" +New(in connection : OleDbConnection, in globalIds() : String, in fromDate : Date, in toDate : Date, in aggregation : DateTimeLevels) +New(in connection : OleDbConnection, in globalIds() : String, in aggregation : DateTimeLevels) +GetData() : DataTable
Figure 9.17: SpacecraftTimeseries_CubeData class
The behavior of both constructors, and the GetData method is the same as in the Space Weather Simple Timeseries Cube Data Class.
- 145 -
Common Components
9.3 DATA VISUALIZATION Data visualization is essential for this project. After retrieving the data from the DW, using the Data & Data Access classes presented above, it’s necessary to convert the data to a readable table, to a single chart, or to a certain chart series.
9.3.1 TIMESERIES DATA VISUALIZATION The following class provides visualization functionalities for Timeseries data. TSDataVisualization -g_Data : DataTable +New(in data : DataTable) +CreateSingleChart(in chart : ChartControl, in globalId : String, in timeFormat : TimeFormat, in interpolation : InterpolationMethod, in seriesColor : Color) +AddChartSeriesValues(in chart : ChartControl, in axis : ChartAxis, in seriesNumber : Integer, in globalId : String) +CreateDataTable(in timeformat : TimeFormat, out style : DataGridTableStyle) : DataTable
Figure 9.18: TSDataVisualization class
The data is given to this class in the constructor. The DataTable expected is the resulting DataTable from the GetData() methods of Space Weather Simple Timeseries Cube Data Class or Spacecraft Timeseries Cube Data Class. This class contains the following methods: •
CreateSingleChart – creates a data catalogue like single chart for visualization of the parameter data.
•
CreateDataTable – creates a data catalogue like data table, and returns also a formatting style (with formatted column names, and background coloured lines).
•
AddChartSeries – adds series values to a certain series in a chart. This method is used mostly when previewing a report chart.
- 146 -
Metadata Access Library
9.4 METADATA ACCESS LIBRARY 9.4.1 INTRODUCTION A metadata access library was created for accessing the metadata in a completely transparent way, in all the application components. This library abstracts the programmer from the XML metadata file. This library provides accessing and searching facilities used by both Data Catalogue and Report Designer.
9.4.2 IMPLEMENTATION DETAILS The library is composed of a single class and a supporting structure as shown below: ParametersMetadata
«struct» ParameterAttribute +name : String +displayable : Boolean +headerText : String +smallDisplayable : Boolean +tooltip : String +sortable : Boolean +searchable : Boolean +searchDisplayLOV : Boolean +datatype : XmlDataType +multipleOccurs : Boolean +order : Integer +headingSpace : Integer +tableName : String +fieldName : String
«uses»
-g_SchemaDocument : XmlDocument -g_Attributes() : ParameterAttribute -g_SourceCategoryAttribute : ParameterAttribute -g_MeasureCategoryAttribute : ParameterAttribute -g_TimeCategoryAttribute : ParameterAttribute -g_ParameterNameAttribute : ParameterAttribute -g_DefaultSortAttribute : ParameterAttribute -g_ParameterSortAttribute : ParameterAttribute -g_ParameterUnitsAttribute : ParameterAttribute -g_FrequencyAttribute : ParameterAttribute -g_InterpolationMethodsAttribute : ParameterAttribute -g_AggregationLevelsAttribute : ParameterAttribute -g_XmlDocumentOK : Boolean +PARAMETER_ID_ATTRIBUTE : String = "globalId" +PARENT_PARAMETER_ID_ATTRIBUTE : String = "parentGlobalId" +New(in schemaDocumentPath : String, in xmlDocumentPath : String) +isXmlDocumentOK() : Boolean +GetParameterNameAttribute() : ParameterAttribute +GetDefaultSortAttribute() : ParameterAttribute +GetParameterSortAttribute() : ParameterAttribute +GetParameterUnitsAttribute() : ParameterAttribute +GetInterpolationMethodsAttribute() : ParameterAttribute +GetSourceCategoryAttribute() : ParameterAttribute +GetTimeCategoryAttribute() : ParameterAttribute +GetMeasureCategoryAttribute() : ParameterAttribute +GetFrequencyAttribute() : ParameterAttribute +GetAggregationLevelsAttribute() : ParameterAttribute +SchemaGetDisplayableAttributes() : ParameterAttribute +SchemaGetSmallDisplayableItemDetails() : ParameterAttribute +SchemaGetSortableAttributes() : ParameterAttribute +SchemaGetSearchableAttributes() : ParameterAttribute +GetDistinctValues(in elementName : String, in criteria : String, in sort : Boolean) : String +SearchForParameters(in criteriaExpression : String, in sortItem : ParameterAttribute) : String +GetAllValues(in item : ParameterAttribute) : String +GetValue(in parameterId : String, in item : ParameterAttribute) : Object +GetAggregationLevels(in parameterId : String) : AggregationLevel +GetInterpolationMethods(in parameterId : String) : InterpolationMethod +GetParameterComponentIds(in parameterId : String) : String +GetParentParameterId(in componentParameterId : String) : String +IsComplexParameter(in parameterId : String) : Boolean
Figure 9.19: ParametersMetadata class
The constructor for this class receives a XML filename containing the metadata and a XSD filename containing the structure (as explained in 8.2 – Metadata XSD Structure). After receiving these parameters the constructor will parse the XSD Structure, retrieving all the metadata elements available (xs:element definitions), and their predicted application behaviour. During this parsing the - 147 -
Common Components available elements are distributed trough several sets: Displayable Elements, Searchable Elements and Sortable Elements, according to their annotations presented earlier. This functionality allows that the changing of metadata elements. It’s easy to add a new metadata element: It’s only necessary to add a new element in the schema, and annotate it. The application will be able to deal and work with it, without changing the code. The class contains several functions used by the application components (both Data Catalogue and Report Designer) to access to metadata, and element behaviour
information.
E.g.:
the
function
SchemaGetDisplayableAttributes
returns all the Displayable Elements found in the schema. For obtaining metadata values out of this file, the class uses XPath queries over the XML file provided in the constructor.
- 148 -
Metadata Access Library
10 Conclusion 10.1 Evaluation of the Developed work _______________________________ 150 10.2 Future work ________________________________________________ 150 10.2.1
Data Catalogue ______________________________________ 150
10.2.2
Report Designer______________________________________ 150
10.2.3
Report Browser ______________________________________ 151
10.2.4
DIM integration ______________________________________ 151
10.2.5
Metadata ___________________________________________ 151
10.3 Internship Evaluation _________________________________________ 151
- 149 -
Conclusion
10.1
EVALUATION OF THE DEVELOPED WORK
The project itself is a set of user applications, that need to access to an infrastructure. One of the most important parts of this infrastructure is the Data Warehouse: it will contain the data for all the available parameters, and the ETL process will feed it with data automatically. As both the ETL and DW physical design are not yet complete, there was the need to develop and intermediate solution: creating a sample database similar to the final design, and load it with some real data. This was outside the scope of the project, but it was necessary. During this project, the research and development have been two important components. Researching and thinking together for finding the best solutions regarding the future, was a good practice. This way the final product is opened for future updates, as it will happen till the end of the SEIS project.
10.2
FUTURE WORK
Most of the work for the project is developed between 50% and 75%). Till the end of the SEIS project (December 2004) the following work must be done:
10.2.1 DATA CATALOGUE The data catalogue component is almost finished. There’s the need to integrate with new parameters. In terms of visualization features, it will also include events viewing capabilities (using Events Charts).
10.2.2 REPORT DESIGNER Enhance previewing, and tune current elements for more needs. Allow free page layout body editing, enhance the header and footer areas (including special fields, like page numbers, or user names). Add the possibility of inserting annotations inside charts based on the data values. Develop Time Window Helper component for picking up time windows.
10.2.2.1 TIME WINDOW HELPER COMPONENT This component will help the user to choose a time window for use with the report charts. The time window can be defined by any of the event based resources in the SEIS system: S/W events, S/C events, Single Event Upsets (SEUs) or MT Alarms. This helper will be done using a control developed by Joaquim Neto (EventsGrid).
- 150 -
Internship Evaluation Time Window helper Current Time Window From:
101.2004 00:30:40
(Start time of SEU #20)
To:
102.2004 14:15:20
(Confirmed time of Alarm #53)
Event and Alarm Browser S/C Events SEU S/C Alarms S/W Events CME Solar Flare ...
100.2004 101.2004 102.2004 103.2004 104.2004 #52
#53
Details on the selected item Trigger time:
102.2004 00:00:00
Confirmed time: 102.2004 14:15:20 Drag the boxes to the current Time window group to set times Cancel
Accept
Figure 10.1: Time Window Helper mock-up form
10.2.3 REPORT BROWSER Fully develop report browser, and possibly allow it to be a standalone component. Include features for printing reports, outputting to an image file, and data exporting. Develop both the SEU predefined report and the World Map Report.
10.2.4 DIM
INTEGRATION
Integrate the application with the physical implementation of the DIM. Add support classes for MDX querying and data access.
10.2.5 METADATA The metadata structure presented in Metadata will need to be extended and enhanced for allowing more metadata in it. Metadata for events, orbits and spacecrafts must be present in the application. This new structure will be based on the concepts added to the Metadata Repository, and the metadata will be obtained by the application via the same repository. The application will continue to use the XML Metadata file the same way it uses now, but that file will be refreshed from the central repository. For obtaining metadata from the repository, several views and transforms will be created.
10.3
INTERNSHIP EVALUATION
Working in this project has been very interesting. I’ve been in this project from October 2003 till now, and the integration with the team has been excellent, as well as the working environment. Everyone in this project is fully motivated and we know that we can count on each other’s. The
5-year
course
in
Computer
Science
gave
the
knowledge
for
the
programming area, and the basis for thinking when researching. - 151 -
Bibliography
11 Bibliography [1] "Part
Pereira, A. Caramagno, M. Pantoquilho, N. Viana, and S. Santos (2003), I:
Proposal,"
Technical
UNINOVA-DEIMOS,
Technical
Proposal
SEIS_PROP_001_1-0 (17519/03/D/HK), 25-07-2003 (www.uninova.pt/ca3). [2]
Donati, F. D. Marco, N. Viana, M. Pantoquilho, A. Baumgartner, and J.
Moura-Pires (2004), "Space Weather and Mission Control: A Roadmap to an Operational Multi-Mission Decision Support System".Proceedings of SpaceOps 2004 - 8th International Conference on Space Operations, Montreal, Canada. [3]
M. Pantoquilho and N. Viana (2003), Parameters Requirement Telecon
Meeting Minute.doc [4]
M. Pantoquilho and N. Viana (2003), "Kick Off Meeting Minute,"
UNINOVA, Meeting Minute SEIS_MTR_005_1-0, 2003-08-21 [5]
M. Pantoquilho and N. Viana (2003), "Other Possible Data Providers,"
UNINOVA SEIS_xxx_109_1-0, [6]
M. Pantoquilho and N. Viana (2003), "Parameter Listing," UNINOVA,
Single Report SEIS_xxx_103_0-1, 02.09.2003 [7]
M. Pantoquilho and N. Viana (2003), "Reporting & Analysis Tool GUI
Description," UNINOVA SEIS_xxx_104_0-1, 02.09.2003 [8]
J. Moura-Pires, M. Pantoquilho, and N. Viana (2004), "Real-Time
Decision Support System for Space Missions Control".Proceedings of IKE'04 - The 2004 International Conference on Information and Knowledge Engineering, Las Vegas, USA. [9]
J.
Moura-Pires,
M.
Pantoquilho,
and
N.
Viana
(2004),
"Space
Environment Information System for Mission Control Purposes: Real-Time Monitoring and Inference of Space Craft Status". Proceedings of 2004 IEEE Multiconference on CCA/ISIC/CACSD, Taipei, Taiwan. M. Pantoquilho, J. Neto, N. Viana, R. Ribeiro, and J. Moura-Pires (2004), "Online and Offline Monitoring and Diagnosis of Spacecraft and Space Weather Status".Proceedings
of
EUROFUSE
Engineering, Warszawa, Poland.
- 152 -
Workshop
on
Data
and
Knowledge
Internship Evaluation [10]
R. Ferreira : BDDW Final Report, Faculdade de Ciências e Tecnologia -
Universidade Nova de Lisboa, Portugal, Janeiro 2004.
- 153 -
Annexes
12 Annexes 12.1 Annex 1 – Report Definition Language Schema_____________________ 155 12.2 Annex 2 – Parameter Metadata Schema __________________________ 165 12.3 Annex 3 – Data Catalogue Tree Definition XML _____________________ 169 12.3.1
Schema for the Tree Definition XML file ___________________ 169
12.3.2
Tree Definition XML File________________________________ 170
12.4 Annex 4 – RDL Classes Diagram ________________________________ 171
- 154 -
12.4.1
Report Class and related classes _________________________ 171
12.4.2
TextboxType Class and related Classes____________________ 172
12.4.3
SingleLinearChartType Class and related classes ____________ 173
12.4.4
SingleGradientChartType Class and related classes __________ 174
12.4.5
SingleXYCorrelationChartType class and related classes ______ 175
12.4.6
MultiChartType class and related classes __________________ 176
Annex 1 – Report Definition Language Schema
12.1
ANNEX 1 – REPORT DEFINITION LANGUAGE SCHEMA
The RDL version used for this Report Description of the Report Report author name The date when this report was created The date when this report was modified The data when the data in the report was last refreshed Whether this report contains saved data or not Report Header details Report Footer details Page Header details Page Footer details Report Body details
- 155 -
Annexes Height of the header / footer section Items in the header / footer Items in the body Left coordinate in pixels Right coordinate in pixels Width in pixels Height in pixels Order number Height in pixels Alpha component
- 156 -
Annex 1 – Report Definition Language Schema Red component Green component Blue component Value Color details Whether this gradient has fixed or relative limits Fixed min limit Fixed max limit Components of the gradient Font name Font size
- 157 -
Annexes Whether bold is active or not Whether italic is active or not Whether underline is active or_u110 ?ot Whether strikeout is active or_u110 ?ot Text alignment Name of the Report Item Layout details TopAlignment details Globalid of the parameter for the series Name of the parameter
- 158 -
Annex 1 – Report Definition Language Schema Units for the parameter Description for the series Aggregation Level to use Parameter Data Interpolation Method to use to_u99 ?onnect points of the series Color to use in the line and/or points Series list Linear Series list If this axis mirrors other
- 159 -
Annexes Whether to use log Scale in this axis or not Series list for this axis Lower limit of the time window Upper limit of the time window Time window details Time format to use
- 160 -
Annex 1 – Report Definition Language Schema Title for the chart Title Font details The position of the title in the chart Whether to show legend or not The position of the legend in the chart Whether to mark points or not Whether to use X Axis guide lines or not Axis Font details Description Font details Axis Description orientation Time window details Series X details Series Y details
- 161 -
Annexes Whether if Series X is dominant Whether if Series Y is dominant Whether the X axis uses a log scale or not Whether the X axis uses a log scale or not Inteporlation Method to use in_u116 ?he resulting series Color for the resulting series Resulting series description XYCorrelationChart details Y Axis details Z Axis details
- 162 -
Annex 1 – Report Definition Language Schema Series details Gradient details X Axis details The chart frequency to display_u116 ?he X Axis Chart list Linear chart details Gradient chart details
- 163 -
Annexes Textbox details SingleLinearChart details SingleXYCorrelationChart details SingleGradientChart details MultiChart details
- 164 -
Annex 2 – Parameter Metadata Schema
12.2
ANNEX 2 – PARAMETER METADATA SCHEMA
Tree level: used to diferenciate the measure category of the parameter Tree level: used to diferenciate the scientific group of the parameter SourceCategory S/W or S/C TimeCategory Timeseries / Events / Orbits MeasureCategory Real / Forecasted Frequency Sampling frequency in minutes
- 165 -
Annexes Description for the sampling frequency Data source name: GOES/NOOA Data source details: satellite_u111 ?r instrument General description for the parameter Relevancy information for the parameter ParameterName Parameter name ParameterUnits
- 166 -
Annex 2 – Parameter Metadata Schema Parameter units Predictive model name (for predicted parameters) Predictive model description (for predicted parameters) AggregationLevels Possible aggregation levels to_u116 ?he parameter if this parameter can be visualized in a different aggregation level InterpolationMethods Possible interpolation methods_u116 ?o use in charts keywords for seach purposes
- 167 -
Annexes
- 168 -
Annex 3 – Data Catalogue Tree Definition XML
12.3
ANNEX 3 – DATA CATALOGUE TREE DEFINITION XML
12.3.1 SCHEMA FOR THE TREE DEFINITION XML
FILE
