svg dynamic tiling and progressive loading

Viewer
Transcript

SVG DYNAMIC TILING AND PROGRESSIVE LOADING Monsak Socharoentum Research and Development Computing Devision (RDC) National Electronic and Computer Technology Center (NECTEC) National Science and Technology Development Agency (NSTDA) Ministry of Science and Technology Email: [email protected], Tel: (66) 2564-6900 Ext. 2273

ABSTRACT SVG vector map is a well known graphic file format for presenting interactive web-maps and used in several Internet map websites for years. On the other hand, it is also known that SVG (a text-based XML) result in a lot overhead information, computationally expensive and memory consumptive. Consequently, it is almost impossible to dump all spatial data in a single SVG file and send it to client machine at one time. As the raised arguments, SVG appears that it may not suitable for a complex or large scale vector map. However, as literature reviewing, there are several techniques proposed to lighten this limitation.

This paper propose an approach for tracking whether data is already existing in client's memory and avoiding redundant data loading. To be precise, the main purpose of this paper, under the environment of thin client's capabilities, is to find the way to identify the map features (point, line or polygon) that overlap with current view box extent and do not exist in the client's memory (SVG DOM) before, then deliver them to client. A feature based dynamic tiling technique for SVG is developed to reduce some SVG's weak points. A proof of concept prototype is conducted to test the approach. Such prototype is composed of two computer programs, one is embedded on client web browser while another operate on server. The programs work together using the designed protocol for requesting and responding processes. For the software tools used in this work, PostgreSQL 1 with PostGIS 2 spatial extension, Apache Web Server and PHP 3 are the three main components ruling the server machine. And also, embedded Java Script and Internet Explorer with Adobe's SVG Viewer cooperate to work as thin client. KEYWORDS feature based dynamic tiling, progressive loading, multiple asynchronous request

1. INTRODUCTION SVG vector map is a well known technique for interactive web-maps displaying and used in many websites for years. On the other hand, it is also known that SVG (a text-based XML) result in a lot overhead information, computationally expensive and memory consumptive. Consequently, it is almost impossible to dump all spatial data in a single SVG file and send it to client machine at one time. As the raised arguments, SVG appears that it may not suitable for a complex or large scale vector map. However, as literature reviewing, there are several techniques proposed to lighten this limitation. A Design and Implementation of Spatial Database Based on XML-SVG [Jianting Wen and Yan Li, 2005]. Sending map layer just on demand is proposed in this work. First, all map layers (in traditional file format) are converted into SVG format and import to relational database. Second, when client make a request for a map layer, web server retrieve all related SVG elements from database and blend them into one package before sending the package to client as a single map layer. In summary, all map layers are not sent to client at one time but only the layer specified in client request. Use of vector and raster tiles for middle-size Scalable Vector Graphics' mapping applications [Benjamin Campin, 2005]. Due to the decreasing of SVG viewer performance when the size of a vector or a raster layer

is around 1.5 MB. there is a need to tile/cut the whole geographic data into several block of SVG files. Client side script inspect the changing of map extent (view box) on user's screen and calculate the suitable SVG file name for requesting the additional pack of SVG file from server machine to be appended into the original document. Adaptive Zooming in Web Cartography [Alesandro Cecconi and Martin Galanda, 2002]. Feature class information displayed on maps should be suitable with map scale showing on the screen. Maps are simplified into several map scales (large map scale is used to generate the smaller map scale). Whereas some feature class decrease in detail through on-the-fly map generalization. Developer has to justify that what method should be used with what feature class. In summary, the output feature class (SVG file size) loaded to client will be smaller. Delivering Interactive Topographic Web-maps using Open-Source Database Technology [Andreas Neumann, 2003]. PostgreSQL, PostGIS and PHP were used as the workhorses of web-maps system, PostGIS get map query parameters from PHP extract the current map-extent from a larger geographic database, and then send map-data appropriating for the current scale and view box extent back to client machine. The approach is somehow using a "brute force" mechanism, meaning that it is not track whether data is already existing and can be re-used. It always replace the whole content of a dynamic data layer, even if the user always pans a little bit. This issue might be addressed in future versions [Andreas Neumann, 2003]. This paper propose an approach for tracking whether data is already existing in client's memory (SVG DOM) and avoiding redundant data loading as raised in Andreas Neumann, 2003. To be precise, the main purpose of this paper, under the environment of thin client's capabilities, is to find the way to identify the map features that both overlap with current view box extent and do not exist in the client's memory before, then deliver them to client. A feature based dynamic tiling technique for SVG is developed to reduce some SVG's weak points. A proof of concept prototype is conducted for testing the approach. PostgreSQL with PostGIS spatial extension, Apache Web Server and PHP are the three main components ruling the server machine. Embedded Java Script and Internet Explorer with Adobe SVG Viewer cooperate to work as thin client.

2. PROTOTYPE ARCHITECTURE As shown in Figure 1, Prototype architecture is composed of three main components. First, Client web browser embedded with JavaScript and original SVG file work as front end application to interact with user. Second, Apache Web Server and PHP work as middle ware to wait for map request from client, generate SQL spatial query 4 for retrieving map feature, and rearrange data getting from database before sending back to client. Third, database which contain spatial data in the form of relational table, make internal query command for extracting the particular map features.

Figure 1: Architecture of prototype system Client and Web server communicate by exchanging two data items. Current view box extent and feature Id of existing map feature are sent to Web server to be used as the restriction for building SQL spatial query.

Responding text is composed of map feature Id and stream of their coordinates those should be loaded into original SVG document. The original document consist of only a few line of SVG code as shown in Figure 2. The document is used as the main structure supporting for adding more map features into it. Figure 2: The original SVG document Another two data items are transferred between Web server and Database. SQL spatial query is generated by PHP code in Web server and sent to PostGIS for querying. Outputs from PostGIS are map feature Id and stream of their coordinates.

3. DATABASE ARRANGEMENT Three administration boundary maps of Thailand composed of 76 provinces, 881 districts, and 7613 subdistricts are chosen to test the prototype as detail shown in Table 1. All map layers were produced at scale 1:50,000 by a government organization. Table 1: Some properties of administration boundary maps of Thailand File size (MB.) Layer name No. of polygons No. of points shp dbf Province

76

918,060

14.4

0.100

Amphoe (district)

881

2,342,806

36.6

0.195

7,613

5,529,257

88.8

0.596

Tambon (subdistrict)

Database arranging processes are set to support map feature retrieval, as shown in Figure 3. All three administration maps (shape files) are converted to data file (encoded in SQL syntax) by “shp2pgsql”, a program coming with PostGIS, and then import into Database by “psql”, a program coming with PostgreSQL.

Figure 3: Diagram of database arranging process The examples of using “shp2pgsql.exe” and “psql.exe” can be shown in Figure 4. Command syntax is shown first and followed by an example of command line. Note: Information shown in Figure 4 is simplified from a screen capture of MS Windows's command prompt. USAGE: shp2pgsql.exe [] [.] -I

Create a GiST index on the geometry column.

Example: C:\Program Files\PostgreSQL\8.1\bin>shp2pgsql.exe -I e:\Work...\province public.province --> province.sql

USAGE:

psql [OPTIONS]... [DBNAME [USERNAME]]

-f FILENAME

execute commands from file, then exit

-d DBNAME

specify database name to connect to

-U NAME

database user name

Example: C:\Program Files\PostgreSQL\8.1\bin>psql -U postges -d thailand -f province.sql Figure 4: Example of using command line “shp2pgsql.exe” and “psql.exe” After the process, three new tables (province, amphoe and tambon) are created in database. All attributes value which is previously contained in dbf file are loaded into the second to the fifth column in “province” table, as shown in Figure 5. Last column in Figure 5, “the_geom”, is a column for containing map feature's coordinates shown in the form of hexadecimal numbers. The first column, “gid”, is automatically generated by “shp2pgsql.exe”. It is an auto increment field and used as the primary key in the table. After running “province.sql”, a GIST 5 spatial index is also automatically created resulting from “-I” option used in “shp2pgsql.exe” command line (see the command line in Figure 4). When a spatial query occur, this index will be very useful for spatial query speeding up.

Figure 5: Province table in PostGIS after running SQL command file resulting from “shp2pgsql.exe”

4. CLIENT SIDE JAVA SCRIPT Besides the preinstalled Adobe's SVG Viewer, the prototype need some additional JavaScript code to control dynamic map loading and presenting on the screen. Three main tasks of the codes are inspecting current view box extent, sending multiple simultaneous requests to server, and appending new map features loaded from server into the original SVG document. The core component supervise the second task and will be explained in detail in this paper. Because of the limitation of thin client, it is very hard to verify that whether or not a map feature is in the current view box extent, and therefore server side processing is chosen instead. Three sets of parameters, list of existing map features Id, current view box extent, and map layer name required, are packed in the request, then sent to process at server side. Parameters in the request are shown in Figure 6. The current view box extent parameters named “ulX”, “ulY”, “lrX” and “lrY” means upper left X, upper left Y, lower left X and lower left Y respectively. The required map layer name is specified in “layername” value. Lastly, “existing” is a parameter for declaring the Id of existing map features broken up by comma. http://127.0.0.1/Tiling/getmap.php? ulX=1225000&ulY=620000&lrX=1405000&lrY=291400&layername=province&exis ting=38,39,40,46,50,41,42,43,44,45,51,52 Figure 6: An example of request pattern To achieve the better loading speed, the current view box extent is split into several smaller extents, then formed into several requests. These requests, multiple requests, are sent simultaneously to server by using AJAX 6 technique. Note: More information about AJAX code is mentioned in acknowledgment section. There are two nested loop of JavaScript code for generating multiple small extent requests shown in Figure 7. The inner loop generate sequence of sub-extents in the horizontal direction. Meanwhile, the sequence of vertical sub-extents is managed by the outer loop. The “xmlreqGET(urlCall)” is a startup function for activating XMLHTTPRequest 7 function set (see the acknowledgment section for more detail).

function loaddoc(ulX,ulY,width,height,layer){ temp=ulX tempY=-1*ulY for (m=1;m<=6 ;m++ ) { ulY=tempY-(m-1)*height/6; lrY=ulY-height/6; for (n=1; n<=6;n++ ) { ulX=temp+(n-1)*width/6 lrX=ulX+width/6 urlCall="http://127.0.0.1/Tiling/getmap_bkk.php?ulX="+ulX+ '&ulY='+ulY+'&lrX='+lrX+'&lrY='+lrY+'&layername='+layer; urlCall=urlCall+'&existing='+map_layer_features[layer]; xmlreqGET(urlCall) } } } Figure 7: Two nested loops of JavaScript code for splitting current view box extent into several subextents and generating multiple request Variable “width” and “height” is the width and height of current view box extent. These values, width and height, are divided by six to divided the whole extent into 6x6 parts (six in the horizontal and six in the vertical). The other variables are “ulX”, “ulY” and “layer” which mean the upper left X value, the upper left Y value of view box extent and layer name required respectively. Within the inner loop, a URL request is formed by setting value of “urlCall”. An array variable “map_layer_features[layer]” contain list of exist map feature Id for each map layer.

5. SERVER SIDE PHP SCRIPT Server side script consist of just several lines of PHP code, as shown in Figure 8. In general, this code play three main roles, opening connection to PostgreSQL database, generating SQL spatial query, and replying client request with a package of map features information.
$SQLCmd="SELECT DISTINCT gid,AsSVG(Simplify(the_geom,20),1,1) FROM ". $_GET["layername"]." WHERE gid NOT IN (".$_GET["existing"].") AND the_geom && 'BOX3D(".$_GET["ulX"]." ".$_GET["ulY"].", ". $_GET["lrX"]." ".$_GET["lrY"].")'::box3d;"; }else { $SQLCmd="SELECT DISTINCT gid,AsSVG(Simplify(the_geom,20),1,1) FROM ". $_GET["layername"]." WHERE the_geom && 'BOX3D(".$_GET["ulX"]." ". $_GET["ulY"].", ". $_GET["lrX"]." ".$_GET["lrY"].")'::box3d;"; } $result = pg_query($dbconn2, $SQLCmd); $numrow=pg_numrows($result)-1; if ($numrow>=0){$svgData="";} for ($i=0;$i<=$numrow;$i++){ $path=pg_fetch_row($result,$i); $svgData=$svgData.$path[0].'%'.$path[1].'#'; } echo $svgData; ?> Figure 8: PHP script for processing map request coming from client The SQL spatial query can be generated in two ways depending on the value of “$_GET["existing"]”. If “$_GET["existing"]” is null, meaning that there are no map features in the client's memory, the SQL “where” clause is only added with map extent. If there is at least one map feature Id in “$_GET["existing"]”, meaning that there are some map features already exist in client's memory, both the list of Id and map extent will be added to the clause. Only map features which conform to the restriction will be retrieved from database, and then encoded in the pattern being similar to the text shown in Figure 9. In the text, “%” is used to separate feature Id from feature coordinates, and “#” is used to separate one feature from the others. 1%M682776.22 -1538987.21 l 225.28 25.76 34.63 3.96 309.83 33.92# 2%M683345.95 -1538923.58 l 22.42 2.45# 3%M685701.63 -1534483.15 l -43 -1.97 Figure 9: Several map features Id and coordinates encoded as text stream Note: Feature's coordinates are encoded in the form of “d” attribute in SVG path shortcut element

6. THE RESULTS The prototype can make feature based tiling and progressive loading according to the display rules predefined by the developer. In Figure 10, Province is load progressively from server and the complete map is shown on the right. When the screen zoom in on some region of map, Amphoe (district) map features are loaded, as the shaded polygon shown on the upper left corner in Figure 11.

Figure 10: Progressive loading of province administration boundary of Thailand

Figure 11: Loading some additional polygons of district map, shaded polygon on the upper left corner, after a zooming event.

7. CONCLUSION This approach can identify the map features that overlap with current view box extent and that do not exist in the client's memory before, then only deliver the necessary map features to client. This also extend the ability of SVG to display large scale or high level of detail vector map. The prototype still has some limitations as mentioned below. 1. Deleting unnecessary map features that far from current view box extent to reduce memory consumption is not supported. 2. View box extent splitting method is straight forward. In other words, the whole view box extent is split into 36 smaller box extents starting from the upper left corner to the right and ending at lower right corner. Because, normally, the most interested area is around center, it may be better to start sending the split extents around center to server first. In practice, this kind of technique required a high performance server to manage many tens of requests coming from only a single client within few seconds. Moreover, when many clients simultaneously send their requests to server, performance tuning will be needed.

ACKNOWLEDGEMENT Drakware: A function set of multiple XMLHTTPRequest source code used in the prototype is modified from http://www.drakware.com/articles/multijax.php

GLOSSARY [1] PostgreSQL: An open source relational database system. PostgreSQL, originally called Postgres, was created at UCB by a computer science professor named Michael Stonebraker. More information at: http://www.postgresql.org/ [2] PostGIS: PostGIS adds support for geographic objects to the PostgreSQL object-relational database. In effect, PostGIS "spatially enables" the PostgreSQL server, allowing it to be used as a backend spatial database for geographic information systems (GIS). More information at: http://www.postgis.org/ [3] PHP: An HTML-embedded scripting language. Much of its syntax is borrowed from C, Java and Perl with a couple of unique PHP-specific features thrown in. The goal of the language is to allow web developers to write dynamically generated pages quickly. More information at: http://www.php.net [4] SQL spatial query: Structured Query Language used in PostGIS for retrieving spatial data. More information at: http://postgis.refractions.net/docs/ch04.html#id2918389 [5] GIST: GiST stands for "Generalized Search Tree" and is a generic form of indexing. In addition to GIS indexing, GiST is used to speed up searches on all kinds of irregular data structures (integer arrays, spectral data, etc) which are not amenable to normal B-Tree indexing. More information at: http://www.sai.msu.su/~megera/postgres/gist/ http://postgis.refractions.net/docs/ch04.html#id2918926 [6] AJAX: Asynchronous JavaScript And XML, or its acronym, Ajax (Pronounced A-jacks), is a Web development technique for creating interactive web applications. The intent is to make web pages feel more responsive by exchanging small amounts of data with the server behind the scenes, so that the entire Web page does not have to be reloaded each time the user makes a change. This is meant to increase the Web page's interactivity, speed, and usability. More information at: http://en.wikipedia.org/wiki/AJAX [7] XMLHTTPRequest: An interface exposed by a scripting engine that allows scripts to perform HTTP client functionality, such as submitting form data or loading data from a remove Web site. More information at: http://www.w3.org/TR/XMLHttpRequest/

REFERENCES Alesandro Cecconi and Martin Galanda, 2002, Adaptive Zooming in Web Cartography. SVG Open / Carto.net Developers Conference. Zurich, Switzerlan. Andreas Neumann, 2003, Delivering Interactive Topographic Web-maps using Open-Source Database Technology. SVG Open Conference 2003.Vancouver, Canada. Andreas Neumann, 2004, Using SVG for Online Digitizing and Editing of Geographic Data. SVG Open Conference 2004.Tokyo, Japan. Benjamin Campin, 2005, Use of vector and raster tiles for middle-size Scalable Vector Graphics' mapping applications. SVG Open Conference 2005. Enschede, Netherlands.

Jianting Wen and Yan Li, 2005, A Design and Implementation of Spatial Database Based on XML-SVG. SVG Open Conference 2005. Enschede, Netherlands. Anne van Kesteren (Opera Software ASA) and Dean Jackson (W3C), 2006, The XMLHttpRequest Object, W3C Working Draft 05 April 2006. Available at http://www.w3.org/TR/XMLHttpRequest/