Transactions in GIS, 2006, 10(3): 377– 394

Research Article

Image-Based Modeling of Urban Buildings Using Aerial Photographs and Digital Maps Byounghyun Yoo

Soonhung Han

Department of Mechanical Engineering Korea Advanced Institute of Science and Technology

Department of Mechanical Engineering Korea Advanced Institute of Science and Technology Keywords: Aerial Photograph, Displacement Mapping, Image-Based Modeling, Urban Modeling, Virtual Environment

Abstract A VR (virtual reality) simulator which is used for helicopter simulations requires a virtual environment of real world urban areas. However, real urban environments are continuously changing. It is necessary to develop a modeling method that makes direct use of GIS (geographical information system) data which is updated periodically. A flight simulation needs to visualize not only buildings in the near distance but also a large number of buildings in the far distance. We propose a method for modeling urban environments from aerial images and digital maps with relatively little manual work. An image-based method is applied to the urban modeling that considers the characteristics of Korean cities. Buildings in the distance can be presented without creating a large number of polygons. The proposed method consists of a pre-processing stage that prepares the model from the GIS data and a modeling stage that creates the virtual urban environment. The virtual urban environment utilizes the height map of buildings.

1 Introduction The virtual environment is an essential part of any virtual reality system such as flight simulations. Previous studies related to construction of virtual environments for flight simulation have concentrated on automation of geometric modeling, data acquisition, data enhancement, and the rendering method.

Address for correspondence: Byounghyun Yoo, ME3080, Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, Daejeon 305-701, Korea. E-mail: [email protected] © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd

378

B Yoo and S Han

The terrain models and virtual environments are becoming diverse according to the intended uses of the simulators. A helicopter flight simulation, for example, needs efficient visualization of a wide range of terrain and artifact models because a helicopter has a wide field of view and high viewing position. The polygon model approach usually neglects the buildings in the far distance to reduce the overhead of visualization, and concentrates on the nearby terrain and buildings, as shown in Figures 1a and b. Buildings in the far distance tend not to be modeled to reduce the cost and time for construction of the virtual environment. Even when they are created, they are often deleted during real-time visualization to maintain a uniform number of polygons for the performance of rendering. For this reason, the virtual environment for flight simulation is distorted, as shown in Figure 1b. Figure 1a shows a typical virtual environment where the polygon models of buildings near the (central) areas of interest are used while only flat texture maps of building images are cast over the terrain model of remote areas. Therefore, the apartment block in the white circle in Figure 1a appears collapsed, as shown in Figure 1b. Creating polygon models of all the buildings in the remote area is impossible. Existing urban models for helicopter simulation only represent several key buildings as polygons and employ texture maps for remote areas. While the virtual environment should be periodically updated to guarantee coincidence with the real environment, the existing approaches of constructing virtual environments with polygon models cannot afford to do this. Geospatial databases in geographic information systems (GIS) are constructed as digital maps. A digital map of a nation can be constructed by means of aerial photographic surveying. These maps may include planar outlines of buildings (Lee et al. 2004). In the national geographic information system (NGIS) project promoted by the Ministry of Construction and Transportation in Korea, it is required that the period of adjustment and renewal of the digital map is less than five years and the monitoring system covers the whole nation (90,000 km2). In order to continuously reflect changes in urban environments, it is better to utilize the geospatial information of the NGIS project. We propose a method of modeling the urban environment for a helicopter simulation that is tailored to the Korean geospatial characteristics. The following two aspects are considered. First, for the efficient utilization of periodically updated geospatial information, the virtual environment should be modeled from the geospatial information with minimal overhead. Second, buildings in the far distance should be represented efficiently for the purpose of the helicopter simulation. The remainder of this paper is structured as follows. Section 2 analyzes related works. Section 3 describes the proposed modeling method, which is tailored to the characteristics of Korean geography. Section 4 analyzes the strengths and weaknesses of the proposed method through modeling experiments.

2 Related Work Construction of virtual cities or cyber cities is similar to modeling a virtual environment for flight simulation. There are several cases of cyber cities. Representative works for virtual cities are the Arena 2000 project in Helsinki, the 3D city database of UDS (Urban Data Solutions; see http://www.u-data.com for additional details) Corporation in the USA, the 3D city model of Xiamen in China, and the 3D GIS data of Mitsubishi © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Image-Based Modeling of Urban Buildings

379

Figure 1 Virtual urban environment for helicopter simulation: (a) polygon-based model of virtual environment for helicopter simulation; and (b) mixture of polygon models with flat texture map. This figure appears in colour in the electronic version of this article and in the plate section at the back of the printed journal

Corporation in Japan. Cyber cities that include terrain, buildings, and road facilities are under construction by city governments in Korea (Choi et al. 2003, Kang and Park 2003). The Virtual Terrain Project (http://www.vterrain.org) is another example of virtual city construction. However, most of these projects are not being widely used or were © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

380

B Yoo and S Han

constructed for one-time use, and most of these cases build 3D polygon models in the form of VRML (Web3D Consortium 2003). Core technologies for virtual environment construction include gathering and processing of 3D spatial information and real time 3D display. There is much ongoing research on city modeling through the gathering and processing of 3D spatial information. Most studies generate 3D shapes of buildings from aerial images and digital maps (Nevatia and Price 2002, Suveg and Vosselman 2004). There have also been studies on the automatic generation of building shapes from range images of lasers such as LiDAR (Fujii and Arikawa 2002). Lee et al. (2002) studied how to generate and map outer shapes and textures in urban buildings. Moons et al. (1998) extracted the building roof shape from an aerial image of high resolution. They tried to extract 3D data of a building such as the vertices, edges, and faces. These studies all use polygonal modeling of buildings. A virtual urban environment consists of the buildings on the terrain. For efficient visualization of the terrain, LoD (Level-of-Detail) techniques such as ROAM (Duchaineau et al. 1997), QuadTree (Roettger et al. 1998), and GeoMipmapping (De Boer 2000) have been developed. However, research on modeling and presentation of urban buildings is insufficient. Previous research on the modeling and presentation of terrain and buildings has focused on polygon models, which are the traditional method in 3D computer graphics. The polygon model requires significant effort to generate the desired outputs and has overhead on visualization in proportion to the number of polygons. The imagebased modeling and rendering has been investigated as an alternative approach. Debevec (1996) proposed a method that generates and visualizes a building model from photographs. McMillan (1997) made progress with image-based modeling methods, and Oliveira (2000) improved the image warping technique that is widely used in imagebased methods. Table 1 provides a comparison between the proposed image-based modeling method for urban buildings and previous studies.

3 Modeling of a Virtual Urban Environment Using Aerial Photographs and Digital Maps 3.1 Virtual Environment for a Helicopter Simulation The virtual environment for a helicopter simulation should satisfy the following two conditions. First, buildings in the far distance should be represented with reasonable reality. In land vehicle simulations that involve movement along the ground, the viewpoint of the rider is near the ground, as visualized by camera C1 in Figure 2, and building façades are mainly visualized. Most of buildings far from the viewpoint C1 cannot be seen as they are hidden by nearby buildings, or they can be simplified because the viewing angle for the buildings is narrow. However, a rider on the helicopter simulation can see many distant buildings, as visualized by camera C2, because the viewpoint is located above the buildings and the helicopter can move freely from low to high altitude. This situation is also different from the case of an airplane. In particular, the line of sight in a flight simulation typically points to the horizon, and hence the rider tends to look at the buildings on area Y in the middle range or area Z in the far distance rather than area X immediately under the viewing position. Second, the real urban areas must be modeled for a realistic simulation. The simulation’s goal is to reproduce real phenomena or to conduct training safely and economically. The urban areas are changing © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Building extraction from GIS data

Tessellation with LoD

Image-based modeling

References

Manual modeling with polygonal model

Fujii and Arikawa (2002), Suveg and Vosselman (2004), and Nevatia and Price (2002)

ROAM (Duchaineau et al. 1997), QuadTree (Roettger et al. 1998), and GeoMipmapping (De Boer 2000)

Present study

Input data

Heterogeneous information

Digital map, aerial photograph, LiDAR

DEM, aerial photograph

Digital map, aerial photograph

Output

Polygonal model (VRML)

Polygonal model

Triangular irregular network

No polygons. Displacement map

Cost and time for modeling

X Large


Medium

 Small

 Small

Periodical update

X Difficult

 Available

 Available

 Available

Representation of near objects

 Suitable

 Available

X Difficult


Available, but not good quality

Detail of representation

 Good


Not bad

X Restricted


Not bad

Representation of far objects

X Difficult


Possible but limited

X Difficult

 Suitable

Overhead to visualize

X Increases with number of polygons

X Increase

 Effective

 Manageable

Applicability to flight simulation


Not bad


Not bad


Adjustable, only for terrain

 Suitable

381

Manual modeling

Image-Based Modeling of Urban Buildings

© 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Table 1 A comparison between existing approaches and the proposed method

382

B Yoo and S Han

Figure 2 Visibility depending on the viewpoint height

continuously as a result of construction of new buildings and maintenance of existing buildings. This requires easy and continuous updating of the virtual environment in order that the virtual environment continues to accurately reflect the real urban environment. The virtual environment of a real city can easily be generated if the proposed method is applied. The overhead for generation of the virtual environment is thereby minimized, and hence this approach can be applied whenever geospatial information is updated.

3.2 Characteristics of the Urban Environment in Korea There are many box-shaped buildings in the urban environment of Korea, which have a simple structure and roof shape. Because the number of box-shaped apartments in residential areas is increasing, it is possible to represent urban buildings with rather simple geospatial information. In the case of typical European buildings as shown in Figures 3a and b, there are many cases where buildings are attached to adjacent buildings and the structure of the roof is complicated because of supplementary structures on top of the roof (Cho 2004). Due to these characteristics, related studies of existing urban modeling have been carried out with a focus on polygon-based modeling. In the case of Korea, the interval between adjacent buildings is quite regular and proportional to the height of buildings owing to regulations related to the right to sunshine, and the roof structure is simple compared with that of European buildings. Apartment complexes occupy a high proportion among all residential buildings, and this ratio is increasing. Therefore, the buildings of Korea, having comparatively simple shape, can be represented by two simple items of information, the planar footprint and height of the building. By adding the image of the roof and the façade, medium distant buildings can be represented realistically. Meanwhile, as there are few conspicuous artifacts in most mountain areas as shown in Figure 3d, the existing geographic modeling method can be applied in these areas.

3.3 Geospatial Information Korean buildings can be reasonably expressed using the planar footprints of the buildings, the heights of the buildings, and texture images of the roofs and sides. The planar footprints (i.e. outlines) of buildings are extracted from digital maps and used to create the virtual buildings. Heights of buildings can be obtained by applying the process of analytical stereo plotting to aerial photographs. The National Geographic © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Image-Based Modeling of Urban Buildings

383

Figure 3 Comparison of urban environments: (a) Heidelberg in Germany (Cho 2004); (b) Prague in Czech Republic (Cho 2004); (c) Apartment complex at Dachi-dong in Seoul; and (d) Bukhan mountain in Seoul, Korea. This figure appears in colour in the electronic version of this article and in the plate section at the back of the printed journal

Information Institute of Korea makes digital maps using the analytical stereo plotting technique with orthoimages. This institute has and maintains the height values of most buildings. However, NGIS (National Geographic Information System) digital map version 1.0, which is distributed to the Korean public, does not contain this information. NGIS digital map version 2.0 will contain properties such as the number of floors and information pertaining to usage of each building. Although the heights of buildings can be calculated using this information, the NGIS digital map version 2.0 has not yet been published. Buildings often appear to lie horizontally in aerial photographs, according to the line of sight of the image. The aerial image with an orthoimage can be obtained through geometric correction and orthographic correction from several aerial photographs. Because this research uses aerial photographs that have been orthographically corrected, it is easy to map the digital map to the aerial image, and there is no interference between buildings and the ground.

3.4 Virtual Environment Modeling Using Geospatial Information Using characteristics of the Korean urban environment and geospatial information, a modeling technique for the virtual environment that is suitable for application to a © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

384

B Yoo and S Han

Figure 4 Modeling process for virtual environment based on geospatial information

helicopter simulator is proposed. A displacement mapping method, based on image-based modeling methods, is applied. The virtual environment can be generated with minimal processing of the digital map and aerial photograph without the burden of modeling numerous building shapes into the polygonal model. Displacement mapping, which was first introduced by Cook (1984), is a method for expressing silhouettes of the perturbing surfaces of a body in 3D graphics. There are two approaches, polygonal representation methods using triangular mesh generation (Doggett and Hirche 2000, Moule and McCool 2002) and methods employing imagebased representation (Schaufler and Priglinger 1999, Kautz and Seidel 2001). Polygonal representation methods, which create triangular meshes to make the shape of a perturbed surface, are not suitable for the purposes of this research, because they create too many polygons. In this research, the relief texture method (Oliveira 2000) has been modified and used. The relief texture method uses image warping, and basically divides the existing image warping process into two separate stages, pre-warping and texture mapping. It has the advantage that the existing texture maps can be used without modification. Figure 4 shows the flowchart of the proposed modeling approach for virtual environments based on geospatial information. The virtual environment modeling process is divided into a pre-processing process and a construction process. The pre-processing creates a height map of the buildings that is suitable for displacement mapping, and a texture image from orthographic aerial photographs. To create the building height map, planar outlines of buildings are extracted from the NGIS digital map, and the heights are calculated from the number of floors of each building. The virtual environment is constructed by pre-warp and texture mapping based on the relief texture. The relief texture is a kind of image-based approach which represents displacement of surface without additional polygonal expression. To apply the conventional texture mapping process, image warping is applied to the corresponding orthographic aerial photograph by using the height map, which is generated by the pre-processing © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Image-Based Modeling of Urban Buildings

385

Figure 5 Pre-warp and texture mapping of high-rise buildings. This figure appears in colour in the electronic version of this article and in the plate section at the back of the printed journal

step. The texture that is edited for representing the planar silhouette of a given building is generated by using an algorithm that is modified to process buildings. A building can finally be represented by applying a generic texture mapping. The details of this process are provided in section 3.5. Figure 5 shows a building model that is generated by applying the pre-warp step and texture mapping to the aerial photograph. Because the three-dimensional image warp process is divided into two stages, the generic texture mapping can be used. In addition, the virtual environment can easily be updated by simple pre-processing of the geospatial information, which is updated periodically.

3.5 Displacement Mapping for Representing Buildings The displacement mapping method proposed by Oliveira (2000) sequentially moves a pixel once along the x-axis and then along the y-axis respectively, so that the desired image is mapped efficiently by two iterations of one-dimensional computations. The original relief texture method is not appropriate if the height map is changed discontinuously as in the case of the boundary of a building. The height between the boundary of a building and the ground surface is changed abruptly. In Figure 6, let A be a pixel on the ground surface and B the pixel of the building top. After warping, pixel A should remain in its original location, and BCDE moves to a new position according to the difference in height. Sequential movements along the x- and y-axis result in Figure 6f. Because A and B, respectively, are the boundary points of the ground surface and the building, Figure 6b is obtained by applying warping along the x-axis. Dotted arrows in Figure 6 show the direction of interpolation between warped pixels. After warping pixels along the x-axis is completed as shown in Figure 6c, the same warping along the y-axis is applied to all pixels, as shown in Figures 6d and e. If the original method is © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

386

B Yoo and S Han

Figure 6 Warping error from the original relief texture method

applied to pixels along both the x- and y-axis, unnecessary pixels, FGH, will be processed as shown in Figure 6f. The use of the original warping method causes undesired interpolation along the boundary of the building, or breaks the relationships between adjacent pixels so that the sides of the building cannot be properly represented. Oliveira (2000) treated the problem of the discontinuous change of the height map as a special case. As a solution, he proposed that the topological information of the pixels be preserved and then restored when warping. However, in the case of a building where all the boundaries have discontinuous height changes, it is difficult to apply Oliveira’s method. Because all the buildings have the characteristic that the height map changes discontinuously from the ground surface, a better warping method is needed. The warping method is modified for building representations. The value of pixels in a height map is checked, and all pixels are classified into pixels corresponding to the buildings and pixels corresponding to the ground. The BCDE pixels of Figure 7 correspond to buildings. During the processes shown in Figures 7b–e, pixels are moved using the original method. Without interpolating the space between two pixels along x and y, only the final positions of BCDE are taken into account as shown in Figure 7e. As a result, the region corresponding to the building top is moved and only the shape of the roof is moved, as shown in Figure 8b. In order to display side walls of the building, another process is needed to compose the outer wall along the moving path of pixels, as shown in Figure 8c. In this study, using the concept of accumulating layers, as utilized in the layered imposter (Schaufler 1998), the pixels for the outer wall of the building are extended to the warping direction. Then, the pixels are filled inside the silhouette of the building with the outer wall geometry through the path extended from the original position BCDE in Figure 7a to BCDE in Figure 7f. Figure 8 shows the results of applying the proposed process to a real height map and source image. Figure 8a shows the image before warping. After warping, Figure 8d is obtained if the final positions of the building tops are taken. By filling the pixels following the moving paths (Figure 8e) we get the result shown in Figure 8f. © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Image-Based Modeling of Urban Buildings

387

Figure 7 Warping pixels for buildings

Figure 8 Extended pre-warp for a real scene. This figure appears in colour in the electronic version of this article and in the plate section at the back of the printed journal

4 Experimental Results 4.1 Implementation and Experiment To verify the proposed method for urban environments, experiments have been conducted for high-rise buildings and an apartment complex. Implementation is © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

388

B Yoo and S Han

Figure 9 Before and after representing an apartment block. This figure appears in colour in the electronic version of this article and in the plate section at the back of the printed journal

achieved with Microsoft Visual C++ and the Nvidia Cg language in an Intel Pentium 4 computer. Figure 9 shows the experimental result for an apartment block that occupies a large portion of the urban area of Korea. Apartment buildings are extracted from the 1:5,000scale digital map and their respective heights are calculated individually. Tens of buildings which have different heights are included in each height map. As the orthographic image is used in producing the digital map, warping can be easily accomplished by adjusting coordinates without any correction or complex mapping process. The results of the application of the proposed displacement mapping are shown in Figures 9d–f. Though the buildings are not modeled as polygons, the outlines of the buildings change according to their height and viewing position. The existing virtual environments are usually made by texture mapping of an aerial photograph on a terrain model that is created from a DEM (digital elevation model), and polygonal modeling of several buildings of interest. Buildings in the far distance are expressed as texture on the terrain model, such as that shown in Figures 9a–c. On the contrary, the height and silhouette of buildings are represented clearly in Figures 9d–f by using the proposed method. When the viewing position of the user is changed from the viewpoint of Figure 9a to that of Figure 9c, a change of scenery, such as parallax and occlusion of buildings, cannot be seen because the silhouettes of the buildings cannot be represented. However, the realism of the flight simulation is improved through the change of scenery with respect to occluded ground and buildings and motion parallax according to the change of the viewing position from Figure 9d to Figure 9f. The texture used in this paper is 1,000 × 1,000 RGB data. The height map is created using 8 bits per texel. Using the software image warping process, the rendering performance is about 10 frames per second for a 640 × 480 output image on a 2.8 GHz PC. The result of software image warping is not sufficient for a real-time application such as a helicopter simulation. © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Image-Based Modeling of Urban Buildings

389

Table 2 Comparison between polygonal modeling and image-based modeling for building representation Polygonal approach

Image-based approach

Number of test model blocks

Number of Polygons

Rendering performance (fps)

Number of Polygons

Rendering performance (fps)

1 2 4 16

2,070 4,140 8,280 33,120

300 162 85 60

2 2 2 2

74 72 68 67

4.2 Experimental Results with Hardware Acceleration In order to accelerate the displacement mapping process, we have implemented the techniques described in the paper as a fragment program (pixel shader) written in HLSL (i.e. Microsoft’s High Level Shading Language) and have used them to map buildings for a quad that is composed of two triangular polygons. With this implementation, all scenes are rendered at more than 60 frames per second for a 640 × 480 output image. The experiment is conducted on a 2.8 GHz PC with 1 GB of memory using a GeForce FX 5950 with 256 MB of memory. All instructions of the mapping process are operated on the GPU, which has programmable graphics hardware, and implemented in a perpixel fragment program. Figure 10 shows a comparison between two different approaches for modeling urban buildings from geospatial information. Both models are generated from the same GIS data, i.e. a digital map and an aerial photograph. Figure 10a is the rendered image using the polygon model and the model is composed of 2,070 triangles. Figure 10b is the rendering result using the image-based approach proposed in this study. Notice that visual effects such as self-occlusion and silhouettes, which are represented using the proposed image-based approach, show no differences from the polygon model. The main application of the proposed approach is representing the numerous buildings in the urban environment as used in a helicopter flight simulation. In order to simulate a case where numerous buildings in the distance are shown in a single scene, we have measured the performance of a tiled representation of the same model used in the previous experiment. Using the polygonal approach, as the tile factor increases, the number of polygons increases in proportion to the tile factor. Then, the rendering performance deteriorates. On the contrary, in the image-based approach, the number of polygons involved in rendering is independent of the number of building models. The rendering performance is also uniform regardless of the tile factor. A detailed comparison of the experimental results is given in Table 2. One model block in Table 2 is equivalent to the model shown in Figure 10. Although the comparison between the two different approaches was not conducted in precisely identical environments due to the distinct implementation requirements of each approach, the results are sufficient to show the effectiveness of the proposed approach. The uniform performance of the image-based approach using hardware acceleration is also appropriate for real-time applications such as a helicopter flight simulation. © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

390

B Yoo and S Han

Figure 10 Comparison between polygonal model and image-based model: (a) rendered image of polygonal model that is generated from a digital map and aerial photograph; and (b) rendered image of image-based model that is generated from the same source data used in (a). This figure appears in colour in the electronic version of this article and in the plate section at the back of the printed journal

Figures 11a and b show an example of a flight simulation where the virtual environment is composed with both a polygon model and the terrain model combined with an aerial photograph. Building A is modeled with polygons, whereas buildings B and C are presented only as an aerial photograph which is texture mapped on the terrain. The height direction of building A as indicated by a white arrow in Figure 11 changes according to the change of viewing position, because it is made as a polygonal model. However, in the case of building B, the direction of the building height is not correct and it appears to be lying horizontally on the ground, because it is not modeled as a polygonal model and only the texture is mapped on the terrain. Furthermore, as the viewpoint is moved clockwise in Figures 11a–b, the direction of the building height rotates in the wrong direction because the texture of building B rotates on a plane. The directionality of the building appears to be different between B and C because the direction from which the aerial photograph was taken is not the same for the buildings. However, buildings D, E, and F of Figures 10c–d are located properly in the height direction. As the viewpoint changes from Figure 10c to d, the outline and direction of © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Image-Based Modeling of Urban Buildings

391

Figure 11 Comparison with the existing method of building modeling. This figure appears in colour in the electronic version of this article and in the plate section at the back of the printed journal

buildings D, E, and F are moved in the proper way. It is possible to accurately represent buildings by creating a height map of buildings from a digital map. In addition, the effort of modeling is small compared to that demanded by polygon modeling. It is easy to apply periodically updated geospatial information. The proposed method cannot be applied to all kinds of representations of virtual urban environments. This approach is not suitable to buildings that have complicated shapes or are located close to the viewpoint because of the lack of detail. It is desirable to simultaneously apply the polygonal modeling method for several buildings of interest when it is applied to close buildings.

5 Conclusions Based on the displacement mapping, which is an image-based modeling method, a method for modeling urban environments has been proposed for a helicopter simulation. We examined the characteristics of Korean urban environments to provide a basis for developing the method that can make efficient use of GIS data, digital maps, and aerial photographs. It is possible to model buildings in an urban area with comparatively small effort. The proposed method is tested by modeling the real urban area where © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

392

B Yoo and S Han

the Korea Advanced Institute of Science and Technology is located as well as a cluster of high-rise office buildings. This method has advantages and disadvantages compared with existing methods. Given the planar outline and height of a building, generating polygon models is also straightforward. However, the texture mapping process, in terms of modeling, must be followed and the mapping between texture coordinates and model space for all surfaces of the polygon model is not straightforward in terms of automating the process. Coordinate mapping of the buildings and terrain surface must also be followed. In many of the practical applications of constructing a virtual environment for a flight simulation, these mapping processes are complex and not easily automated. The proposed method can utilize periodical updates of geospatial information. The renewal of the urban environment can be realized by simply changing the height and texture maps. On the contrary, the polygon models are generally updated individually according to changes in each building. If the DEM, which has the height values of terrain and the DSM (digital surface model), which has the real height values of terrain including perturbation of buildings, can be used in this approach, the height field of buildings for displacement mapping can be directly acquired without a pre-processing stage. Small polygons in the far distance create tiny surfaces. For visual effects and performance of polygonal representation, the technique referred to as LOD (level of detail) is used. The static LOD must be manually applied to polygon models and a popping problem also remains. The dynamic LOD is another problem of rendering. In the proposed approach, the process of displacement mapping is performed at the pixel level, and the load of computation is proportional to the number of pixels rendered on the image plane. As such, a supplementary LOD process is not required in the proposed approach. Compared with polygon-based modeling methods, the proposed approach can create a virtual environment quickly and economically. However, it is not suitable for near distance buildings with complex shapes. The main application domain of the proposed method is buildings in the medium to far distance of an urban environment, which is necessary in a helicopter simulation. Because the proposed method can use conventional texture mapping, the polygonal models of buildings can also be added. It is possible to use polygon-based models for buildings close to the viewpoint together with the proposed method. The following additional research is needed in the future: • Use of the NGIS version 2.0 digital map of Korea for the height map of buildings: Because the NGIS version 1.0 digital map does not have height information for the buildings, heights are calculated based on the number of floors in buildings. The exact height information of buildings will be available in the NGIS version 2.0 digital map and utilizing this information would further reduce the modeling effort. • Adding façade information for buildings: The proposed method does not use the side wall information of buildings. Song (2003) produced a building model using a texture mapping technique based on a library of 3D urban models. For effectiveness, Song (2003) classified buildings according to the size, height, shape, and position using LiDAR data and digital maps. If we can use the NGIS version 2.0 digital map, it would be simpler and more accurate to choose the texture of buildings, because NGIS version 2.0 contains feature information pertaining to the purpose of the building. © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

Image-Based Modeling of Urban Buildings

393

Acknowledgements This work is sponsored by MOST (Ministry of Science and Technology), Korea, under Management #M10329110001-03B4311-00110 and MIC (Ministry of Information and Communication), Korea, under the ITRC (Information Technology Research Center) support program.

References Cho Y 2004 Reconstruction of 3D Building Model by Predicting Occlusion Points: Technical Analysis Considering the Difference of Korean and European Building Style. Unpublished MS Thesis, School of Civil, Urban and Geo-System Engineering, Seoul National University (in Korean with English abstract) Choi J et al. 2003 A Study of Strategic Plan for 3D Geospatial Information Generation. Seoul, Korea Ministry of Construction and Transportation Report No 63 (in Korean with English abstract) Cook R L 1984 Shade trees. In Proceedings of the International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’84), Minneapolis, Minnesota: 223–31 De Boer W H 2000 Fast Terrain Rendering Using Geometrical MipMapping. WWW document, http://www.flipcode.com Debevec P E 1996 Modeling and Rendering Architecture from Photographs. Unpublished PhD Dissertation, Department of Computer Science, University of California at Berkeley Doggett M and Hirche J 2000 Adaptive view dependent tessellation of displacement maps. In Proceedings of the SIGGRAPH/Eurographics Workshop on Graphics, Interlaken, Switzerland: 59–66 Duchaineau M, Wolinsky M, and Sigeti D E 1997 ROAMing terrain: Real-time optimally adapting meshes. In Proceedings of IEEE Visualization Conference, Phoenix, Arizona: 81–8 Fujii K and Arikawa T 2002 Urban object reconstruction using airborne laser elevation image and aerial image. IEEE Transactions on Geoscience and Remote Sensing 40: 2234 – 40 Kang I and Park C 2003 Construction of cyber territory using digital maps and GIS. KSCE (Korean Society of Civil Engineers) Journal of Civil Engineering 23: 901–8 (in Korean with English abstract) Kautz J and Seidel H 2001 Hardware accelerated displacement mapping for image-based rendering. In Proceedings of the Graphics Interface Conference, Ottawa, Ontario: 61–70 Lee S, Jung S, and Nevatia R 2002 Integrating ground and aerial views for urban site modeling. In Proceedings of the International Conference on Pattern Recognition, Quebec City, Quebec: 107–12 Lee W, Kim J, Yu K, and Kim Y 2004 A study on the 3-dimensional modeling of buildings in urban areas using digital maps and LiDAR data. KSCE (Korean Society of Civil Engineers) Journal of Civil Engineering 24: 311–8 (in Korean with English abstract) McMillan L 1997 An Image-based Approach to Three-Dimensional Computer Graphics. Unpublished PhD Dissertation, Department of Computer Science, University of North Carolina at Chapel Hill Moons T, Frere D, Vandekerckhove J, and Van Gool L 1998 Automatic modeling and 3D reconstruction of urban house roofs from high resolution aerial imagery. In Burkhardt H and Neumann B (eds) Computer Vision: Proceedings of the Fifth European Conference on Computer Vision, Freiburg, Germany (ECCV’98). Berlin, Springer-Verlag Lecture Notes in Computer Science No 1406: 410 –25 Moule K and McCool M D 2002 Efficient bounded adaptive tessellation of displacement maps. In Proceedings of the Graphics Interface Conference, Calgary, Alberta: 171– 80 Nevatia R and Price K 2002 Automatic and interactive modeling of buildings in urban environments from aerial images. In Proceedings of the IEEE International Conference on Image Processing (ICIP-02), Rochester, New York: 525–8 Oliveira M M 2000 Relief Texture Mapping. Unpublished PhD Dissertation, Department of Computer Science, University of North Carolina at Chapel Hill © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)

394

B Yoo and S Han

Roettger S, Heidrich W, Slusallek P, and Seidel H 1998 Real-time generation of continuous levels of detail for height fields. In Proceedings of the Sixth International Conference in Central Europe on Computer Graphics and Visualization, Bory, Czech Republic: 315–22 Schaufler G 1998 Per-object image warping with layered impostors. In Proceedings of the EuroGraphics Workshop on Rendering Techniques, Vienna, Austria: 145–56 Schaufler G. and Priglinger M 1999 Efficient displacement mapping by image warping. In Proceedings of the EuroGraphics Workshop on Rendering Techniques, Granada, Spain: 175–86 Song J 2003 Library-Based Texture Mapping Method for 3D City Modeling. Unpublished MS Thesis, School of Civil, Urban and Geo-System Engineering, Seoul National University (in Korean with English abstract) Suveg I and Vosselman G 2004 Reconstruction of 3D building models from aerial images and maps. ISPRS Journal of Photogrammetry and Remote Sensing 58: 202–24 Web3D Consortium 2003 VRML Archives. WWW document, http://www.web3d.org/x3d/vrml

© 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd Transactions in GIS, 2006, 10(3)