Detecting window line using an improved stacked hourglass network based on new real-world building façade dataset

In this thesis, concepts for developing Spatial Data Infrastructures with an emphasis on visualizing 3D landscape and city models in distributed environments are discussed. Spatial Data Infrastructures are important for public authorities in order to perform tasks on a daily basis, and serve as research topic in geo-informatics. Joint initiatives at national and international level exist for harmonizing procedures and technologies. Interoperability is an important aspect in this context - as enabling technology for sharing, distributing, and connecting geospatial data and services. The Open Geospatial Consortium is the main driver for developing international standards in this sector and includes government agencies, universities and private companies in a consensus process. 
\n3D city models are becoming increasingly popular not only in desktop Virtual Reality applications but also for being used in professional purposes by public authorities. Spatial Data Infrastructures focus so far on the storage and exchange of 3D building and elevation data. For efficient streaming and visualization of spatial 3D data in distributed network environments such as the internet, concepts from the area of real time 3D Computer Graphics must be applied and combined with Geographic Information Systems (GIS). For example, scene graph data structures are commonly used for creating complex and dynamic 3D environments for computer games and Virtual Reality applications, but have not been introduced in GIS so far. 
\nIn this thesis, several aspects of how to create interoperable and service-based environments for 3D spatial data are addressed. These aspects are covered by publications in journals and conference proceedings. The introductory chapter provides a logic succession from geometrical operations for processing raw data, to data integration patterns, to system designs of single components, to service interface descriptions and workflows, and finally to an architecture of a complete distributed service network.
\nDigital Elevation Models are very important in 3D geo-visualization systems. Data structures, methods and processes are described for making them available in service based infrastructures. A specific mesh reduction method is used for generating lower levels of detail from very large point data sets. An integration technique is presented that allows the combination with 2D GIS data such as roads and land use areas. This approach allows using another optimization technique that greatly improves the usability for immersive 3D applications such as pedestrian navigation: flattening road and water surfaces. It is a geometric operation, which uses data structures and algorithms found in numerical simulation software implementing Finite Element Methods. 
\n3D Routing is presented as a typical application scenario for detailed 3D city models. Specific problems such as bridges, overpasses and multilevel networks are addressed and possible solutions described. The integration of routing capabilities in service infrastructures can be accomplished with standards of the Open Geospatial Consortium. An additional service is described for creating 3D networks and for generating 3D routes on the fly. Visualization of indoor routes requires different representation techniques.
\nAs server interface for providing access to all 3D data, the Web 3D Service has been used and further developed. Integrating and handling scene graph data is described in order to create rich virtual environments. Coordinate transformations of scene graphs are described in detail, which is an important aspect for ensuring interoperability between systems using different spatial reference systems. The Web 3D Service plays a central part in nearly all experiments that have been carried out. It does not only provide the means for interactive web-visualizations, but also for performing further analyses, accessing detailed feature information, and for automatic content discovery.
\nOpenStreetMap and other worldwide available datasets are used for developing a complete architecture demonstrating the scalability of 3D Spatial Data Infrastructures. Its suitability for creating 3D city models is analyzed, according to requirements set by international standards. A full virtual globe system has been developed based on OpenStreetMap including data processing, database storage, web streaming and a visualization client.
\nResults are discussed and compared to similar approaches within geo-informatics research, clarifying in which application scenarios and under which requirements the approaches in this thesis can be applied.
\n

2.5D map is a convenient and efficient approach to exploiting a massive three-dimensional (3D) city model in web GIS. With the rapid development of oblique airborne photogrammetry and photo-based 3D reconstruction, 3D city models are becoming more and more accessible. 3D Geographic Information System (GIS) can support the interactive visualization of massive 3D city models on various platforms and devices. However, the value and accessibility of existing 3D city models can be augmented by integrating them into web-based two-dimensional (2D) GIS applications. In this paper, we present a step-by-step workflow for generating orthorectified oblique images (2.5D maps) from massive 3D city models. The proposed framework can produce 2.5D maps from an arbitrary perspective, defined by the elevation angle and azimuth angle of a virtual orthographic camera. We demonstrate how 2.5D maps can benefit web-based visualization and exploitation of massive 3D city models. We conclude that a 2.5D map is a compact data representation optimized for web data streaming of 3D city models and that geometric analysis of buildings can be effectively conducted on 2.5D maps.

Abstract. In an era of rapid urbanization, three-dimensional (3D) city models have emerged as crucial tools for managing infrastructure, planning new developments, and maintaining accurate cadastral records. Recent advances in photogrammetry and geographic information systems (GIS) have allowed us to generate incredibly detailed representations of urban spaces. The General Directorate of Land Registry and Cadastre has played a pivotal role in creating 3D city models by capturing aerial imagery and converting it into CityGML-based solid and architectural models. Although CityGML offers a standardized framework for storing and sharing city data, its potential can be further magnified by integrating immersive technologies such as Augmented Reality (AR). This paper presents a mobile AR application, developed with Google’s ARCore, that displays CityGML-based building data at street level, allowing for real-time visualization and interaction. Our pipeline simplifies raw CityGML data, ensuring optimal rendering while preserving essential details. Through effective georeferencing and advanced pose estimation, these digital models are accurately overlaid on real-world scenes, enabling government agencies, urban planners, and cadastral managers to analyze and make informed decisions in situated places. Beyond planning and cadastre, this AR system could benefit broader fields such as environmental monitoring and public infrastructure management. We conclude by discussing future steps, including expanded data layers (e.g., utility networks) and performance optimizations for large-scale city models, thus highlighting the transformative role of AR in shaping next-generation urban environments.

Abstract. The interest in studies on 3D city models has renewed over time thanks to technological innovations that concern the methods of data acquisition and automatic classification of unstructured point clouds, to model increasingly specialized geometric objects for the growing needs of urban transformation management. Managing geometric and semantic information in cognitive systems, based on standards, increasingly open to the inquiries, analyses, and predictions that today are necessary to implement for future assessments, can digitally twin reality. The problem focused on in this work is the modelling phase, which requires efficient automatisms to make the production phase of geometric objects more sustainable, whether they are then destined to become parametric features in a BIM/CIM environment or 3D geometric objects of a geodatabase.A comprehensive methodology for automating the generation of 3D meshes from point cloud data using Python code, with a focus on practical implementation is presented parallelly to evaluations regarding the generation of the geometric 3D contents of the urban environment representation, depending on whether or not it includes examples of architectural heritage (of all ages). Leveraging the Open3D library and NumPy, the methodology provides different experiences, tests, and related discussions in the field of architectural and urban 3D surface reconstruction and visualization. The paper outlines essential steps including data loading and preparation, meshing strategy selection, data processing using Poisson reconstruction, and mesh exportation and visualization. Additionally, a method for generating multiple Levels of Detail (LoD) meshes is introduced, enhancing the applicability of the approach for city modelling framework applications, including architectural elements of buildings. The presented Python code serves as a practical resource for implementing 3D mesh generation workflows seamlessly.

3D city models are nowadays used in very different applications. Due to this, the commercial exchange and, thus, the demand for high quality data becomes more and more important. We describe a quality model that defines common parameters used for spatial quality measurement, especially when dealing with 3D city models. Therefore, we explore different representations of city models: the reality, the user's idea and the digital data set. A welldefined mathematical formalism which addresses the different quality parameters is presented. This formalism also helps to create algorithms for measurement and improvement of spatial data quality. The aim is to be able to define uniformed criteria which can be easily transformed into software. The implemented prototype serves as base for the evaluation in which we provide examples based on actual data sets using typical quality requirements.

Current 3D city models are already available for many cities world-wide. However, the production of historical city models is still in its infancy. In this paper a procedure is presented that combines different data sources in order to derive individual 3D city models of different time periods using the example of the Free and Hanseatic City of Hamburg. A wooden model of the city from the year 1644 and an official map from 1859 have been used as a basis for the generation of the 4D city model. The physical model (~1:1000) was scanned by a fringe projection system for 3D modelling, while the digitized data from the map were combined with height information from different data sources. These two geo-referenced 3D city models were used to derive further epochs (1200, 1400, 1589 and 1700) using different historical bird's-eye views (isometric views) of the city. For interactive navigation and visualization of the 4D city model a program was developed using the Google Earth Application Programming Interface.

D City Models (3DCM) are key features into decision making of several urban related problems. Therefore 3DCM are needed by several applications, but the required level-of-detail (LoD) of the model depends on the application. Our goal is to propose a multi-scale 3DCM production and use method. Our approach consists of merging, procedural modeling, graph rewriting techniques, and a generalization technique to handle all different kinds of LoD of a 3DCM. In this way, it allows to handle various heterogeneous LoDs of a complete urban city model. We test our proposal with the 3DCM of the City of Nantes for a rendering application. Our results can also be applied to other LoDs criteria to match other 3DCM-based needs. 3D city models (3DCM) are fully recognized as useful tools in several urban usages, and these models are used in several fields related to urban planning: decision-making processes, analysis of city characteristics, architectural and urban design, simulations of physical phenomena, information or scientific visualization, official communication, and so on. Besides the problem of a unified data format to share the 3DCM between these various applications, each application needs specific geometric data in order to perform its own process, and sometimes a single application needs the same city model at various scales. In order to avoid a costly multiplication of several models of the same city for each specific need, the 3DCM can be adapted to each application by using a multi-scale 3DCM. The production of a multi-scale 3DCM implies a multilevel modeling and a multi-representation of the city model; so different representations of 3D city models at different Levels of Detail (LoDs) are required. There are two ways of obtaining a 3D model at a given LoD: modeling from scratch or deriving from existing models. Modeling depends on data acquisition techniques or production techniques, whereas derivation is confined by the starting models. Photogrammetry and laser scanning technologies are usually used to capture 3D urban data. These techniques are very efficient to obtain detailed models but there is too much data to remove to obtain low precision model. Procedural modeling is used to produce data according to specific rules. Recent research shows it possible to fit rulesets used for procedural modeling to existing datasets (e.g. images and LIDAR data), so procedural modeling is a technique that is now able to model any kind of urban environment, from modeling synthetic cities to represent accurate existing ones. This method can produce 3DCM with details efficiently but lacks to control the amount of data generated. In general, LoD can be manually set at the rules to have local control density. At the other end of the processing scheme, generalization techniques are used to simplify existing data. They allow to simplify the model but precision depends on input data.

Three‐dimensional city models are becoming a valuable resource because of their close geospatial, geometrical, and visual relationship with the physical world. However, ground‐oriented applications in virtual reality, 3D navigation, and civil engineering require a novel modeling approach, because the existing large‐scale 3D city modeling methods do not provide rich visual information at ground level. This paper proposes a new framework for generating 3D city models that satisfy both the visual and the physical requirements for ground‐oriented virtual reality applications. To ensure its usability, the framework must be cost‐effective and allow for automated creation. To achieve these goals, we leverage a mobile mapping system that automatically gathers high‐resolution images and supplements sensor information such as the position and direction of the captured images. To resolve problems stemming from sensor noise and occlusions, we develop a fusion technique to incorporate digital map data. This paper describes the major processes of the overall framework and the proposed techniques for each step and presents experimental results from a comparison with an existing 3D city model.

This chapter introduces a novel technique for post-disaster mapping and disaster scale estimation based on an integrated framework of SAR remote sensing and a 3D city model database, optical remote sensing imagery is used for validation purpose. SAR based urban damage detection is well established and has been used for many years. We have showed that how the existing 3D City Model can be updated with the information extracted from satellite remote sensing data. The third dimension will play a very crucial role in evacuation rout planning of damaged or affected buildings. In this study in the proposed–three-level (L1, L2, L3)–model damage assessment information is integrated with the semantic knowledge of 3D city models in order to better organize the search and rescue operation. L1 includes remotely sensed Synthetic Aperture Radar (SAR) space-borne data collection from the affected areas; L2 includes a change detection process; and L3 includes the estimation of the most affected building(s). Using this model, we show how the day-night image acquisition capability of a SAR sensor and semantic information from a 3D city model can be effectively used for post disaster mapping for rapid search and rescue operations. For L1, the combination of very high resolution SAR data and a 3D city model in CityGML format is used. L2 works under predefined conditions to detect the types of changes that have occurred. In L3, an interval-valued fuzzy soft set theory method is proposed with which to estimate the scale of damage to the affected structures (buildings). In this study, we show the potential application of existing 3D city models (with semantic knowledge) in combination with SAR remote sensing for post disaster management activities, especially for search and rescue operations.

Abstract. A 3D city model should be constantly updated with new versions, either to reflect the changes in its real-world counterpart, or to improve and correct parts of the model. However, the current standards for 3D city models do not support versioning, and existing version control systems do not work well with 3D city models. In this paper, we propose an approach to support versioning of 3D city models based on CityJSON and the concepts behind the Git version control system, including distributed and non-linear workflows. We demonstrate the benefits of our approach in two examples and in our software prototype, which is able to extract a given version of a 3D city model and to display its history.

In a workflow creating 3D city models, facades of buildings can be reconstructed from oblique aerial images for which the extrinsic and intrinsic parameters are known. If the wall planes have already been determined, e.g., based on airborne laser scanning point clouds, facade textures can be computed by applying a perspective transform. These images given, doors and windows can be detected and then added to the 3D model. In this study, the “Scaled YOLOv4” neural network is applied to detect facade objects. However, due to occlusions and artifacts from perspective correction, in general not all windows and doors are detected. This leads to the necessity of automatically continuing the pattern of facade objects into occluded or distorted areas. To this end, we propose a new approach based on recurrent neural networks. In addition to applying the Multi-Dimensional Long Short-term Memory network and the Quasi Recurrent Neural Network, we also use a novel architecture, the Rotated Multi-Dimensional Long Short-term Memory network. This architecture combines four two-dimensional Multi-Dimensional Long Short-term Memory networks on rotated images. Independent of the 3D city model workflow, the three networks were additionally tested on the Graz50 dataset for which the Rotated Multi-Dimensional Long Short-term Memory network delivered better results than the other two networks. The facade texture regions, in which windows and doors are added to the set of initially detected facade objects, are likely to be occluded or distorted. Before equipping 3D models with these textures, inpainting should be applied to these regions which then serve as automatically obtained inpainting masks.KeywordsDeep learningLSTMFacade reconstructionStructure completionImage inpainting

Nowadays, 3D city models are used by the increasing number of applications. Most applications require not only geometric information but also semantic information. As a standard and tool for 3D city model, CityGML, provides a method for storing and managing both geometric and semantic information. Moreover, it also provides the multi-scale representation of 3D building model for efficient visualization. In CityGML, building models are represented in five LODs (Level of Detail), start from LOD0, LOD1, LOD2, LOD3, and LOD4. Each level has different accuracy and detail requirement for visualization. Usually, for obtaining multi-LOD of 3D building model, several data sources are integrated. For example, LiDAR data is used for generating LOD0, LOD1, and LOD2 as close-range photogrammetry data is used for generating more detailed model in LOD3 and LOD4. However, using additional data sources is increasing cost and time consuming. Since the development of TLS (Terrestrial Laser Scanner), data collection for detailed model can be conducted in a relative short time compared to photogrammetry. Point cloud generated from TLS can be used for generating the multi-LOD of building model. This paper gives an overview about the representation of 3D building model in CityGML and also method for generating multi-LOD of building from TLS data. An experiment was conducted using TLS. Following the standard in CityGML, point clouds from TLS were processed resulting 3D model of building in different level of details. Afterward, models in different LOD were converted into XML schema to be used in CityGML. From the experiment, final result shows that TLS can be used for generating 3D models of building in LOD1, LOD2, and LOD3.

Ray trace based radio propagation prediction in urban environments has been recognized as a remarkable method today. It provides a more accurate prediction than other methods such as empirical statistic models. But this method requires 3D city model database and it takes much time to execute simulation, so it is not easy to apply this method for large cities. This paper presents GIS (Geographical Information System) based radio propagation simulator using ray trace method. This system generates a 3D city model automatically from 2D digital maps stored in GIS. The generated city model is managed using hierarchical bounding volume to effective collision test between rays and buildings. Simulation results such as propagation paths and power levels of receivers are shown over the 3D city model. It helps users to analyze the simulation results and also to predict power delay profile. Based on the experimental results using an actual digital map, the effectiveness of this system is totally discussed from the viewpoint of practical usage.

Impact of the Quality of Spatial 3D City Models on Sensor Networks Placement Optimization

A large number of cities have created semantic 3D city models, but these models are rarely used as input data for simulations, such as noise and flooding, in the urban planning process. Reasons for this are that many simulations require detailed land cover (LC) and elevation data that are often not included in the 3D city models, and that there is no linkage between the elevation and land cover data. In this study, we design, implement and evaluate methods to handle LC and elevation data in a 3D city model. The LC data is stored in 2.5D or 3D in the CityGML modules Transportation, Vegetation, WaterBody, CityFurniture and LandUse, and a complete 3D LC partition is created by combining data from these modules. The entire workflow is demonstrated in the paper: creating 2D LC data, extending CityGML, creating 2.5D/3D data from the 2D LC data, dividing the LC data into CityGML modules, storing it in a database (3DCityDB) and finally visualizing the data in Unreal Engine. The study is part of the 3CIM project where a national profile of CityGML for Sweden is created as an Application Domain Extension (ADE), but the result is generally applicable for CityGML implementations.