Distributed software framework and continuous integration in hydroinformatics systems

For many years, modeling and GIS applications have been used as separate systems for planning, design and operation of water distribution systems. As digital information becomes increasingly available, the potential benefits of an integrated model and GIS system become more obvious. These benefits of integration and resulting automation include: better service to internal and external customers, improved work processes and higher productivity, reduced data entry errors and duplication, improved data security and reliability, and improved emergency response. Las Vegas Valley Water District (LVVWD) is developing an integrated system that includes a hydraulic and water quality model. Geographic Information System (GIS), Supervisory Control and Data Acquisition (SCADA), and other database systems. Model data is dynamically linked to the GIS information with unique identification features. The model utilizes GIS and other linked data to compute hydraulic and water quality results for master planning, operational planning, facility outage scenarios, and other what-if scenarios. The integrated system enables the data process to be automated to the maximum extent possible so that manpower and time is saved. This paper describes the significance of such integration and demonstrates its value with example applications. The examples encompass master planning, water quality planning and operational planning for rapid-growing communities. This paper was presented at the 8th Annual Water Distribution Systems Analysis Symposium which was held with the generous support of Awwa Research Foundation (AwwaRF).

International audience

Today's advanced agile practices such as Continuous Integration and Test-Driven Development support a wide range of software development activities to facilitate the rapid delivery of high-quality software. However, the reuse of pre-existing, third-party software components is not one of them. Software reuse is still primarily perceived as a time-consuming, unsystematic and ultimately, discontinuous activity even though it aims to deliver the same basic benefits as continuous software engineering - namely, a reduction in the time and effort taken to deliver quality software. However, the increasingly central role of testing in continuous software engineering offers a way of addressing this problem by exploiting the new generation of test-driven search engines that can harvest components based on tests. This search technology not only exploits artifacts that have already been created as part of the continuous testing process to harvest components, it returns results that have a high likelihood of being fit for purpose and thus of being worth reusing. In this paper, we propose to augment continuous software engineering with the rapid, continuous reuse of software code units by integrating the test-driven mining of software artifact repositories into the continuous integration process. More specifically, we propose to use tests written as part of the Test-First Development approach to perform test-driven searches for matching functionality while developers are working on their normal development activities. We discuss the idea of rapid, continuous code reuse based on recent advances in our test-driven search platform and elaborate on scenarios for its application in the future.

Geographic Information Systems (GIS) are defined as: “An integrated collection of computer software and data used to view and manage information about geographic places, analyze spatial relationships, and model spatial processes.” (Wade & Sommer, 2006) Originally, GIS users were comprised primarily of scientists and engineers. In time, hardware and software advancements provided the catalyst for expanded use by the global community. In addition, the continued integration and access to information systems via the Internet and mobile devices has contributed to the success of GIS in society. Aside from commercial training, institutions of higher learning have played a major role in the technical training of GIS professionals. As with other technically orientated topics of study, pedagogical considerations exist that include: problem solving skills, information retention, and approaches towards learning. A pedagogical approach utilizing student generated and existing Internet GIS resources may provide an effective method of addressing GIS education.

In this paper, we present a coverage-based regression test selection (RTS) approach and a developed tool for Python. The tool can be used either on a developer’s machine or on build servers. A special characteristic of the tool is the attention to easy integration to continuous integration and deployment. To evaluate the performance of the proposed approach, mutation testing is applied to three open-source projects, and the results of the execution of full test suites are compared to the execution of a set of tests selected by the tool. The missed fault rate of the test selection varies between 0–2% at file-level granularity and 16–24% at line-level granularity. The high missed fault rate at the line-level granularity is related to the selected basic mutation approach and the result could be improved with advanced mutation techniques. Depending on the target optimization metric (time or precision) in DevOps/MLOps process the error rate could be acceptable or further improved by using file-level granularity based test selection.

An ontology-based framework for semantic geographic information systems development and understanding

We propose a new approach to solving important practical problems of complex debugging, joint testing, and analysis of the execution time of software module versions in a heterogeneous distributed computing environment that integrating Grid and cloud computing. These problems arise in the process of supporting the continuous integration of modules of distributed applied software packages. The study focuses on the packages that are used to conduct large-scale computational experiments. The scientific novelty of the proposed approach is to combine the methodology for creating the packages with modern software development practices based on its continuous integration using knowledge about the specifics of the problems being solved. Our contribution is multifold. We expanded the capabilities of continuous integration tools by developing new additional tools for the markup and transformation of data from poorly structured sources and predicting modules execution time. In addition, we developed a technological scheme of the joint applying our developed tools and external systems for continuous integration. Therefore, we provide a more large range of capabilities of continuous integration in relation to the processes of creating and using the packages in comparison with the well-known tools. The fundamental basis of their functioning is a new conceptual model of the packages. This model supports the specification, planning, and execution of software continuous integration processes taking into account the specific subject data and problems being solved. Applying the developed tools in practice leads to a decrease in the number of errors and failures of applied software in the development and use of the packages. In turn, such decrease significantly reduces the time for large-scale computational experiments and increases the efficiency of using resources of the environment. The results of practical experiments on the use of system prototype for continuous integration of applied software show their high efficiency.

Unified Modeling Language (UML) plays an important role in the modern software development process by offering standardized methods for visual modeling, analysis, and design of complex systems. This paper discusses the main aspects of storing, editing, and integrating UML documents with integrated development environments (IDEs). The main focus is on comparing the two main formats for storing UML models – XML Metadata Interchange (XMI) and JavaScript Object Notation (JSON). XMI, being an official standard developed by Object Management Group, provides a high degree of detail and compatibility with various UML tools, making it preferable for large and complex projects that require support for the full UML specification. At the same time, JSON is simple and flexible, which makes it suitable for projects where speed of development and ease of integration are important, although it is inferior to XMI in describing complex aspects of UML models. The paper analyzes in detail the methods of editing UML documents, including manual editing and automated approaches, using APIs and scripts. Manual editing is useful when you need to make small changes to the structure of a UML document, but it can be time-consuming and error-prone when working with large projects. Automation, on the other hand, provides effective tools for mass editing and generating UML elements, which significantly speeds up development and minimizes the likelihood of errors, especially since modern tools such as StarUML offer the necessary tools for programmatically modifying UML documents by integrating them into development processes. Combining of UML documents with integrated development environments also plays a key role in increasing development efficiency, as this allows automating code generation based on UML models, maintaining synchronization between code and diagrams, and facilitating visualization and documentation of architectural solutions. Despite the significant advantages, there are challenges associated with limited support for all UML features and potential conflicts between models and code. This only emphasizes the need for further research in this area. Prospects for further research include the development of new methods and tools for working with UML documents, improving their integration with development environments, and using artificial intelligence to automate the analysis and design of UML models. Attention should also be paid to the using of UML with continuous integration processes, which can significantly increase the flexibility and adaptability of software development. Thus, the study of the structure of storing, editing, and integrating UML documents is an important area for optimizing the design and development of modern software systems.

This paper covers the challenges in software validation, especially in the context of complex, embedded systems. Problems related to limited validation environment resources (DUT - Devices Under Test, workload and traffic generators, measurement setups, test probes), stability and complexity of systems will be highlighted. Authors will describe validation strategy which responds to quality assurance and risk management requirements applied for real products provided to the market. Presented technologies covers solutions based on Continuous Integration and Continuous Delivery process of modern, complex, embedded systems. Authors present both known and new solutions and approaches to known challenges, also suggest possible further development directions to improve efficiency and optimize product development cost.

This paper focuses on the utilization and integration of geographic information systems (GIS) oriented to tragic safety in vehicles. We have primarily analyzed the benefits of introducing GIS systems that support the driver and provide processed data to collision warning and collision avoidance systems located in vehicles. Further on, we have defined a set of information that is relevant for traffic safety and appropriate to be used within an in-vehicle GIS system. Based on these data, we propose in this paper a specific design for the GIS system. This system integrates the relevant data for traffic safety and takes into consideration the specific requirements of the vehicular domain. The integration of the GIS system within vehicles introduces a series of problems that are discussed in the paper.

Presented on Wednesday 18 May: Session 18 With the drive to continually reduce the cost of connecting new Coal Seam Gas (CSG) wells, Worley developed new methodologies using commercial off-the-shelf software, and custom development, to automate engineering and improve operational efficiency for pipelines and gathering systems. One of the recent developments is an automated and data-driven approach to produce Gathering Network Diagrams (GNDs), an alternative to conventional Piping & Instrumentation Diagrams (P&IDs). On a recent project, Worley implemented a GND workflow capable of showing ~40 wells per A3 deliverable in conformance with intelligent drafting standards. This reduced ~2000 as-built P&IDs to ~200 GNDs (and associated standard details) with a significant improvement in quality and usability. Prior to generation, a site walkdown was conducted to address mismatches between as-built P&IDs, Geographic Information System (GIS) and what was actually installed at site. Our client now has a suite of GNDs that match the installed asset and are produced from the corporate GIS database, essentially their Digital Twin. Worley then focused on integrating flow assurance software with GIS allowing data-driven execution of future design packages to be incorporated into the master GNDs. We successfully integrated with PipeSIM and GAP, allowing for seamless transfer of hydraulic inputs from GIS to the modelling software. Engineers can now synchronise the confirmed process design back to GIS without producing red-lined and/or CAD-drafted deliverables which greatly improves efficiency and reduces human error. This paper demonstrates how digital automation can be applied to the CSG industry to significantly reduce cost and schedule whilst enabling data-driven operational efficiencies. To access the presentation click the link on the right. To read the full paper click here

Considerable progress has been made in developing physically based, distributed parameter, hydrologic/water quality (H/WQ) models for planning and control of nonpoint‐source pollution. The widespread use of these models is often constrained by the excessive and time‐consuming input data demands and the lack of computing efficiencies necessary for iterative simulation of alternative management strategies. Recent developments in geographic information systems (GIS) provide techniques for handling large amounts of spatial data for modeling nonpoint‐source pollution problems. Because a GIS can be used to combine information from several sources to form an array of model input data and to examine any combinations of spatial input/output data, it represents a highly effective tool for H/WQ modeling. This paper describes the integration of a distributed‐parameter model (AGNPS) with a GIS (ARC/INFO) to examine nonpoint sources of pollution in an agricultural watershed. The ARC/INFO GIS provided the tools to generate and spatially organize the disparate data to support modeling, while the AGNPS model was used to predict several water quality variables including soil erosion and sedimentation within a watershed. The integrated system was used to evaluate the effectiveness of several alternative management strategies in reducing sediment pollution in a 417‐ha watershed located in southern Iowa. The implementation of vegetative filter strips and contour buffer (grass) strips resulted in a 41 and 47% reduction in sediment yield at the watershed outlet, respectively. In addition, when the integrated system was used, the combination of the above management strategies resulted in a 71% reduction in sediment yield. In general, the study demonstrated the utility of integrating a simulation model with GIS for nonpoint‐source pollution control and planning. Such techniques can help characterize the diffuse sources of pollution at the landscape level.

Considering water quality is an essential requirement in terms of environmental planning and management. To protect and manage water resources effectively, it is necessary to develop an analytical decision-support system. In this study, a systematic approach was suggested to evaluate the lake water quality. The methodology includes the prediction of the values in different locations of the lakes from experimental data through inverse distance weighting (IDW) method, creation of maps by using Geographic Information System (GIS) integrated with analytic hierarchy process (AHP) from multi-criteria decision analysis (MCDA), reclassification into five class, combining the time-related spatial data into a single map to predict the whole lake water quality from the data of sampling points, and finally overlapping the final maps with topography/geology and land use. The proposed approach was verified and presented as case study for Meke and Acigol Lakes in Konya/Turkey which were affected by human and natural factors although they have ecological, hydromorphological, and socio-economic importance. In the proposed approach, categorizing water quality parameters as “hardness and minerals,” “substrates and nutrients,” “solids content,” “metals,” and “oil-grease” groups was helpful for AHP with the determined group weights of 0.484, 0.310, 0.029, and 0.046, respectively. Assigning weights within each group and then assigning weights between groups resulted in creating accurate final map. The proposed approach is flexible and applicable to any lake water quality data; even with a limited number of data, the whole lake water quality maps could be created for assessment.

Geographic Information Systems (GIS) integration into the industrial management is a crucial aspect that has helped in boosting the operational performance and sustainability. The tools offered by GIS technology for spatial data analysis, real time data visualization and informed decision making add to greater efficiency of resource allocation and improved supply chain logistics for the industries. The aim of this paper is to identify the main reasons for the use of GIS in the industrial environments, which include the recent technological advances, the regulatory compliance, and the necessity of the operational efficiency. By illustrating how GIS tools help companies fulfill sustainability goals, including better tracking of environmental impacts, reduction of waste, and of energy, the document identifies industries that are immediately utilizing GIS tools. Additionally, the paper talks about the use of GIS in enabling competitive advantages, minimizing costs and work flow optimization. Real time data collection systems, predictive modeling and automation is studied as a feature of bringing about continuous improvement of industrial practice. Examples of applying the GIS approach to optimization of process in manufacturing, transportation and energy sectors are presented in case studies. It also addresses the challenges of GIS implementation, other than high initial costs and data integration, as the future trends and innovations that will influence the evolution from GIS in industrial management. In this paper, we stress the key role of GIS in making industrial management practices change in a way that provides organizational success in the long term due to increased efficiency and sustainability.

This paper presents a framework for improving campus management and enhancing academic capability through the integration of geographic information systems (GIS) in the functions of Philippine higher education institutions (HEIs). Institutional units applied with a GIS are described. Methods are briefly discussed to implement the plan as well as to sustain and institutionalize GIS.