More

How to run a GAM model in GIS?


How do you generate a species distribution probability map from GAM models? Or in other words, how do you set up a GAM equation in GIS to produce probability of species distribution?

It is simple to apply generalized linear models (GLM) or equation in GIS due to the independent variables have a liner relationship with dependent variable but for GAM, I am not able to do it because the parameters in GAM are non-linear predictors and also their estimates (β) are not be calculated.


I do not believe that there is a package for GAM's in Python. I would recommend the gam package in R. For spatial predictions you can utilize the raster package to wrap the generic predict function in a spatial prediction. There is ample information on implementing these types of models. If you would like some introduction on working with spatial objects in R I have some tutorials on my website (evansmurphy.wix.com/evansspatial) and there is plenty of additional information to be found through an internet search.


Probably the most popular tool for fitting and projecting Species Distribution Models (SDMs) is R http://www.r-project.org/. There are several very good packages (e.g. 'dismo', 'raster') which offer a lot more functionality for SDM than the common GIS softwares like Arc or QGIS.

If you have not had much experience with R there will inevitably be a learning curve, but if you will be doing a lot of modelling then it might be worth learning.


I haven't used this toolset myself yet but some researchers at Duke University built the Marine Geospatial Ecology Tools. It looks like you can run R script and GAM models integrated directly into ArcGIS.


Aviation is one of the biggest beneficiaries of GIS. Geographic systems are used in various tasks in the airport as well as management of aircraft. The kinds of GIS used in aviation must be very accurate, efficient, and reliable. It must also work on a real-time base to ensure a smooth running of the airports and flights at large. GIS ability to offer distributed and multi-tasking processing enables various activities to be coordinated all at the same time. This is not achievable through manual methods.

1. Aviation structures management: Many tall structures like towers and buildings are monitored using GIS. Surveying on these structures is very important. GIS can be integrated even to manage control of automated tasks in some of these structures found at the airport. Communication between different towers or buildings at the airport is also manageable through GIS.

2. Airport layout planning: 3D visualization is among the pros of having GIS. GIS is, in most cases applied during the design period. Three-dimensional layouts of land and structures are presented between teams of engineers and experts to realize a secure and efficient system of transport. The designs can be tested for errors, a simulation is done as well as ideas discussed on how the designs can be improved. The 3D ability of geographic information systems also ensures specific properties to be captured hence, finer designs and model are achieved. GIS application does not only end during the construction of the airport but is also continued during the running of daily activities at the airport.

3. Flightpath monitoring: Most airports and airstrips have radars that make use of radio waves to communicate. These radars are usually integrated with geographic information systems to offer real-time tracking of aircraft. An aircraft traveling over the sea or even land is identifiable by the nearest radar. GIS has for long been used in aviation to provide path monitoring for the aircrew and airport management. Routes can be mapped digitally and availed to the pilots while at any place on earth. Incase emergencies the same system is used to identify last known locations for emergency response. Investigations on the cause of flight tragedies can be done through flight monitoring to understand what might have caused failures by the aircraft with relation to the routes undertaken.

4. Risk and security evaluation: Security authorities at the airport can conduct crime mapping and prediction for quick and informed decisions making as the kind of GIS used student are very accurate on probabilistic issues. During terror attacks that may happen any time, GIS integrated satellite imaging services may be used to provide visualization on the airport.

5. Airport lighting control: Airport lighting is very crucial. It ensures that operations at the airport are done with ease and normally. Most runways are also marked with lighting facilities to guide pilots as they land and take-off during the night. These lights are best controlled by use of Geographic information systems that offer real-time control mechanisms. The system automatically lights the airport when it’s dark by sensing the absence of sunlight as well as performs shutting down of lighting utilities during the day. This saves humans the significant work that would be involved in lighting the airport manually. It also proves to be quicker and efficient.

6. Environmental and weather assessment: storms are a great hindrance to flight operations. GIS can be integrated with weather systems to ensure that conditions that flights are canceled for safety reasons when necessary. Also, in places that experience lots of volcanic activities that can prevent aircraft from flying, GIS is solely responsible for gathering crucial datasets that can be sampled to produce reports. These reports can then be used to plan whether airplanes fly or not and if yes, the routes to be taken are also identified. Lots of research work done about the effects and impacts that are brought about by the airport being in a given location are also analyzed and stored by GIS. GIS is thus of great help in weather and environmental assessment.

7. Capacity planning: Everything starting from cargo areas, aircraft parking bays, passenger terminus, and many other areas should be well crafted to hold even bigger capacities of traffic, vehicles, aircraft, and people. Strategic planning on space and mapping of airports are some of the critical roles achieved by GIS in aviation.

8. Parking management: Most people who are willing to travel bring along their cars. The airport must, therefore, have a well-integrated system to monitor parking areas all around. GIS may be well designed to serve as a platform where users can query and lookup for map addresses on empty parking slots. This can fasten activities at the airport for visitors who wish to park their vehicles.


Geographic Information Systems (GIS)

Out team utilises GIS across all areas of the business by incorporating spatial analysis and map production into our projects. GIS can help visualise data and solve complex problems across a wide range of industries including:

  • Land use planning
  • Catchment scale water use and management
  • Horticulture industry development
  • Agriculture industry strategic planning
  • Planning and evaluation of natural resource management.

We have the skills to analyse spatial data produce high quality maps in print, digital and online interactive formats and develop, verify and maintain spatial datasets. RMCG and our Tasmanian team also have the capacity to provide project support in mapping to other consultancy firms who wish to outsource the mapping component of their projects, with QGIS being our preferred GIS platform.

Using data in a spatial form adds significant value and clarity to the story being told. We are able to display data such as ABS Census data in a meaningful way that is easy to understand. We can utilise spatial data to analyse risk, allowing us to identify locations that have a higher risk factor based on the likelihood and consequence of a potential threat occurring. We can also produce high quality maps that add a valuable visual element to the data analysis and narrative.

An example of a GIS map of Ecological Vegetation Classes in Victoria.
Our GIS offering
  • Property maps: We can produce the maps required for all the services we provide. We can also produce maps as required by clients, for example farm layouts or concept plans showing paddocks, pivots, infrastructure, etc.
  • GIS constraints model: We have developed a GIS model for determining land that is constrained for agricultural use. The outputs of the model assist in determining appropriate zoning boundaries.
  • Crop suitability model: Our Tasmanian team developed industry specific models to determine suitability of land for particular developments, for example a vineyard suitability model identifies land in the West Tamar Municipality that is suitable for vineyard development. These models pre-date the State Government’s Enterprise Suitability models and include input from industry specialists. They are designed to be used in conjunction with other tools such as Constraints Analysis and Enterprise Scale to assist with determining agricultural potential of land which is sought after for alternative use such as residential purposes.
  • Geo-referenced mapping: We can produce geo-referenced maps compatible with software such as Avenza Maps, which is free to download and install on smartphones or tablets. Georeferenced maps provide location in the field and can be used for ground truthing the location of a new dam, fenceline, etc.
  • Training in using field based applications: We provide training in the use of georeferenced maps for field based applications.

Click here for more information on how GIS has been used to support RMCG’s recent projects.


Applications

Over the years the GIS office has developed various, mostly web-oriented, applications which are utilized by users both inside and outside the institute.

  • Entobase
    This application is for managing entomological data acquired from the Parasitology laboratory.
    Go to the application >
  • IZSVe-GIS
    Web-GIS application for managing the veterinary geographic data of the Istituto Zooprofilattico Sperimentale delle Venezie.
    Go to the application >

Integrating GIS and Hydraulic Modeling

Does it make sense for water utilities to integrate GIS (geographic information system) technology with hydraulic modeling? In most cases, the answer is yes.

In the past, hydraulic models were reconstructed every few years, a necessary but time-consuming process. However, as hydraulic models have increased in complexity and need to be updated more frequently, utilities are looking for more cost-effective and timesaving methods to renew these models. GIS may provide the answer.

Editor's note: This article previously appeared in the November/December 2014 issue of Water Efficiency magazine. Does it make sense for water utilities to integrate GIS (geographic information system) technology with hydraulic modeling? In most cases, the answer is yes. In the past, hydraulic models were reconstructed every few years, a necessary but time-consuming process. However, as hydraulic models have increased in complexity and need to be updated more frequently, utilities are looking for more cost-effective and timesaving methods to renew these models. GIS may provide the answer. [text_ad] Typically, hydraulic analyses, such as determination of system capacities, development of what-if scenarios, and planning of improvements have been performed independent of GIS technology. [caption align="alignleft"] Credit: Moore County, Inc.
The GIS hydraulic water model can run scenarios for capital improvement plans, fire flow analyses, water quality, and future growth of the distribution system.[/caption] Originally, the primary function of a GIS for a water or wastewater utility was to be able to map capital assets. However, it quickly became obvious that GIS was more than just a mapping tool. When integrated with hydraulic modeling, GIS also provides opportunities for spatial analysis and data management. "Increasingly, GIS is becoming a system of record for all assets in a water utility system," says Joe Ball, director of marketing, water, North America, for Itron. "Integrating hydraulic modeling with GIS makes sense, because it allows both systems to share a single database. It simplifies data entry, since elevation data, pipe sizes, devices, and many other parameters are normally identified in GIS. This makes the use and implementation of any hydraulic modeling software easier." Ed Kura, GIS manager for Aclara, says, "It makes sense for utilities to include all of their hydraulic modeling within their GIS, because the GIS is a great place to collect this information and edit it as time goes on, so they can keep track of their network." Kura has spent the majority of his career working on building GIS databases for water and wastewater networks, including the integration of hydraulic modeling. "When hydraulic modeling is integrated with GIS, it provides utilities with a very powerful tool to let them know how their system is performing at the time the model is created, as well as how it should perform," says Steve Bruskiewicz, product manager, water, for Aclara who spent almost 25 years managing water and wastewater plants before coming to the company. "Without hydraulic modeling and GIS, utilities really end up working "˜blind.'" Are there any instances when it might not make sense to integrate the two? "Integration of GIS and hydraulic modeling may not make sense if GIS data is not accurate or up-to-date," says Ball. "It may be necessary to update, or even upgrade, the GIS system to make sure that hydraulic modeling software integration is effective." Benefits of GIS and Hydraulic Modeling Integration Integrating GIS and a hydraulic model provides planners and technicians access to more reliable, up-to-date information, reduced response time, and accessibility of modeling elements and data. According to Lori Armstrong, GIS and hydraulic modeling are complementary technologies. "By integrating them with each other, water utility companies can reap substantial time and cost savings," adds Armstrong, who is global water/wastewater industry manager for Esri and author of Hydraulic Modeling and GIS (Esri Press 2011). "A well-designed integration of the two systems provides ready access to mission-critical data," she says. "As a result, risk-of-failure analysis, repair and replacement, capacity assessment, capital improvement planning, and other water utility applications run more efficiently and effectively." Integration of a GIS with a hydraulic model allows utilities to get the most from their GIS investment. "Integrating GIS and hydraulic modeling enables utilities to quickly run scenarios for capital improvement plans, fire flow analyses, water quality, and future growth of the distribution system," says Armstrong. It also significantly improves the ability to update and enhance modeling efforts and minimize future costs of hydraulic model development and maintenance, all while using the most correct and best data available, and limiting risk and unnecessary assumptions. [caption align="alignleft"] Credit: ESRI
A water distribution system mapped in ArcGIS, using the World Imagery Base Map[/caption] A water distribution system mapped in ArcGIS, using the World Imagery Base Map As Armstrong sees it, the uniting of GIS, CMMS, LIMS, AMR/AMI, and the hydraulic model creates a connected communication decision support platform that allows operators and managers to make the best, most efficient decisions about how to operate their systems. "There is no need to maintain and update two separate systems," says Ball. "Updating one system automatically updates the other system. It also means that no updates are required whenever models for additions or upgrades need to be performed." Results for "what if" scenarios can be quickly obtained, minimizing costs and maximizing efficiency. This close relationship between systems also allows for a more robust model analysis. Data can also be shared with other systems not related to hydraulic modeling. "There are many benefits to the integration of GIS and hydraulic modeling," says Bruskiewicz. "For one, you know where all of your assets are. They are labeled, located, and you also usually know what condition they are in, because they were physically inspected when they were added to the GIS database." You can also start to do water loss auditing and modeling, because you can look at the age of your system and various pressures in each zone. "For example, if you know where your zones of high pressure are, you can estimate what your water losses will be in those zones and also estimate what your water losses will be if you reduce the pressure in those zones," he says. How to Get Started According to Armstrong, leveraging a utility's existing GIS is the easiest way to begin the development of a hydraulic model. "Because GIS manages the utility's most current data, including the majority of data needed to develop a model, it is critical to maintain the link between the model and GIS, while recognizing inherent differences between a water system GIS and model," she says. "Municipalities are now moving toward modeling using regularly-updated GIS information to create hydraulic simulations, and, in some instances, even run "˜up-to-the-minute' information from other systems," says Armstrong. "For example, data from an asset management system can be integrated and correlated with hydraulic performance, then compared against SCADA trends to validate and respond to a field event or perform a model calibration." This can be done in real time. Data from the automatic meter reading (AMR)/Advanced Metering Infrastructure (AMI) system, including leaks, are sent for instant communication to field staff to investigate and respond. According to Faheem Jalali, marketing manager, Aqualyze, the definition of the integration of GIS and hydraulic modeling can vary from one application to another. "Typically, hydraulic models are constructed using GIS databases containing physical properties of assets to be modeled," he says. In a wastewater system, this data will be a combination of nodes and links representing maintenance holes (MH) and pipes at the simplest level. "The GIS data will hold information such as how a given pipe is connected, i.e., node on the upstream and node on the downstream end," he says. "Pipe and MH data, such as their sizes and invert elevations, are also contained within the GIS database." When a hydraulic model is developed using this GIS data, the process can be automated through data import tools that are commonly available in all modeling platforms. The definition of integration may vary at this point, depending on how a utility wants to manage its data. "Generally, it is not a good idea, in my opinion, to have a live link between the GIS and hydraulic modeling, for various reasons," says Jalali. "For example, in most utilities, modeling falls under engineering, and GIS does not, which creates data ownership issues." Furthermore, he notes, modeling is a planning tool and is done interactively, evaluating many scenarios that need not be managed in GIS. GIS is a true inventory of a given system and should be kept as such. "Different levels of accuracy are available, so it is important to determine which level of accuracy is required to ensure that all necessary data is available and with the desired level of detail," says Itron's Ball. "It is also important to identify which tasks can be best performed within the GIS system, and which ones can be best performed within the hydraulic modeling system. Those tasks that can be performed in either system should be identified as well." According to Aclara's Kura, integration of GIS and hydraulic modeling depends on individual circumstances. "If a utility is starting from scratch and doesn't have a GIS, it can design the GIS database to include the data they will need for the hydraulic modeling and then populate it from that point," he says. However, if the utility already has a GIS, it will need to determine what changes will be needed to the database. "That is, the GIS generally needs to be made to work with the hydraulic model," he says. "It doesn't work the other way around, because the models are usually pretty rigid." https://www.youtube.com/watch?v=30jNrNrzzsA Challenges to Integration According to Esri's Armstrong, there are often pipeline connectivity issues and data deficiencies when developing a model from GIS. These challenges can be addressed through a combination of standard operating procedures, tools within the GIS and modeling software that are used to improve data accuracy, as well as protocols that can be established to allow the transfer of data corrections and modifications between the model and the GIS. "This can be done while also allowing for a repeatable process that can be duplicated on future model updates," she says. "More communication is also needed between the model developer, other system administrators, and GIS staff, which will improve data coordination and effective data management in order to successfully maintain a hydraulic model." "Hydraulic modeling is a way to analyze system capacity and future needs," says Jalali. "The source data of any model should be reflected the same in both platforms, and can therefore be considered integrated." Additional information, such as modeling results under various scenarios, can be provided back to the GIS system. This requires close coordination between engineers/modelers and the GIS data management staff. Challenges can be posed by how various departments are structured within a utility, by communications and data sharing between departments, by read/write access to databases, by integrating modeling analysis results from third parties such as consultants, or by design of data schemas to hold or link GIS and modeling information. "The key to overcoming these challenges is to define objectives, to document and follow protocols and guidelines, and to ensure close coordination and communication between interested parties in a given utility," says Jalali. According to Itron's Ball, GIS data may not be accurate enough for the hydraulic model to develop useful data. Examples of this data include pipe diameters, pipe length, and elevations. "Periodic model updates and maintenance of the databases are paramount," he says. "Ongoing maintenance work, pipe replacement programs, and changes in demand require updates to the hydraulic model to ensure accuracy." To overcome these challenges, thorough preparation work is required. "The utility needs to make sure that hydraulic modeling software and GIS used, or planned to be used, are compatible," says Ball. "The utility also needs to make sure data in GIS is up-to-date and accurate--that correct information is available, that no duplicates exist, et cetera." Data in GIS needs to be complete and accurate enough for use with hydraulic modeling. GIS and Hydraulic Modeling Integration in Action One government agency that has had significant success integrating GIS and hydraulic modeling is Moore County (Carthage, NC). In fact, its integration of GIS and hydraulic modeling won the North Carolina GIS Award (G. Herbert Stout Award) in 2013, as well as the North Carolina IT Award (Government Innovation Grant Award) in 2014. The initial motivation for creating the GIS hydraulic water model was the GIS Department's recognition of the costs of the existing procedure--contracting out hydraulic water model requests to engineering firms. These contracts proved costly because the engineering firms would often need to recreate a water model and would only do so for the area of concern. There were also many assumptions made by the engineering firms in creating these water models in order to produce hydraulic estimates for design work. "The Moore County Public Works Department never had a model that represented the entire system," says Chris Butts, GISP, PLS, the GIS coordinator and interim IT Director of the Moore County GIS Department. "It was very difficult to understand where system improvements could be made. It was also difficult to know what system improvements needed to be made for long-range planning." The GIS Department realized that it could create an in-house hydraulic water model that would use its high-quality utility data for the network. The vision was to create a hydraulic water model for the entire Moore County-maintained distribution system. This model could then be continued as part of the GIS utility data maintenance program. The GIS Department also obtained model parameters from the Public Utilities Operation Manager in order to decrease the assumptions that were made in previous water models. The GIS hydraulic water model could then quickly run scenarios for capital improvement plans, fire flow analyses, water quality, and future growth of the distribution system. [caption align="alignleft"] Credit: ESRI
Water and Sewer Networks mapped in ArcGIS, using the World Topographic Map[/caption] The first task before a hydraulic water model could begin was to have accurate utility GIS data. This was accomplished by scanning all record drawings related to utility information and georeferencing them. These georeferenced record drawings were then used to update the GIS data and to serve as a reference for Public Utility staff, so they could quickly pull up a digital record drawing. However, some of these record drawings were very old, and there had been improvements and maintenance done that were not recorded. This is why it was important to field-verify the utility information. This was mostly done using handheld Trimble GPS units with TerraSync and ArcPad, which allowed customized data dictionaries for easy cataloging. Where elevation data was needed, a more precise survey-grade GPS receiver was used equipped with RTK (real-time kinematic). "Making sure the network topology was correct in GIS was critical in having the hydraulic water model run successfully," says Butts. "We were able to use many automated tools within GIS to check for network connectivity and errors." The next step was to obtain a water modeling software. The Moore County GIS and Public Utilities Department requested the help of the North Carolina local government community to see what others were using with GIS and how satisfied they were. It was then decided to use the Innovyze product, InfoWater, which offers a fully integrated water distribution modeling software application. They also offered onsite training, so staff could fully understand the intricacies of the modeling software. The next step was to obtain practical data about the distribution system on an operational level. This meant identifying and describing wells, tanks, booster stations, and water sources. Gathering information about pump controls, pump curves, tank volumes, well depths, and such was essential in understanding how the distribution system operated, so these same parameters could be used to control the model. Water billing data was joined to address data, to determine where to allocate demand in the model. Most models will give ballpark estimates on water demand, but the Moore County GIS Department wanted to be as accurate and detailed as possible. The next phase was to create the hydraulic water model using the existing GIS utility data. Before this could be done, network validation was used to check for any possible errors within the network, using ArcGIS Network Analyst. Tools from Innovyze were used to import the GIS utility SDE (spatial database engine) data to a model database that could be viewed within ArcGIS. Operational distribution data was then used to control the flow of water through the water model network. The final step was cleaning up any hydraulic errors that were preventing the model from having successful runs. Once the model was running, the next step was to make sure that it was producing results that matched the real-world distribution system. This was done by comparing fire flow tests that were conducted by the Public Utilities Division with fire flow analysis results from the model. Variables were changed to calibrate the model, such as the pipe roughness coefficient, which is affected by pipe age, water quality aggressiveness, and solids depositing in the pipe. The more solids depositing in a pipe, the smaller the diameter becomes, which limits the flow of water. Initially, this coefficient was assigned by the age and material of the pipe, but it was changed during the calibration based on actual field measurements. The next phase was to create an extended-period simulation so that the distribution system could be monitored through an interval of time. Applying average and maximum day multipliers to the system allows one to see how the distribution system could be viewed on an hourly basis. The hydraulic water model was not only constructed to report existing conditions, but also for long-range planning. The benefits have been many. Having a GIS hydraulic water model allows Moore County to plan improvements and design, as well as better operate and manage a water distribution system. This allows for optimal water distribution performance concerning hydraulics, water quality, and future growth. Not only has this increased the water quality and efficiency of the water distribution system, but it has also been a cost-savings measure for the residents of Moore County. The hydraulic water model has strengthened the GIS enterprise and has enhanced effective communication between county departments. In addition, the Public Utilities Division now has an accurate model portrayal of its system. It can more effectively manage capital improvement plans of the water distribution system, while making smarter decisions. It has also allowed for better and more informed communication between the Public Works Director and the Utility Operations Manager when gaining an understanding of the distribution system. The Public Utilities Division is now able to run as many scenarios as they see fit, as well as analyze any location in the distribution system, looking at a variety of factors. The Public Utilities Division has benefited financially by not having to outsource the modeling work. It has been able to change the direction and scope quickly on an as-needed basis. "We were able to develop a 10- and 20-year water distribution plan for Moore County," says Butts. "The GIS water model allowed us to review several water source ideas and then estimate the cost for each of these plans. This gave our Board of Commissioners a thorough review, and they were able to make a better decision based on the information the water model provided."

Typically, hydraulic analyses, such as determination of system capacities, development of what-if scenarios, and planning of improvements have been performed independent of GIS technology.

Credit: Moore County, Inc.
The GIS hydraulic water model can run scenarios for capital improvement plans, fire flow analyses, water quality, and future growth of the distribution system.

Originally, the primary function of a GIS for a water or wastewater utility was to be able to map capital assets. However, it quickly became obvious that GIS was more than just a mapping tool. When integrated with hydraulic modeling, GIS also provides opportunities for spatial analysis and data management.

“Increasingly, GIS is becoming a system of record for all assets in a water utility system,” says Joe Ball, director of marketing, water, North America, for Itron. “Integrating hydraulic modeling with GIS makes sense, because it allows both systems to share a single database. It simplifies data entry, since elevation data, pipe sizes, devices, and many other parameters are normally identified in GIS. This makes the use and implementation of any hydraulic modeling software easier.”

Ed Kura, GIS manager for Aclara, says, “It makes sense for utilities to include all of their hydraulic modeling within their GIS, because the GIS is a great place to collect this information and edit it as time goes on, so they can keep track of their network.” Kura has spent the majority of his career working on building GIS databases for water and wastewater networks, including the integration of hydraulic modeling.

“When hydraulic modeling is integrated with GIS, it provides utilities with a very powerful tool to let them know how their system is performing at the time the model is created, as well as how it should perform,” says Steve Bruskiewicz, product manager, water, for Aclara who spent almost 25 years managing water and wastewater plants before coming to the company. “Without hydraulic modeling and GIS, utilities really end up working “˜blind.'”

Are there any instances when it might not make sense to integrate the two?

“Integration of GIS and hydraulic modeling may not make sense if GIS data is not accurate or up-to-date,” says Ball. “It may be necessary to update, or even upgrade, the GIS system to make sure that hydraulic modeling software integration is effective.”

Benefits of GIS and Hydraulic Modeling Integration
Integrating GIS and a hydraulic model provides planners and technicians access to more reliable, up-to-date information, reduced response time, and accessibility of modeling elements and data.

According to Lori Armstrong, GIS and hydraulic modeling are complementary technologies. “By integrating them with each other, water utility companies can reap substantial time and cost savings,” adds Armstrong, who is global water/wastewater industry manager for Esri and author of Hydraulic Modeling and GIS (Esri Press 2011).

“A well-designed integration of the two systems provides ready access to mission-critical data,” she says. “As a result, risk-of-failure analysis, repair and replacement, capacity assessment, capital improvement planning, and other water utility applications run more efficiently and effectively.”

Integration of a GIS with a hydraulic model allows utilities to get the most from their GIS investment.

“Integrating GIS and hydraulic modeling enables utilities to quickly run scenarios for capital improvement plans, fire flow analyses, water quality, and future growth of the distribution system,” says Armstrong. It also significantly improves the ability to update and enhance modeling efforts and minimize future costs of hydraulic model development and maintenance, all while using the most correct and best data available, and limiting risk and unnecessary assumptions.

Credit: ESRI
A water distribution system mapped in ArcGIS, using the World Imagery Base Map

A water distribution system mapped in ArcGIS, using the World Imagery Base Map

As Armstrong sees it, the uniting of GIS, CMMS, LIMS, AMR/AMI, and the hydraulic model creates a connected communication decision support platform that allows operators and managers to make the best, most efficient decisions about how to operate their systems.

“There is no need to maintain and update two separate systems,” says Ball. “Updating one system automatically updates the other system. It also means that no updates are required whenever models for additions or upgrades need to be performed.”

Results for “what if” scenarios can be quickly obtained, minimizing costs and maximizing efficiency. This close relationship between systems also allows for a more robust model analysis. Data can also be shared with other systems not related to hydraulic modeling.

“There are many benefits to the integration of GIS and hydraulic modeling,” says Bruskiewicz. “For one, you know where all of your assets are. They are labeled, located, and you also usually know what condition they are in, because they were physically inspected when they were added to the GIS database.”

You can also start to do water loss auditing and modeling, because you can look at the age of your system and various pressures in each zone.

“For example, if you know where your zones of high pressure are, you can estimate what your water losses will be in those zones and also estimate what your water losses will be if you reduce the pressure in those zones,” he says.

How to Get Started
According to Armstrong, leveraging a utility’s existing GIS is the easiest way to begin the development of a hydraulic model. “Because GIS manages the utility’s most current data, including the majority of data needed to develop a model, it is critical to maintain the link between the model and GIS, while recognizing inherent differences between a water system GIS and model,” she says.

“Municipalities are now moving toward modeling using regularly-updated GIS information to create hydraulic simulations, and, in some instances, even run “˜up-to-the-minute’ information from other systems,” says Armstrong. “For example, data from an asset management system can be integrated and correlated with hydraulic performance, then compared against SCADA trends to validate and respond to a field event or perform a model calibration.”

This can be done in real time. Data from the automatic meter reading (AMR)/Advanced Metering Infrastructure (AMI) system, including leaks, are sent for instant communication to field staff to investigate and respond.

According to Faheem Jalali, marketing manager, Aqualyze, the definition of the integration of GIS and hydraulic modeling can vary from one application to another. “Typically, hydraulic models are constructed using GIS databases containing physical properties of assets to be modeled,” he says.

In a wastewater system, this data will be a combination of nodes and links representing maintenance holes (MH) and pipes at the simplest level. “The GIS data will hold information such as how a given pipe is connected, i.e., node on the upstream and node on the downstream end,” he says. “Pipe and MH data, such as their sizes and invert elevations, are also contained within the GIS database.” When a hydraulic model is developed using this GIS data, the process can be automated through data import tools that are commonly available in all modeling platforms.

The definition of integration may vary at this point, depending on how a utility wants to manage its data. “Generally, it is not a good idea, in my opinion, to have a live link between the GIS and hydraulic modeling, for various reasons,” says Jalali. “For example, in most utilities, modeling falls under engineering, and GIS does not, which creates data ownership issues.”

Furthermore, he notes, modeling is a planning tool and is done interactively, evaluating many scenarios that need not be managed in GIS. GIS is a true inventory of a given system and should be kept as such.

“Different levels of accuracy are available, so it is important to determine which level of accuracy is required to ensure that all necessary data is available and with the desired level of detail,” says Itron’s Ball. “It is also important to identify which tasks can be best performed within the GIS system, and which ones can be best performed within the hydraulic modeling system. Those tasks that can be performed in either system should be identified as well.”

According to Aclara’s Kura, integration of GIS and hydraulic modeling depends on individual circumstances. “If a utility is starting from scratch and doesn’t have a GIS, it can design the GIS database to include the data they will need for the hydraulic modeling and then populate it from that point,” he says.

However, if the utility already has a GIS, it will need to determine what changes will be needed to the database. “That is, the GIS generally needs to be made to work with the hydraulic model,” he says. “It doesn’t work the other way around, because the models are usually pretty rigid.”

Challenges to Integration
According to Esri’s Armstrong, there are often pipeline connectivity issues and data deficiencies when developing a model from GIS. These challenges can be addressed through a combination of standard operating procedures, tools within the GIS and modeling software that are used to improve data accuracy, as well as protocols that can be established to allow the transfer of data corrections and modifications between the model and the GIS.

“This can be done while also allowing for a repeatable process that can be duplicated on future model updates,” she says. “More communication is also needed between the model developer, other system administrators, and GIS staff, which will improve data coordination and effective data management in order to successfully maintain a hydraulic model.”

“Hydraulic modeling is a way to analyze system capacity and future needs,” says Jalali. “The source data of any model should be reflected the same in both platforms, and can therefore be considered integrated.”

Additional information, such as modeling results under various scenarios, can be provided back to the GIS system. This requires close coordination between engineers/modelers and the GIS data management staff. Challenges can be posed by how various departments are structured within a utility, by communications and data sharing between departments, by read/write access to databases, by integrating modeling analysis results from third parties such as consultants, or by design of data schemas to hold or link GIS and modeling information.

“The key to overcoming these challenges is to define objectives, to document and follow protocols and guidelines, and to ensure close coordination and communication between interested parties in a given utility,” says Jalali.

According to Itron’s Ball, GIS data may not be accurate enough for the hydraulic model to develop useful data. Examples of this data include pipe diameters, pipe length, and elevations.

“Periodic model updates and maintenance of the databases are paramount,” he says. “Ongoing maintenance work, pipe replacement programs, and changes in demand require updates to the hydraulic model to ensure accuracy.”

To overcome these challenges, thorough preparation work is required. “The utility needs to make sure that hydraulic modeling software and GIS used, or planned to be used, are compatible,” says Ball. “The utility also needs to make sure data in GIS is up-to-date and accurate–that correct information is available, that no duplicates exist, et cetera.” Data in GIS needs to be complete and accurate enough for use with hydraulic modeling.

GIS and Hydraulic Modeling Integration in Action
One government agency that has had significant success integrating GIS and hydraulic modeling is Moore County (Carthage, NC). In fact, its integration of GIS and hydraulic modeling won the North Carolina GIS Award (G. Herbert Stout Award) in 2013, as well as the North Carolina IT Award (Government Innovation Grant Award) in 2014.

The initial motivation for creating the GIS hydraulic water model was the GIS Department’s recognition of the costs of the existing procedure–contracting out hydraulic water model requests to engineering firms. These contracts proved costly because the engineering firms would often need to recreate a water model and would only do so for the area of concern. There were also many assumptions made by the engineering firms in creating these water models in order to produce hydraulic estimates for design work.

“The Moore County Public Works Department never had a model that represented the entire system,” says Chris Butts, GISP, PLS, the GIS coordinator and interim IT Director of the Moore County GIS Department. “It was very difficult to understand where system improvements could be made. It was also difficult to know what system improvements needed to be made for long-range planning.”

The GIS Department realized that it could create an in-house hydraulic water model that would use its high-quality utility data for the network. The vision was to create a hydraulic water model for the entire Moore County-maintained distribution system. This model could then be continued as part of the GIS utility data maintenance program. The GIS Department also obtained model parameters from the Public Utilities Operation Manager in order to decrease the assumptions that were made in previous water models. The GIS hydraulic water model could then quickly run scenarios for capital improvement plans, fire flow analyses, water quality, and future growth of the distribution system.

Credit: ESRI
Water and Sewer Networks mapped in ArcGIS, using the World Topographic Map

The first task before a hydraulic water model could begin was to have accurate utility GIS data. This was accomplished by scanning all record drawings related to utility information and georeferencing them. These georeferenced record drawings were then used to update the GIS data and to serve as a reference for Public Utility staff, so they could quickly pull up a digital record drawing. However, some of these record drawings were very old, and there had been improvements and maintenance done that were not recorded. This is why it was important to field-verify the utility information. This was mostly done using handheld Trimble GPS units with TerraSync and ArcPad, which allowed customized data dictionaries for easy cataloging. Where elevation data was needed, a more precise survey-grade GPS receiver was used equipped with RTK (real-time kinematic).

“Making sure the network topology was correct in GIS was critical in having the hydraulic water model run successfully,” says Butts. “We were able to use many automated tools within GIS to check for network connectivity and errors.”

The next step was to obtain a water modeling software. The Moore County GIS and Public Utilities Department requested the help of the North Carolina local government community to see what others were using with GIS and how satisfied they were. It was then decided to use the Innovyze product, InfoWater, which offers a fully integrated water distribution modeling software application. They also offered onsite training, so staff could fully understand the intricacies of the modeling software.

The next step was to obtain practical data about the distribution system on an operational level. This meant identifying and describing wells, tanks, booster stations, and water sources. Gathering information about pump controls, pump curves, tank volumes, well depths, and such was essential in understanding how the distribution system operated, so these same parameters could be used to control the model. Water billing data was joined to address data, to determine where to allocate demand in the model. Most models will give ballpark estimates on water demand, but the Moore County GIS Department wanted to be as accurate and detailed as possible.

The next phase was to create the hydraulic water model using the existing GIS utility data. Before this could be done, network validation was used to check for any possible errors within the network, using ArcGIS Network Analyst. Tools from Innovyze were used to import the GIS utility SDE (spatial database engine) data to a model database that could be viewed within ArcGIS. Operational distribution data was then used to control the flow of water through the water model network. The final step was cleaning up any hydraulic errors that were preventing the model from having successful runs.

Once the model was running, the next step was to make sure that it was producing results that matched the real-world distribution system. This was done by comparing fire flow tests that were conducted by the Public Utilities Division with fire flow analysis results from the model. Variables were changed to calibrate the model, such as the pipe roughness coefficient, which is affected by pipe age, water quality aggressiveness, and solids depositing in the pipe. The more solids depositing in a pipe, the smaller the diameter becomes, which limits the flow of water. Initially, this coefficient was assigned by the age and material of the pipe, but it was changed during the calibration based on actual field measurements.

The next phase was to create an extended-period simulation so that the distribution system could be monitored through an interval of time. Applying average and maximum day multipliers to the system allows one to see how the distribution system could be viewed on an hourly basis. The hydraulic water model was not only constructed to report existing conditions, but also for long-range planning.

The benefits have been many. Having a GIS hydraulic water model allows Moore County to plan improvements and design, as well as better operate and manage a water distribution system. This allows for optimal water distribution performance concerning hydraulics, water quality, and future growth. Not only has this increased the water quality and efficiency of the water distribution system, but it has also been a cost-savings measure for the residents of Moore County. The hydraulic water model has strengthened the GIS enterprise and has enhanced effective communication between county departments.

In addition, the Public Utilities Division now has an accurate model portrayal of its system. It can more effectively manage capital improvement plans of the water distribution system, while making smarter decisions. It has also allowed for better and more informed communication between the Public Works Director and the Utility Operations Manager when gaining an understanding of the distribution system. The Public Utilities Division is now able to run as many scenarios as they see fit, as well as analyze any location in the distribution system, looking at a variety of factors. The Public Utilities Division has benefited financially by not having to outsource the modeling work. It has been able to change the direction and scope quickly on an as-needed basis.

“We were able to develop a 10- and 20-year water distribution plan for Moore County,” says Butts. “The GIS water model allowed us to review several water source ideas and then estimate the cost for each of these plans. This gave our Board of Commissioners a thorough review, and they were able to make a better decision based on the information the water model provided.”


EXAMPLES OF GIS IN HEALTH RESEARCH

A study performed by Dulin et al.30 determined the need for increased access to primary care services in various communities using what they termed a Multiple Attribute Primary Care Targeting Strategy through the implementation of GIS technology. This proved to be extremely valuable in evaluating health services across rural areas as interventions could be applied. A second example determined the walkability of local communities to help encourage physical activity.26 , 31 It was identified that various communities do not always contain adequate walking areas. GIS not only identified these problem areas, but the technology was also used to determine how to improve those walking systems. This can be useful for elderly populations as they are in an age group of declining health and many do not have a valid driver’s license.30 Such conclusions may not have been possible without GIS.

A third example of GIS in epidemiology and health informatics includes a study whose goal was to ‘determine the importance of geographic and spatial behavioural factors as predisposing and enabling factors in health care utilization of rural communities’.32 This is essentially the definition of what GIS in health aims to accomplish. In this study, GIS was used through the completion of questionnaires addressing participants’ demographic and socioeconomic characteristics, health status, health insurance coverage, medical care options, location of care providers, personal beliefs about health care, use of health services, health prevention behaviours, locations of daily activities and degree of social isolation from others.32 It was found that geographic and spatial factors can have significant impacts in the utilization of health care services.

Similarly, the Patient Access Area Model was developed by executing a GIS.23 This allowed for the evaluation of medical supply and demand to make informed predictions about access to hospitals. Through this, GIS allowed for the conclusion that over 9000 citizens in a southwest area of Japan would not receive proper hospital care and an intervention was planned accordingly.


Model builder vs scripting

Anyways building something in model builder is not the same as using a scripting language to build something is it?

I think the answer to your question is "yes" (I think--question has awkward wording), to a certain degree. Page 39 of the book Python Scripting for ArcGIS explains Python vs ModelBuilder well enough. I'll try to summarize:

Some lower-level geoprocessing tasks are only possible in scripts (i.e. using cursors to manipulate rows in an attribute table).

Scripting allows for advanced programming logic such as error handling and more advanced data structures

A script can run outside of ArcGIS a ModelBuilder "script" cannot.

Scripts can be scheduled to run, Models to my knowledge, can't.

ModelBuilder "models" cannot be intergrated with other software.

In general, a script allows you more flexibility in both where/how/in what situation you need to use it. Creating a model in ModelBuilder and then converting it into a (python) script can give you the framework/basic idea of how a process may need to be automated.


Why Greenplum Database in Integrated Analytics

The importance of advanced analytics in its various forms is growing rapidly in enterprise computing. Key enterprise data typically resides in relational and document form and it is inefficient to copy data between systems to perform analytical operations. Greenplum is able to run both traditional and advanced analytics workloads in-database. This integrated capability greatly reduces the cost and the silos created by procuring and maintaining multiple tools and libraries.

Greenplum Database advanced analytics can be used to address a wide variety of problems in many verticals including automotive, finance, manufacturing, energy, government, education, telecommunications, on-line and traditional retail.


NIS – a network information system – is the further development and adaption of GIS to the specific needs of grid and network owners. Typical users are utility and telecom companies.

A NIS provides the tools to understand and gain control over a network. It constantly answers business-critical questions about a grid or network, such as:

  • What are my assets?
  • Where are my network assets?
  • How are these assets connected?
  • Who is using the network assets?

In addition, it supports specific workflows for network owners, from planning to maintenance, with the same starting point: the map.


GIS: Geographic Information Systems: About GIS Services

Geospatial information science, or geographic information system (GIS), is a "framework for gathering, managing, and analyzing data (Esri, 2019)." There are many applications of GIS, spanning a wide variety of disciplines. Some examples include:

Biological Sciences: migration patterns, species diversity, fisheries management

Health Sciences: epidemiology, noise pollution

Emergency Services: wildfire spread, missing person location, least-cost paths to accident sites

Environmental Sciences: soil/groundwater contamination, drainage networks, erosion, pollution

Social Sciences: crime rates, food deserts, population dynamics, education

Northwestern University Library (NUL), independently and in partnership with other units, consults with and educates to ensure that students, faculty, and staff can pursue their research using an array of geospatial data, methods, analysis, and visualization tools. NUL cultivates an institutional environment of experts to maximize NU geospatial resources.


Watch the video: Change Projection - Coordinate System in ArcMap (October 2021).