ColdFusion Summit 2016 - Powering GIS Operations with ColdFusion
- 5. 10/11/2016 5
About Me
• GIS Analyst for Central Contra Costa Sanitary District (www.centralsan.org)
• GIS Technician (2000-2009), GIS Analyst (2009-present)
• Tools I Use:
• ColdFusion, JavaScript, Python, Microsoft SQL Server, Esri ArcGIS Desktop and
Server, Geocortex, Safe FME (spatial ETLs)
• Manager of Bay Area ColdFusion User Group (BACFUG) – on hiatus
• Adobe Community Professional for ColdFusion
• Married, two grown children
• Unreformed DIY remodeler
• Passionate about craft beer!
- 7. 10/11/2016 7
What Do We Do?
• “Protecting Public Health and the Environment”
• Wastewater collection and treatment
• Household hazardous waste collection
• Treat 30-50 million gallons wastewater per day
• Produce 600 million gallons a year of recycled water
• Maintain 1,500 miles of collection system pipelines
- 8. 10/11/2016 8
Dictionary Definition
“An integrated collection of
computer software and data used to
view and manage information
about geographic places, analyze
spatial relationships, and model
spatial processes. A GIS provides a
framework for gathering and
organizing spatial data and related
information so that it can be
displayed and analyzed”
-A to Z GIS, Esri Press
You use or benefit from GIS every day
• Google/Bing/Apple Maps
• Google Earth
• GPS navigation systems
• Product or store locators in
apps/websites
• 911 emergency response
• Package delivery (UPS/FedEx)
What Is A Geographic Information System?
- 10. 10/11/2016 11
2000 - Connect the “Data Silos”
GIS
CMMS
Pipeline
CCTV
Permits &
Parcel
Records
Environmental
Compliance
Hydraulic
Modeling
Scanned
Documents
- 11. 10/11/2016 12
Planning begins in early 2000’s
• Started planning an Intranet GIS portal
• Requirements:
• Run in web browser
• Interactive
• Easy to use
• Easy to maintain
• Integrate GIS data with other non-spatial data
- 12. 10/11/2016 13
Departmental Databases to Link Up
• Operations department Sussex Computerized Maintenance Management
System (CMMS) – Microsoft Access based
• Land use, ownership and sewer permits (Sungard ERP on IBM AS/400)
• Pipe flow capacity modeling system – Microsoft Access
• Environmental Compliance enforcement – Microsoft Access
- 13. 10/11/2016 14
Pilot GIS Portal Project (2000-2002)
• Selected AutoDesk MapGuide for in-browser map interface
• Server component on Windows/IIS
• ActiveX control (Internet Explorer) or Java applet (Netscape Navigator) for browsers
• How do we connect the databases?
• MapGuide bundled with license for Allaire ColdFusion 4.5
• Hired consultant to develop proof-of-concept site
• Developed spatial Extract-Transform-Load (ETL) processes
• Created a map
• Wrote data reports (in CFML)
- 14. 10/11/2016 15
A Few Silos Connected (2000-2002)
GIS +
ColdFusion
CMMS
Pipeline
CCTV
Permits &
Parcel
Records
Environmental
Compliance
Hydraulic
Modeling
Scanned
Documents
- 15. 10/11/2016 16
Portal Has Limited Success (2002-2005)
• Infrequent map content updates
• Slow GIS data maintenance process
• Instability of production GIS data files
• Not enough ancillary information linked
• Reports missing key information attributes
• Inconsistent look and feel
- 16. 10/11/2016 17
We Can Do Better
• Fix the production GIS problems
• Develop/connect more data
• Take over development of MapGuide site/ColdFusion reports
- 17. 10/11/2016 18
Back-End GIS Overhaul (2005)
• Replaced production GIS with Intergraph GeoMedia in 2005
• Benefits
• Map data stored in single seamless database (SQL Server) – no more data corruption
• Increased efficiency
• New web+print optimized GIS data model
• Frequent updates to the MapGuide data for GIS Portal
• Opportunity to rewrite GIS portal from scratch to match new data model
- 18. 10/11/2016 19
GIS Portal Rewrite (2005-2006)
• Started rewriting GDI in late 2005
• Switched to Application.cfc
• Main map page with MapGuide interface
• ColdFusion controls behavior of map legend and report linking
• Reports loaded into pop-up child windows of main map page
• Used AJAX to load report content
• JavaScript ties back to parent window
- 20. 10/11/2016 21
ColdFusion Reports
• Report for each primary GIS element:
• Parcels
• Sewer mains
• Sewer structures
• Recycled water mains
• Recycled water valves and fittings
- 21. 10/11/2016 22
Reports Link to More Reports
• Sewer mains/structures
• Maintenance history (Sussex/Accela
CMMS, now on SQL Server)
• CCTV inspection history (WinCan,
several Access databases)
• Hydraulic capacity analysis (Access
database)
• Construction job data and As-Builts
• Recycled Water mains/structures
• Construction job data and As-Builts
• Parcels
• Assessor Maps
• Permit history (Sungard ERP)
• Environmental Compliance inspection
history (Microsoft Access)
- 22. 10/11/2016 23
Search Tools
• Search by various criteria:
• Sewer structures (by common structure IDs)
• Parcels (by address components and/or Assessor Parcel Numbers)
• Infrastructure job (by number or name)
• Map grid
• GIS ID number
• Addresses (full-text search)
- 24. 10/11/2016 25
• IBM iSeries server running
AS/400
• Slow performance over ODBC
• Better performance over JDBC, but
still slow due to missing indexes
• System instability + poor error
handling in our CFML code
• Solution: Cache data on SQL
Server
• SQL Server Integration Service
(SSIS) nightly cache rebuild
• Design indexes to support
ColdFusion reports
Data Integration Challenges – Sungard ERP
- 25. 10/11/2016 26
• Data owned by Collection System
Operations (CSO) department
• Located at separate campus in
Walnut Creek, CA
• Connected by a 10Mb data line
• Originally a 200MB+ Microsoft Access
database
• Copied .MDB file to GDI server
monthly
• Barely adequate performance
• IT migrated Acella to SQL Server
database
• SQL Server physically located at CSO
campus
• Solution: Cache data on our SQL
Server
• Stored procedure to refresh copies
of relevant tables
Data Integration Challenges – Accela CMMS
- 26. 10/11/2016 27
• Inspection data spread across
numerous Microsoft Access
databases
• Inspections for a single pipe could
be in any database
• Pipelines change over time
• Not always a 1:1 match between
inspections and pipes
• CCTV program staff owned the
databases
• Could add new data, correct
existing data at any time
• Solution: Consolidate into single
SQL Server database
• SQL Server Integration Service
(SSIS) nightly rebuild
• Imported records get tagged with
ID of the origin database (DB_ID)
• Build “bridge” table between
inspections and GIS sewer mains
• Maintain “origin” table with DB_ID
and folder paths for location of
associated photos/videos
Data Integration Challenges – WinCan CCTV
- 27. 10/11/2016 28
• Laserfiche web client
unsatisfactory
• Document viewer not designed for
large-format drawings
• UI clumsy, hard to zoom/pan
within documents
• Wanted to continue serving
PDFs to GDI users
• Acrobat reader plugin is familiar,
easy to use
• Solution:
• FOSS GNUWin32 TIFF Editor
library – command line tools
• MERGETIFF.exe combine multiple
TIFFs into single multi-page TIFF
• TIFF2PDF.exe convert TIFF to PDF
• <CFEXECUTE> to call command-
line tools to generate PDFs on-the-
fly
Data Integration Challenges - Laserfiche
- 28. 10/11/2016 29
GDI Usage Takes Off
• Staff adoption factors
• Confidence in reliable access to up-to-date data
• Quick access to the various department data sets
• Savings of 100’s of hours of staff time through:
• Online access to As-Builts
• Research sewer CCTV and maintenance data, parcel data on demand
• Permit counter put in dedicated computers for customer interactions
• Improved customer service
- 30. 10/11/2016 31
ColdFusion Enabled Us To Connect the
“Data Silos”
GIS +
ColdFusion
CMMS
Pipeline
CCTV
Permits &
Parcel
Records
Environmental
Compliance
Hydraulic
Modeling
Scanned
Documents
- 31. 10/11/2016 32
Treatment Plant Needs GIS Too! (2009)
• Success of collection system web GIS inspired trying to do same for our
Treatment Plant
• Built a pilot Facility GIS site in 2009
• Provided map of treatment plant facility
• High resolution flyover imagery
• Linked to treatment plant’s Computerized Maintenance Management System
(Mainsaver)
• Quick access to maintenance history for treatment plant assets
- 32. 10/11/2016 33
GIS Department Uses ColdFusion Internally
• Workflow tracking application started as a Microsoft Access app
• 2005/2006: Hired developer to migrate database to SQL Server, build
ColdFusion front-end application
• Developer ran out of budget before completing scope
• We finished application and then continued further development
- 33. 10/11/2016 34
GIS Department Uses ColdFusion Internally
• Built bespoke document management system for As-Builts
• Later removed in favor of leveraging Laserfiche system
• Automation tools for GeoMedia workflows
• Built scripts for mapbook production
• Synthesized some GIS data
- 35. 10/11/2016 36
Map Change Request Tools
• Writes entry to GIS
Workflow database
• Sends email
confirmation to User
GDI Map Change
Request (MCR) Form
Submitted
• MCR will be
highlighted in
Technician’s GIS
Workflow view
GIS Analyst Assigns or
Closes MCR
• On completion,
notification email
sent to originator
• MCR no longer
highlighted in view
GIS Technician Logs
Completion or Interim
Status
- 36. 10/11/2016 38
Another GIS Platform Shift Needed (2010-
2016)
• GIS Portal running since 2006
• GeoMedia back-end GIS software frozen at the version 5.1 (released 2005)
• Autodesk Mapguide version 6.5 (released 2005, product discontinued in 2006)
• Sussex CMMS (bought up by Accela, discontinued before 2010)
• WinCan CCTV software (discontinued in U.S.)
- 37. 10/11/2016 39
CentralSan Chooses New Platforms (2014)
• Everything to be GIS-centric
• Esri ArcGIS platform to replace desktop GIS and provide map services for web
• 2016 - Latitude Geographics Geocortex to provide web map UI and reports
• Azteca Cityworks to replace both Accela and Mainsaver CMMS systems
• 2016 - Infrastructure Technologies ITpipes to replace WinCan CCTV software
• Will ColdFusion have a part to play?
- 38. 10/11/2016 40
ColdFusion in a Modern Web GIS (2016)
• Complex nested reports beyond capability of Geocortex
• Migrated old GIS Portal reports into new “GeoFusion” site
• Implemented MVC design pattern using Framework/One
• Moved all database and web service interactions into an OOP model
(beans/services)
• Responsive HTML5 design
• Bootstrap 3 JS/CSS library
• Datatables library
• Switched to Laserfiche Weblink interface for scanned documents
- 40. 10/11/2016 42
ColdFusion – Beyond 2016
• Continue to link other departmental “data silos” to GIS
• Rewrite the GIS/CAD workflow tracking system
• Improve UI/UX
• Separate GIS and CAD processes
• Expand scope to track infrastructure jobs “cradle to grave”
- 41. 10/11/2016 43
ColdFusion – Beyond 2016
GIS +
ColdFusion
CMMS
Pipeline
CCTV
Permits &
Parcel
Records
???
Environmental
Compliance
Hydraulic
Modeling
Scanned
Documen
ts
???
- 42. And now for something completely
different…
10/11/2016 44
- 45. 10/11/2016 47
• 1600+ members and growing
• Channels for various CFML-
related topics and frameworks
• Real-time communication
(think IRC+)
• Free to join:
http://cfml-
slack.herokuapp.com/
CFML Slack Team!
- 46. 10/11/2016 48
I need a CFML function/tag syntax
reminder:
• Type a question mark and the name of the
function or tag:
• ?cfquery
• Will return this:
Cool Slack Team Features
- 47. 10/11/2016 49
I need to embed a code snippet or some plaintext
information to get feedback:
• If it’s a single line or less, wrap in single back quotes
• `ValueArray`
• Will end up looking like this:
• If it’s 2 or more lines, wrap in triple back quotes
• ```myStruct = {"num"="123"};
myStruct.setMetadata( { "num": "String" } );
writeOutput(serializeJSON(myStruct));```
• Will give you this:
• If it’s fairly long, put it in a Plain Text snippet, or create a GitHub Gist and paste the link.
Cool Slack Team Features
- 48. 10/11/2016 50
Slack Alternatives
• Adobe forums https://forums.adobe.com/community/coldfusion
• Try to provide a Short, Self-Contained, Correct, Example (SSCCE, see
http://sscce.org/)
• http://stackoverflow.com/ for code issues, or http://serverfault.com/ for
server issues
• Consider a SSCCE a requirement
- 49. 10/11/2016 51
Documentation
• Current ColdFusion docs hard to find on Google
• Community docs at http://cfdocs.org
• Each tag or function has basic syntax info and usage examples
• Links to the corresponding Adobe, Lucee or OpenBlueDragon docs
• Adobe links go the current CF10+ documentation!!!
• Additional community-provided reference information
• Entire site’s source is on GitHub so anyone can contribute
• Fork the project, edit online or locally, submit pull request
• Edit any page online in GitHub, automatically creates a fork and submits a pull request
• Or report an issue and hope someone in the community will fix it
- 50. 10/11/2016 52
Learning & Other Resources
• Learn CF In a Week (http://www.learncfinaweek.com/)
• Free, self-paced tutorial
• Resource links for more documentation and community projects (frameworks, etc.)
• TryCF.com
• Nice playground for testing small amounts of code
• CF 10, 11 & 2016 engines, Lucee 4.5 & 5.0 too
• Can save code snippets to Gists
• Can load code from GitHub Gists or Pastebin
• CommandBox (https://www.ortussolutions.com/products/commandbox)
• ColdFusion package manager (connects to Forgebox, Github, local repositories)
• Can download and spin up multiple versions of ColdFusion and Lucee
• Learn more: Brad Wood’s session tomorrow at 4PM:
• Herding Cats: A new way to manage all your Adobe servers on one dev machine
• What about the ColdFusion Web Application Construction Kit (CFWACK)?
• Honestly, leave it on the shelf
- 51. Thank you!!!
• Questions???
• Contact me:
• Email: cvonstetten@centralsan.org
• Twitter: @cfvonner
• CFML Team on Slack: @cfvonner
• Don’t forget to fill out session evaluation
• You can do it in the Mobile App on iOS and Android!