Third Party Security Testing for Advanced Metering Infrastructure Program
In July 2010, BC Hydro, the electric utility and grid operator of British Columbia began implementation of its AMI program, formally known as the Smart Meter & Infrastructure (SMI) program. The SMI program transformed BC Hydro from a traditional metering utility to a smart metering utility by implementing smart meters on the customer service points. It was the first step in the smart grid transformation. The SMI program required the introduction of many new devices and applications into BC Hydro’s infrastructure. Some of these had never been deployed before anywhere in the world. Many were field deployed, outside of BC Hydro’s physical security perimeter. The SMI Security Delivery Team was formed to deliver on these commitments and to take responsibility for the end to end security of the SMI program. The Team implemented a multi-pronged approach to securing SMI including security risk assessments, security penetration testing by the team, design reviews, whole project risk assessments and third party security penetration testing. A standards based approach was required to ground the test plan both in best practice and in a common set of principles that BC Hydro and its vendors could accept. The Advanced Metering Infrastructure (AMI) Risk Assessment document prepared by the Advanced Metering Infrastructure Security (AMI-SEC) Task Force was used as a basis for the test plan. This document has since been passed to the National Institute of Standards and Technology (NIST) Cyber Security Working Group and was integrated into NIST IR 7628. NIST IR 7628 contains a comprehensive list of possible threats to AMI systems. The program was highly successful. Test results informed BC Hydro’s deployment decisions and allowed the manufacturers to improve their products. Lessons were learned about how best to conduct third party security testing. A full lessons learned section is included in the presentation.
Third Party Security Testing for Advanced Metering Infrastructure Program
- 1. Third Party Security Testing for Advanced Metering Infrastructure Projects Steve Vandenberg and Robert Hawk
- 2. Disclaimer Please note that the observations and commentary presented here are based on industry experience and do not represent information or positions specific to any project, utility, its vendors or partners.
- 3. BC Hydro’s AMI Deployment • BC is larger than CA, OR and WA put together • BC Hydro SMI Project - CAD $2 Billion • 1.8 million meters • Thousands of Field Area Routers (FAR) • Itron Meters, Cisco Network (IPv6) • Started deployment 2011 • Meter to Cash, Energy & Theft Analytics • Southwest Research Institute (SwRI) chosen as third party test lab through RFP
- 4. Example AMI Hacks • Looking into the eye of the meter, (Optical Port Hack) Don C. Weber, InGuardians, Defcon, 2012 • Hacking For Privacy Discovery 28C3, (RF Hack) Dario Carluccio and Stephan Brinkhaus, 28th Chaos Communication Congress 2011 • Puerto Rico Utility electricity theft via Smart Meter, (Optical Port Hack) US FBI / Krebs on Security, 12 April 2009 • We are so screwed, (Connected Grid Router Hack) Larry Pesce, SANS, Defcon 21 Comedy Jam, 2013
- 5. AMI Projects vs IT Projects IT • 3 to 5 Year Lifespan • 1 to 3 Fiscal Quarters for Deployment • 10’s of servers • 100’s of network devices • Thousands of end nodes AMI • 50+ Year Lifespan • 2 to 5 Year Deployment • 100’s of servers • Thousands of network devices • Millions of end nodes
- 6. Security Test Scope • Home Area Network (HAN) interface • Smart Meter ZigBee network • Smart Meters • Optical Port Interface of the Smart Meter • RFLAN Range Extenders • RFLAN Smart Meter mesh network • Field Area Routers (FAR) • Smart Meter Data Collection Systems • Software Releases (Versions & Patches)
- 7. Security Test Plans • Common set of principles the Utility, vendors and service providers would accept • Standards based approach: AMI-Sec & NIST IR 7628 • Standards were developed using risk assessment methodology with interfaces and controls being the primary focal points • A standards based risk rating modified by use case for each test case allowed prioritization of security testing Note: Be prepared to change test plans and priorities during test cycle in response to results
- 8. Develop Test Plans Beforehand • Create use case documents to shape the test plan • Do a security assessment of the product to shape the test plan • Tie the test plan back to a recognized standard e.g. NIST • Consider all threat vectors – cyber, physical, social
- 9. Threat Vectors • Physical – Magnetic attack, Meter / Router Tampering • Cyber – RF Denial of Service, Optical Probe Hack, Zigbee or DNP3 Hack • Social – Current meter status info or improper disconnect from call center • Combination – HAN device association to target home/business, SD Card Hack Note: Turn off the power, mess with billing, know when people are home and what they’re doing…
- 10. Before Starting… • Get results of security testing carried out by the manufacturer or others • Eliminate redundant, expensive testing • Shape the test plans • Pick up where other testing left off • Further explore their findings
- 11. Ensure Production-Like Environment • You are spending a lot of money on these results, be sure they are valid • Production-like equipment is not enough, Production-like configuration is needed • Document any differences between environments and discuss with the vendor and implementation team how these differences will impact results • Who will build the test environment? Who will troubleshoot it? Who will maintain it?
- 12. Support Structure for Security Testing • Third party testing needs substantial support from the manufacturer • Test environment construction and maintenance • Test environment troubleshooting • Defect triage – vetting of false positives • Defect resolution • Information for white box testing • Engage the manufacturer and test lab beforehand to form a common understanding • Make this part of the contract
- 13. Handling the Results • Confidentiality vs Availability – results are there to be used • How will the information be handled? • Document Control Systems… • Encryption? Password Management ? Document Repositories? • Who will see it and how? • Know that you are responsible for the information. Information leakage could damage: • The public, The utility, The manufacturer, Other users of the product
- 14. Using the Results • How will defects be mitigated? • What conditions if any would be applicable for deployment? • What findings would change the use case? • What will the criteria be for halting deployment ?
- 15. Setting an Achievable Goal • What does done look like? • All test findings will point to need for more testing… • Where to stop?
- 16. Handling Transition to Production • After deployment is over, patching and new versions of applications and devices will occur • What if some defects survive the project? How to manage them to closure? Retesting challenges. What evidence is acceptable for closure? • Will the third party test program continue? • Maintaining and upgrading the security test environment • Location of the test environment?
- 17. Questions ? Steve Vandenberg http://www.linkedin.com/in/vandenbergsteve Robert Hawk http://www.linkedin.com/in/ironmanrbh