This presentation was run as a Light Reading Webinar. Describes what DevOps means for network engineers and service providers.
Report
Share
Report
Share
1 of 30
More Related Content
DevOps for Network Engineers
1. Stefan Vallin, Ph. D.
Principal Engineer
29 June 2016
Use Things You Can Program,
and Program the Things You Use
DevOps for
Networking
2. Walls of Confusion
New and Changed
Requirements
Users
Product Mgr
New/Changed App Code
Configuration Change
Application
Developers
Network
Engineers
Operations
3. What is DevOps?
http://dev.mobify.com/blog/devops-101-best-practices/
“The cool-kids way of
stringing stuff together with
shell scripts.
It exists because we can
now wrap things in shell
scripts that we used to only
dream of; the world is now
programmable at a much
larger scale, and we have
many new tools and
techniques for taking
advantage of it.”
Takes a new or
enhanced feature all
the way to production—
everyone is adding
value to the customer
A method that stresses
communication, collaboration,
integration, automation,
and cooperation across
organizational units to
deliver features quickly
and efficiently.
Based on lean and
agile principles
DevOps is the marrying
of process, infrastructure,
and product
New and Changed
Requirements
Customers
Users
New/ChangedApp Code
Configuration Change
Application
Developers
Network
Engineers
Operations
4. Why Should You Care?
Better Customer Experience
Faster Better
Increased capacity to innovate
Let a small team multiply
their effectiveness
Faster time to value
Fewer failures
Recover faster from failures
More predictable behavior
5. Why Should You Care?
We want
more change!
Dev
“
”
We want
less change!
Ops
“
”
You can have both!
6. Network Automation
Historically DevOps
Tools: GIT, Python, NETCONF
• An IT/OSS project
• Waterfall
• Few network domain-experts
• Driven from the network
• Agile
• Designed and implemented by network
engineers with a DevOps mentality
8. • Sharing
• Collaboration
• Learning
• Understanding other roles
• Understanding business
• No blame-culture
People, Process, Culture
Source: IBM: “DevOps for Dummies”
People, Process
Culture
9. Across Roles
Product
Owners
DevOps
Team
AT&T Domain 2.0:Product Owners
Business
People
People, Process
Culture
Everyone need to change-
-not just network engineers
- Meet in same practices
- Meet in same tools
- Learn and educate
- More hands-on
10. • Hypothesis-driven
• Slices that can be implemented in a
week
• Teams have a good understanding
of the flow of work from the
business all the way through to
customers
• Visibility
• Regularly seek customer feedback,
and incorporate this feedback into
the design of their products.
Lean Product Management
Source: Puppe Labs: “2016 State of DevOps Report”
11. DevOps Networking Skills
Culture
Networking
Software
Engineering
Education ?
“ There remains much to do before this
vision can be implemented, including
pivots from networking craft to software
engineering, and from carrier operations
models to cloud “DevOps” models. ”
AT&T Domain 2.0:
People, Process
Culture
12. • An increased knowledge of other IT disciplines
• More knowledge of the business
• More understanding of applications
• More emphasis on programming
Changes in Network
Professionals’ Skills
Stallings: Foundations of Modern Networking: SDN, NFV, QoE, IoT, and Cloud
People, Process
Culture
14. • Infrastructure as code
• Automation
• Deploy with repeatable, reliable processes
• Self-service environments
• Automated environment for de-provisioning
• Automated recovery roll-back and roll-forward
• Hypothesis-driven development
• Dev and ops team shall use same tools
• Develop and test against production-like systems
• Amplify feedback loops
• Continuous integration
• Automated testing
• Continuous deployment
• Availability monitoring
• Monitor and validate operational quality
• Release management
• App performance monitoring
• Load-testing and auto-scale
Practices
15. Before After
Before and After
Tools: Ticketing, Excel, Device CLI Tools: GIT, Python, NETCONF
• Operator role picks up ticket
in ITSM tool
• Implements the configuration
change manually or pseudo-
automatically
Cut and paste snippets
• Closes ticket
• Users provision their network
services directly
• DevOps role implements automation
• What do I do manually?
• How can I automate?
• How can I validate it?
• How do I handle failures?
16. So DevOps is just scripting?
More than a shell script in:
~/my-cool-automation-scripts
Distribution, Execution, Monitoring, Error-Recovery, Logging,
Validation, Versioning, Feature Management, No-Hands
17. Open Source, Open APIs, OpenFlow,
Open vSwitch, OpenDaylight, OpenConfig…
• Things can be said about all of the above,
but clear values coming out of it
• Standard APIs like NETCONF
• Scripting on the devices
• Linux-based devices, the shell prompt
• Networking devices are now open for
programming
Enabler: The Open Networking Trend
~ Use Things You
Can Program,
and Program the
Things You Use ~
Stallings: Foundations of Modern Networking: SDN, NFV, QoE, IoT, and Cloud
19. Separation of Concerns, Change life-cycles
Run-time fine-grained configuration
changes for individual services
• Modify QoS policy
• Add e-mail user
• Modify firewall rule
• Add backend to load-balancer
Network
Services
VNFs/VMs
Applications
Platforms
Backbone /
DC Network
Short
Lifecycle
Long
Lifecycle
Do Not
Blur/Mix
Make resources available to
provide services
• Install
• Start
• Stop
• Upgrade
• Provision L2 network in data center
connecting VNFs
20. Nature of Configuration Changes
• Static “Golden Configs”
• Configuration
templates with variable
substitution
• Configuration
templates with loops,
if-statements
• Scripted configuration
based on stateServices
VNFs/VMs
Applications
Platforms
Backbone /
DC Network
Short
Lifecycle
Long
Lifecycle
Do Not
Blur/Mix
Different requirements for tools
21. Comparison: Changes in Different Domains
Examples Changes Change Rate
Applications Latest code-base for web-shop In-house developed code.
Software changes
Daily
IT-Platforms New patch on WordPress Server
New Database Server
External Software releases and
patches.
Weekly
mysql config file Config file contents
Network Functions New VPN
Add leg to VPN
Add new VAS to VPN
Change QoS params for VPN
Configuration changes Thousands
per day
New device OS version New and upgraded devices Quarterly
27. • Winner in Service Agility:
• Business Innovation and Product Owners in close co-op with DevOps team
• Data-Models and Running Code
• Sign of lagging behind
• Organization controlled by
“Architects in PowerPoint”
• OSS-projects
• Internal “purchase-orders”
Summary
Start creating your DevOps culture now
28. Material and Reading
• John Willis, ”The Convergence of DevOps”
• http://itrevolution.com/the-convergence-of-devops/
• Kyle Young, Mobify: “DevOps 101: Best Practices for Optimizing and Automating
Your Infrastructure"
• http://dev.mobify.com/blog/devops-101-best-practices/
• David Tesar, “DevOps Practices”
• http://www.itproguy.com/devops-practices/
• Adrian Cockcroft: “Ops, DevOps and PaaS (NoOps) at Netflix”
• http://perfcap.blogspot.com/2012/03/ops-devops-and-noops-at-netflix.html
• Ashton, Metzler & Associates: “The Changing Role of the IT & Network Professional”
• http://www.ashtonmetzler.com/Quali%20Fifth%20Paper%20V2.0.pdf
• Lori MacVittie:”Network Engineers: Don’t Fear the Code”
• http://www.networkcomputing.com/careers/network-engineers-dont-fear-code/1347793692
29. Material and Reading
• IBM: “DevOps for Dummies”
• https://www.ibm.com/marketing/iwm/iwm/web/signup.do?source=swg-rtl-sd-wp&S_PKG=ov18162
• Barry O’Reilly: “How to Implement Hypothesis-Driven Development”
• https://www.thoughtworks.com/insights/blog/how-implement-hypothesis-driven-development
• Julien Stroheker: “DevOps: Where do I start ? Cheat Sheet”
• https://blogs.technet.microsoft.com/juliens/2016/02/14/devops-where-do-i-start-cheat-sheet/
• Stefan Vallin and Caroline Chappel, Light Reading Upskill: “Virtualization, Automation”
• http://www.lightreading.com/virtualization-automation/l/d-id/722320
• Stallings: Foundations of Modern Networking: SDN, NFV, QoE, IoT, and Cloud
• Puppet, “State of DevOps Report”
• https://puppet.com/resources/white-paper/2016-state-of-devops-report
Editor's Notes
There is a constant feed of new and changed requirements from customers and users.
Normally the developers and network engineers are not in close contact with the end-users.
In contrast they get new requirements thrown over the wall without interaction.
In turn:
- the developers develop new apps to fulfill the requirements and throw over the wall to operations team.
- network engineers manually reconfigures the network and throw that over the wall to the operations team.
These two walls create:
- confusion: from requirements to implementation, from reconfigured network to operational issues
- time lag: the hand-over process is time-consuming
It is very important to distinguish different lifecycles.
Two important layers are illustrated in the figure.
BOTTOM: The long-lived lifecycle with running VMs/VNFs and
TOP: the short-lived providing end-user services: telco:services, IT:applications
Walk through the example bullets.
Talk bottom-up
Design separately, maybe different tools (more on this soon).
BUT, they act together in a client-server relationship.
Needs clear API linkage between the layers.
The top-layer requests resources from the bottom layer.
The top-layer does run-time config of the bottom layer.
The bottom layer notifies the top layer on changes. The top layer need to adopt.
It is very important to distinguish different lifecycles.
Two important layers are illustrated in the figure.
BOTTOM: The long-lived lifecycle with running VMs/VNFs and
TOP: the short-lived providing end-user services: telco:services, IT:applications
Walk through the example bullets.
Talk bottom-up
Design separately, maybe different tools (more on this soon).
BUT, they act together in a client-server relationship.
Needs clear API linkage between the layers.
The top-layer requests resources from the bottom layer.
The top-layer does run-time config of the bottom layer.
The bottom layer notifies the top layer on changes. The top layer need to adopt.
Look at the tools to the right. There are more well-known tools for the bottom layer.
This also means that there is a risk that these tools are applied in the top-layer since they are known.
Top-layer: APIs and fine-grained operations
Bottom-layer: more templating
Also, the “services” to use an over-loaded word in the top-layer varies much more than in the bottom-layer.
A business VPN is very different between service providers, whereas starting a MYSQL server is more main-stream.
Tools in the top-layer also depends heavily on if you are an IT Application-centric shop or a network service provider.
For the first category the tools in the bottom layer might be ok.
If you are selling applications, the tools for long-lived life-cycles are appropriate also in the top-layer.
The networking can be abstracted in the bottom layer.
VIMs like OpenStack will hide/hard-code networking.
Same with the tuning for the VMs, less effort in tuning the VIM for different VMs.
I am not saying that IT is easy. Main challenge:
Deployment of new application code several times per day.
Needs extreme control on DevOps process for changing the application code itself.
If you have internal control this is of course easier.
External dev team, puts extreme focus on regression tests and deployment process.
For a network centric offering:
Only the underlay network is performed in the bottom layer
The top layer does all overlay networking
The network layer can not be abstracted by for example OpenStack and VMWare or in the automation tools like Juju, HEAT,…
More effort in tuning the virtual infrastructure VIMs for packet processing applications
In the previous slide we talked about application centric offerings. There we said that a major challenge is that the application code
might change several times per day.
For a network centric offering it is the configuration that changes extremely frequently, thousands per day.
the application code is changed with very low frequency.
Since the top-layer is about high-frequency fine-grained configuration changes for varying offerings we need:
Explicit data-models for the offerings
APIs that support fine-grained operations
Process:
Learning
Sharing
Collaboration
Understanding business, other roles
Practices:
Automation
Lean and agile
Infrastructure as code
Self-Service
Automated rollback and recovery
Tools:
Share tools across organisation
Configuration changes of different nature, different tools
Pick-up tools from distributed programming, tools act upon text, not vendor blobs