SlideShare a Scribd company logo

Ci tips and_tricks_linards_liepins

•
•
Linards Liep
Linards Liep

Covering topics like: CI CD DevOps Jenkins TFS TeamCity Compile Test Package Delpoy See Disclaimer in the last slide and/or in file comments, if available.

Related slideshows

Agile Testing
Agile Testing Agile Testing
Agile Testing
 
Neal Ford Emergent Design And Evolutionary Architecture
Neal Ford Emergent Design And Evolutionary ArchitectureNeal Ford Emergent Design And Evolutionary Architecture
Neal Ford Emergent Design And Evolutionary Architecture
 
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
Devops Strategy Roadmap Lifecycle Ppt Powerpoint Presentation Slides Complete...
 
1 of 21
Download to read offline
Continuous Integration “Make Love, Not War” Linards Liepiņš SIA LATAKKO 2014
Flavors and Concepts  To Build or Not to Build?  To Test or Not to Test?  To Analyze or Not to Analyze?  To Deploy or Not To Deploy?
Advantages and Disadvantages  Advantages: When unit tests fail or a bug emerges, developers might revert the codebase to a bug-free state, without wasting time debugging Developers detect and fix integration problems continuously — avoiding last-minute chaos at release dates, (when everyone tries to check in their slightly incompatible versions). Early warning of broken/incompatible code Early warning of conflicting changes Immediate unit testing of all changes Constant availability of a "current" build for testing, demo, or release purposes Immediate feedback to developers on the quality, functionality, or system-wide impact of code they are writing Frequent code check-in pushes developers to create modular, less complex code[citation needed] Metrics generated from automated testing and CI (such as metrics for code coverage, code complexity, and features complete) focus developers on developing functional, quality code, and help develop momentum in a team[citation needed] Disadvantages Initial setup time required Well-developed test-suite required to achieve automated testing advantages Source: http://en.wikipedia.org/wiki/Continuous_integration#Advantages_and_disadvantages
Implementations and Realizations  Open / Free:  Jenkins  BuildHive ( Cloud-based )   Closed / Trials; Freemium; Subscriptions, etc.  Team Fundation Server  Hudson  TeamCity +
Representation(s) and Accessibility ● Cloud → Web-browser → IDEs ● Though it means bunch of VM, you still will have best margin of wysiwyg from IDE ● Server → Web-browser → CLI ● Though it means bunch of OS-specific coding, you still do not want surprises from custom regexp :) ● Workstation→ Data Transmission Protocols → JSON ; REST(ful) ; XML ; etc. ● It depends from knowledge of developer itself. It is possible to launch most actions in CI software, like Jenkins, from any objects above. ● Exclusion For TFS: using Team Foundation Server OData API
Examples of Web GUI Jenkins Static Code Analysis
Examples of IDE GUI Team Foundation Server 2010  Something similar to PivotTables...
Problem(s) – Everyone wants to be happy ● ● Define in most possible detail what your CI solution is for. For example, You cannot argue with colleges that they cannot access their required data because they are lazy or not willing to spend time searching for particular stuff. You must guide them so the search feels natural and comfortable – but only transparent guidance is process-based. Set up reoccurring events or meet-ups with different colleges of what their experiences was lately with CI solution. If CI solution is closed source, ensure you advise users do Feedback. Users must feel they are backed up and can consult and do not get in frustration of the chaos they might think CI is. Programmer or Developer will force on fast and straight result; SysAdmin will ask to provide small and compact packaging; Solution / System Architect will always have its own legit view. Distribution / Deployment will want for tested & verified data.
Problem(s) – Executable is high-priority element in the build-process Define relation(s) between what you see and what you get out of something CI solutions invoked build-process utilizes. For example, avoid thinking only in one-way pattern that call to executable “x” only references to actual task or sub-task. Each executable is just one element of whole process and all it does it transforms data. Once you parallelize two or more of them, you must acknowledge you might start to complicate things. Maybe your CI solution is just too powerful or too extensible, causing confusion?
Problem(s) – Too much elements, too few layers, zero understanding Each new introduction of new “tool” is a new challenge of evaluation of sustainability of your CI setup as whole. Group all tools you use ( executables, at this point ), define their tasks and expected outcome, and group them, making layers. For example, if you use subversion and git in single (build)system, label it as “svn”. Each group might intersect some of its behavior by others. Do not expect it will not happen; instead, ensure accessibility of debug logging assets and monitor their usage. You might be surprised what you will see there, especially if you present results to colleges.
Solutions(s) – Visualize! Nothing frustrates DevOps more than (build)system users doing or even performing changes that do not follow strictly defined description of process. Once you introduce something new, redraw this image and see, how big difference from default you are having now.
Solution(s) – Different README for everyone Best way to ensure all the parties will understand they are dealing with is not to write it on your own. Company ecosystem is very dynamic place. Let the core tools provide help content, because they ignore subjective view of any involved person in such README content.
Solution(s) Capability Maturity Model Integration ( CMMI ) Auditing your (build)system using CMMI can also be used to discover issues between different layers of your CI solution / (build)system. Level 1-4 can be fully put side-by-side with main CI tasks: Compile; Tests; Package; Deploy. Each of tasks may contain multiple layers.
Case-study: Jack-of-all-trades / Master Exploder? Single server → specified IDEs → specified language pre-pocessing capabilities → specified post-processing capabilities → limited Quality-Assurance support → usability-driven CI server support and maintenance plan → Knowledge-base!!! Single server → unspecified IDEs → unspecified language pre-processing capabilities → unspecified language post-processing capabilities → unlimited QA support → capacity-driven CI server support and maintenance plan → CMMI & Knowledge-base!!! Team City Server Container $ .NET Ant, Rake, ( scripti Maven, Foxcop ng ) Gradle, , CMD IDEA Subversio n, CVS, Git, Mercurial, AccuRev SCM, Clearcase, Perforce, SourceSaf e, StarTeam, TFS, Vault E-mail, XMPP, RSS, IDE, SysTray Eclipse, VS, IntelliJ IDEA, RubyMine, PyCharm, PhpStorm, WebStorm → Jetbrains Youtrack, JIRA, Bugzilla, FishEye, FindBugs, PMD, dotCover, NCover TFS Win, VSTM $ .NET ( IDE ) TFS E-mail, SOAP VS, Eclipse Ant, Maven Scripting, CMD
Case-study: Make CI work for you, not You for CI ● ● ● Any stack-trace or debug info you see in Web-browser representation of CI server is your friend. Once user sees it, always advice to register it in internal CI bugtracking software. Any more detailed log than usual shows both your professionalism and actually represents you do care about future. ● OS: ${JENKINS_HOME}*.log Any non-build reports of statistics or actions performed is extremely valuable if suddenly CI server fails and gets unreachable. ● JVM: -Xdebug
Experience(s) – It's all about echo... ● Jenkins / Hudson – it should be all about command line. Once you press “Build” button, each external tool is new layer between build-process and build-result. :) ● ● Supports Puppet both ways! TeamCity – it should be all about task-to-task, where developer do not directly interfere with build-scripting :) ● ● Puppet supports TeamCity :) Team Foundation Server – it all should be about Windows ecosystem supported products. You got good salary - “You do it right” ; You got Microsoft Logo poster in your bedroom - “You do it wrong”. ● Ignores Puppet ; Got VS Release management / Team System.
Experience(s) – Half planned CI is dangerous ● Plan fully all the data flow ( in blocks ) from the initiation of build-process to the moment of build-result. If you do not determine and cannot characterize in full detail each and every I/O operation your CI solution performs, you might not understand the purpose of that examined process. Block schemes is your friend, when used with, for example, Jenkins. ● On other hand, TFS has limited extensibility, which leads to more determined outcome.
Experience(s) – Do not let become wiki dumpsite ( “build monkey” )! ● Working in Company that both is Software Developer and Software System Maintainer forces individuals to start to become something like dumpsite of all wiki-fiable information about Companies IT Ecosystem. If it is happening, it is first signal that Layers either are too much intersecting, either there is critically insufficient HR and documentation of build-process as whole. It means you gave to get back to block-scheme ; internal auditing and CMMI. ● On other hand, TFS, as strictly defined environment, provides more tighten standardization
Experience(s) – Hardware-oriented tasks are overhead It is not very rare that you might be forced to integrate some “cleanup” behavior for some cases, especially if you use Compilation, which intends to create garbage in your physical data storage. Once situation is acknowledged, it is necessary to optimize data flow and data life-cycle.
Summary ● ● ● Extensibility starts from Shell / Command line – if CI solution you use starts from here, you MUST support full and SEPERATED Product Release and Infrastructure Maintenance teams. ● Jenkins Flexibility starts from build-scripts, for example, MAKE, ANT, NANT, PHING, etc. If you you go from here, you MUST ensure process describes both its own capabilities and Human utilizing this (build)system. ● Jenkins ; TeamCity Reliability / Stability starts from language CI has been built-on. If your CI documentation states it is, for example, Java-based, then you MUST keep full Performance and Reliability logging in all levels of (build)system in the way any system crash is fully transparent to users. Fear not – weekly metrics with JRockit rock! ● Jenkins, TeamCity, Team Fundation Server
Thank you!  Resources: http://kief.com/ http://linardsliepins.wordpress.com/ http://continuousdelivery.com/          Disclaimer: Some portions of images is not property of author and are used without any copyright infringements intended to their respective owners. Author keeps full rights to textual content only.   References:    See QR code. Questions?

More Related Content

Ci tips and_tricks_linards_liepins

  • 1. Continuous Integration “Make Love, Not War” Linards Liepiņš SIA LATAKKO 2014
  • 2. Flavors and Concepts  To Build or Not to Build?  To Test or Not to Test?  To Analyze or Not to Analyze?  To Deploy or Not To Deploy?
  • 3. Advantages and Disadvantages  Advantages: When unit tests fail or a bug emerges, developers might revert the codebase to a bug-free state, without wasting time debugging Developers detect and fix integration problems continuously — avoiding last-minute chaos at release dates, (when everyone tries to check in their slightly incompatible versions). Early warning of broken/incompatible code Early warning of conflicting changes Immediate unit testing of all changes Constant availability of a "current" build for testing, demo, or release purposes Immediate feedback to developers on the quality, functionality, or system-wide impact of code they are writing Frequent code check-in pushes developers to create modular, less complex code[citation needed] Metrics generated from automated testing and CI (such as metrics for code coverage, code complexity, and features complete) focus developers on developing functional, quality code, and help develop momentum in a team[citation needed] Disadvantages Initial setup time required Well-developed test-suite required to achieve automated testing advantages Source: http://en.wikipedia.org/wiki/Continuous_integration#Advantages_and_disadvantages
  • 4. Implementations and Realizations  Open / Free:  Jenkins  BuildHive ( Cloud-based )   Closed / Trials; Freemium; Subscriptions, etc.  Team Fundation Server  Hudson  TeamCity +
  • 5. Representation(s) and Accessibility ● Cloud → Web-browser → IDEs ● Though it means bunch of VM, you still will have best margin of wysiwyg from IDE ● Server → Web-browser → CLI ● Though it means bunch of OS-specific coding, you still do not want surprises from custom regexp :) ● Workstation→ Data Transmission Protocols → JSON ; REST(ful) ; XML ; etc. ● It depends from knowledge of developer itself. It is possible to launch most actions in CI software, like Jenkins, from any objects above. ● Exclusion For TFS: using Team Foundation Server OData API
  • 6. Examples of Web GUI Jenkins Static Code Analysis
  • 7. Examples of IDE GUI Team Foundation Server 2010  Something similar to PivotTables...
  • 8. Problem(s) – Everyone wants to be happy ● ● Define in most possible detail what your CI solution is for. For example, You cannot argue with colleges that they cannot access their required data because they are lazy or not willing to spend time searching for particular stuff. You must guide them so the search feels natural and comfortable – but only transparent guidance is process-based. Set up reoccurring events or meet-ups with different colleges of what their experiences was lately with CI solution. If CI solution is closed source, ensure you advise users do Feedback. Users must feel they are backed up and can consult and do not get in frustration of the chaos they might think CI is. Programmer or Developer will force on fast and straight result; SysAdmin will ask to provide small and compact packaging; Solution / System Architect will always have its own legit view. Distribution / Deployment will want for tested & verified data.
  • 9. Problem(s) – Executable is high-priority element in the build-process Define relation(s) between what you see and what you get out of something CI solutions invoked build-process utilizes. For example, avoid thinking only in one-way pattern that call to executable “x” only references to actual task or sub-task. Each executable is just one element of whole process and all it does it transforms data. Once you parallelize two or more of them, you must acknowledge you might start to complicate things. Maybe your CI solution is just too powerful or too extensible, causing confusion?
  • 10. Problem(s) – Too much elements, too few layers, zero understanding Each new introduction of new “tool” is a new challenge of evaluation of sustainability of your CI setup as whole. Group all tools you use ( executables, at this point ), define their tasks and expected outcome, and group them, making layers. For example, if you use subversion and git in single (build)system, label it as “svn”. Each group might intersect some of its behavior by others. Do not expect it will not happen; instead, ensure accessibility of debug logging assets and monitor their usage. You might be surprised what you will see there, especially if you present results to colleges.
  • 11. Solutions(s) – Visualize! Nothing frustrates DevOps more than (build)system users doing or even performing changes that do not follow strictly defined description of process. Once you introduce something new, redraw this image and see, how big difference from default you are having now.
  • 12. Solution(s) – Different README for everyone Best way to ensure all the parties will understand they are dealing with is not to write it on your own. Company ecosystem is very dynamic place. Let the core tools provide help content, because they ignore subjective view of any involved person in such README content.
  • 13. Solution(s) Capability Maturity Model Integration ( CMMI ) Auditing your (build)system using CMMI can also be used to discover issues between different layers of your CI solution / (build)system. Level 1-4 can be fully put side-by-side with main CI tasks: Compile; Tests; Package; Deploy. Each of tasks may contain multiple layers.
  • 14. Case-study: Jack-of-all-trades / Master Exploder? Single server → specified IDEs → specified language pre-pocessing capabilities → specified post-processing capabilities → limited Quality-Assurance support → usability-driven CI server support and maintenance plan → Knowledge-base!!! Single server → unspecified IDEs → unspecified language pre-processing capabilities → unspecified language post-processing capabilities → unlimited QA support → capacity-driven CI server support and maintenance plan → CMMI & Knowledge-base!!! Team City Server Container $ .NET Ant, Rake, ( scripti Maven, Foxcop ng ) Gradle, , CMD IDEA Subversio n, CVS, Git, Mercurial, AccuRev SCM, Clearcase, Perforce, SourceSaf e, StarTeam, TFS, Vault E-mail, XMPP, RSS, IDE, SysTray Eclipse, VS, IntelliJ IDEA, RubyMine, PyCharm, PhpStorm, WebStorm → Jetbrains Youtrack, JIRA, Bugzilla, FishEye, FindBugs, PMD, dotCover, NCover TFS Win, VSTM $ .NET ( IDE ) TFS E-mail, SOAP VS, Eclipse Ant, Maven Scripting, CMD
  • 15. Case-study: Make CI work for you, not You for CI ● ● ● Any stack-trace or debug info you see in Web-browser representation of CI server is your friend. Once user sees it, always advice to register it in internal CI bugtracking software. Any more detailed log than usual shows both your professionalism and actually represents you do care about future. ● OS: ${JENKINS_HOME}*.log Any non-build reports of statistics or actions performed is extremely valuable if suddenly CI server fails and gets unreachable. ● JVM: -Xdebug
  • 16. Experience(s) – It's all about echo... ● Jenkins / Hudson – it should be all about command line. Once you press “Build” button, each external tool is new layer between build-process and build-result. :) ● ● Supports Puppet both ways! TeamCity – it should be all about task-to-task, where developer do not directly interfere with build-scripting :) ● ● Puppet supports TeamCity :) Team Foundation Server – it all should be about Windows ecosystem supported products. You got good salary - “You do it right” ; You got Microsoft Logo poster in your bedroom - “You do it wrong”. ● Ignores Puppet ; Got VS Release management / Team System.
  • 17. Experience(s) – Half planned CI is dangerous ● Plan fully all the data flow ( in blocks ) from the initiation of build-process to the moment of build-result. If you do not determine and cannot characterize in full detail each and every I/O operation your CI solution performs, you might not understand the purpose of that examined process. Block schemes is your friend, when used with, for example, Jenkins. ● On other hand, TFS has limited extensibility, which leads to more determined outcome.
  • 18. Experience(s) – Do not let become wiki dumpsite ( “build monkey” )! ● Working in Company that both is Software Developer and Software System Maintainer forces individuals to start to become something like dumpsite of all wiki-fiable information about Companies IT Ecosystem. If it is happening, it is first signal that Layers either are too much intersecting, either there is critically insufficient HR and documentation of build-process as whole. It means you gave to get back to block-scheme ; internal auditing and CMMI. ● On other hand, TFS, as strictly defined environment, provides more tighten standardization
  • 19. Experience(s) – Hardware-oriented tasks are overhead It is not very rare that you might be forced to integrate some “cleanup” behavior for some cases, especially if you use Compilation, which intends to create garbage in your physical data storage. Once situation is acknowledged, it is necessary to optimize data flow and data life-cycle.
  • 20. Summary ● ● ● Extensibility starts from Shell / Command line – if CI solution you use starts from here, you MUST support full and SEPERATED Product Release and Infrastructure Maintenance teams. ● Jenkins Flexibility starts from build-scripts, for example, MAKE, ANT, NANT, PHING, etc. If you you go from here, you MUST ensure process describes both its own capabilities and Human utilizing this (build)system. ● Jenkins ; TeamCity Reliability / Stability starts from language CI has been built-on. If your CI documentation states it is, for example, Java-based, then you MUST keep full Performance and Reliability logging in all levels of (build)system in the way any system crash is fully transparent to users. Fear not – weekly metrics with JRockit rock! ● Jenkins, TeamCity, Team Fundation Server
  • 21. Thank you!  Resources: http://kief.com/ http://linardsliepins.wordpress.com/ http://continuousdelivery.com/          Disclaimer: Some portions of images is not property of author and are used without any copyright infringements intended to their respective owners. Author keeps full rights to textual content only.   References:    See QR code. Questions?