From Pilot to Product - Morning@Lohika
- 10. Pilot Overview
• Target audience - SMB
• Business pilot
• Limited TTL
• Preferably short TTM
• Isolated from the core platform
- 13. Pilot Overview
• Target audience
• Business pilot
• Limited TTL
• Preferably short TTM
• Isolated from the core platform
- 16. Initial Key Decisions
• Multitenant
• Extract data from single tenant DBs
• Functionality spread between modules
• All business logic in Java
– No stored procedures
- 18. Initial Key Decisions
• Multitenant
• All business logic in Java
– No stored procedures
• Extract data from single tenant DBs
• Functionality spread between modules
- 21. Endpoints
• Asynchronous – AMQP endpoints for
prod, qa, dev
• Synchronous – HTTP endpoints for
IT
• Endpoints switched on/off depending
on spring profile
- 23. No Downtime Deployment
• Maintain an application mode
– LIVE
– MAINTAINCE
• Application mode is used by LB
• Switch mode to MAINTAINCE before
updating the application
• Switch mode to LIVE after re-
deployment
- 26. TechStack– Before
Tools
• Java 6
• Jenkins
• Maven 3
• MySQL 5.1
• Jetty 6
• RabbitMQ
• Git/Github
Libs and Frameworks
• Spring
– IOC
– AMQP
– MVC
– JDBC Template
– Aspects
• JUnit
• slf4j
• Mockito, Powermock
• Joda Time
- 27. Database Schema Migrations
• Startup
• Patch Level (patch0001_test_patch.sql)
• Patch Loading
• Patch Ordering
• Patch Execution
• Post-Patch Tasks
- 28. Newest DB Schema
• Shell script generating DB dump
• Runs on Jenkins
• Commits newest dump to git repo
• Used to create empty test DBs
- 30. BDDandJBehave
BDD:
• is an evolution of test-driven development (TDD)
and acceptance-test driven design…
• shifts the vocabulary from being test-based to
behaviour-based
• Was developed as a response to the issues
encountered teaching test-driven development
Author: Dan North
http://en.wikipedia.org/wiki/Behavior-driven_development
- 33. CentralizedLogging
A tool for managing events
and logs. You can use it to
• collect logs
• parse them
• and store them for later
use (like, for searching)
http://logstash.net/
- 42. PDF Validation
Impossible to:
• reliably verify absence of an element
• associate present text to particular
report attribute
– there may be false positives
• verify correct order of the text elements
• parse/verify graphs
http://pdfbox.apache.org/
Limitations
Verify:
• number of missing files
• number of unexpected files
• file content
Scope
- 44. TechStack–After
Initial tool kit
• Java 6
• Jenkins
• Maven 3
• MySQL 5.1
• Jetty 6
• RabbitMQ
Added/updated tools
• Autopatch
• Configuration Service
• JBehave + Thucydides
• Java 7
• Logstash
• Metrics library
• Amazon S3
- 45. TechStack–After
Original tech stack
• Spring
– IOC
– AMQP
– MVC
– JDBC Template
– Aspects
• JUnit
• slf4j
• Mockito, Powermock
• Joda Time
Added later
• JPA/Hibernate
• MyBatis
• Freemarker
• Thymeleaf
• Js:
– AngularJS
– RequireJS
– Bower
• Apache PDFBox
- 49. Scrum Rules
Principle Is Adopted
Product Owner +
Development Team +
Scrum Master +/-
Sprint planning meeting +
Daily Scrum meeting +
Sprint Review +
Sprint Retrospective +
Product backlog +
Sprint backlog +
Burndown chart -
- 51. Planningprocess
• Preplanning (30-60 min)
– high level plans and priorities
• Grooming (up to 3 hours)
– discussion
– estimation
– Poker
– Story points (0-8)
• Planning (30-60 min)
– final estimation
– defining sprint scope (commitment)
- 53. Task Workflow
• Analysis and writing AC
• AC Review
• Coding
– Unit tests
– Functionality
– Integration tests
– Writing resolution comments
• Code review
• Verification on QA env
- 62. Code ReviewTools
– Github
• Pros
– inline comments
– no extra software required
• Cons
– emails per comment (it's even not configurable)
– inline diffs, no side-by-side comparison
– small diff context
– directory structure is not visible
– Side-by-side comparison (Beyond compare)
• lines comparison in the bottom (above status bar)
• compare different files
• compare with clipboard content
– Reviewer checklist
- 65. ReviewChecklist
• The Obvious
– Code compiles, it’s
clear, test cases
etc.
• Design
• Coding Standards
• Error Handling
• Security
• Thread Safety
• Control Structures
• Functions
• Performance
• Bug Fixes
• Math
• Maintainability
• Documentation
• Requirements Tracing
- 67. PeopleManagement
• Let people choose the task they like (if
possible)
• Avoid knowledge islands
• Let people be initiative
• Minimize micromanagement
• Semi-periodic face to face meetings
- 68. Peoplemanagement
• Dealing with highly
motivated/enthusiastic people
• Trainings
– Agile (scrum)
– Code review
– Thucydides
– Virtualization
– TDD and Refactoring
• Dinners
- 78. Code Review(ers)
• Start with seniors only
• Establish code review process before
extending reviewers list
• Extend reviewers list gradually
• “Code Reviewer” title should be
honorable
- 84. From PilotToProduct
• Successful production launch
• Unexpectedly big demand
• Extended contract
• Rewriting on core platform
– New set of exciting technologies
- 85. Resources
SaaS Maturity Model:
• http://www.avuetech.com/wp-content/uploads/sites/7/2012/03/SaaS-Maturity-Levels.pdf
• http://blogs.msdn.com/b/architectsrule/archive/2008/08/18/saas-maturity-model-
according-to-forrester.aspx
• http://www.infoq.com/news/2008/02/saas-architecture-maturity-model
Code review:
• http://en.wikipedia.org/wiki/Code_review#Types
• http://www.softwaretestinggenius.com/istqb-foundation-level-exam-crash-course-part-6
• https://www.owasp.org/images/2/2e/OWASP_Code_Review_Guide-V1_1.pdf
• http://smartbear.com/SmartBear/media/pdfs/WP-CC-11-Best-Practices-of-Peer-Code-
Review.pdf
Cross functional team
• http://en.wikipedia.org/wiki/Cross-functional_team
Customer management
• http://habrahabr.ru/company/stratoplan/blog/202028/
Editor's Notes
- Explain the SMB term
Explain difference between Individual and SMB products
Gas stations
- Gas station example
Tips – pick up few passengers
- Similar businesses
Business segment
Schedule
Length of the route
Tips – pick up few passengers – doesn’t work
- Explain “Business pilot” term
Reasons for isolation
Level of isolation
- Change slide template
- SMB services:
web app running on Jetty.
Every service contains “similar” functionality
There are 2 instances of a service
Started with smb-dataimport and smb-reportgeneration services
- Multitenancy with multiple app instances (SaaS Level 4)
Pros
Support costs
Ability to trigger a job for multiple clients in a single click
Cons
Integration with lot’s of single-tenant data sources
Data isolation between clients
- Rule engine
Separate business logic from system logic
Created custom RE not based on JSR94 (JavaTM Rule Engine API)
Used in
Eligibility
Pre-report checks
Profile completeness %
- Connects to multiple Legacy DBs for data extraction
Just a restart is needed to add new DB (no build required)
Used org.springframework…AbstractRoutingDataSource
Abstract {@link javax.sql.DataSource} implementation that routes {@link #getConnection()} calls to one of various target DataSources based on a lookup key.
Created custom BeanFactoryPostProcessor to dynamically determine list of DBs
- Explain purpose of every file
TSV file generation:
Third parties data providers
TSV Report data file
Freearker vs Velocity vs Tiles
- Verify
Seed files
How PDFs are validated?
Limitations
Generally impossible to verify absence of an element
There may be false positives – PDF may contain a text (e.g. month name) in some place, but this is not the place we expect
Impossible to associate present text to particular report attribute
Impossible to verify correct order of the text elements
Impossible to parse/verify graphs
PDF generator for integration testing
No need to wait until printer provides generated seed reports
- What is Angular
Client side MVC framework
How it works
reads in HTML that contains additional custom tag attributes;
it then obeys the directives in those custom attributes, and binds input or output parts of the page to a model represented by standard JavaScript variables
Angular teaches the browser new syntax through a construct we call directives. Examples include:
Data binding, as in {{}}.
DOM control structures for repeating/hiding DOM fragments.
Support for forms and form validation.
Attaching code-behind to DOM elements.
Grouping of HTML into reusable components.
Why Angular
- Tell why we have chosen every technology
AMQP
load balancing
Consuming Stargate messages
Spring MVC:
to make it easier to trigger a process, we used HTTP endpoints to send an AMQP message
later, we added a “synchronous” implementation of the HTTP endpoints for IT
JDBC
Initially used everywhere
Now, used only in “Reading” part of legacy data extraction
Aspects
Refresh properties
Tracking job start/end time, input parameters and status
- SQLs testing
Dev Tools testing
- Few sentences about process:
Scrum etc.
- Scrum master –
removing impediments
ensures that the Scrum process is used as intended.
- How branching is dictated by code review?
Why do we use forks
Types of code review
- https://wiki/display/PD/Code+Review+Checklist
- Team lead is between two fires – customers on one side and motivated people on the other
Goal: ensure the customer is happy and people do what they like to do.
- Team lead is between two fires – customers on one side and motivated people on the other
Goal: ensure the customer is happy and people do what they like to do.
- Dinners
- A cross-functional team is a group of people with different functional expertise working toward a common goal
- Іцхак Калдерон Адізес — один з провідних світових експертів в області підвищення ефективності ведення бізнесу і урядової діяльності шляхом внесення кардинальних змін, які не породжують хаосу і деструктивних конфліктів, що перешкоджають змінам.
producers - those who work, like developers
administrators - those who introduce processes
entrepreneurs - new ideas, lets change everything - disruptions
integrators - communicate, social glue
no 4 qualities can be present, you should find others
jobs - entrepreneur,
- We don’t need testers
Sometimes micromanagement is a must
- Only senior and motivated person should review the code at first stage
Establish code review process before extending reviewers list
Extend reviewers list gradually
“Code reviewer” title should be honorable