SlideShare a Scribd company logo
From PilotToProduct
By Ivan Verhun
Notice
• Ask questions during presentation
• Time is limited
• Business domain is not important
Agenda
• Core product
overview
• Pilot architecture
•
• Dev Process
CoreProduct Overview
• B2B
• Import data from clients
• Process data
– Validate
– Filter
– Generate report
Data Flow
SaaSMaturity Models
Main Disadvantages
• Single tenant
• Tightly coupled components –
impossible to reuse
• Business logic in stored procedures
This slide intentionally left blank
Pilot Overview
Pilot Overview
• Target audience - SMB
• Business pilot
• Limited TTL
• Preferably short TTM
• Isolated from the core platform
Individuals
Businesses
Pilot Overview
• Target audience
• Business pilot
• Limited TTL
• Preferably short TTM
• Isolated from the core platform
This slide intentionally left blank
Pilot Architecture
• 1000 meters view
• Technologies and Tools
• Mistakes
Initial Key Decisions
• Multitenant
• Extract data from single tenant DBs
• Functionality spread between modules
• All business logic in Java
– No stored procedures
Multitenant
Initial Key Decisions
• Multitenant
• All business logic in Java
– No stored procedures
• Extract data from single tenant DBs
• Functionality spread between modules
Extract data from single tenant DBs
Modules(Services)
Endpoints
• Asynchronous – AMQP endpoints for
prod, qa, dev
• Synchronous – HTTP endpoints for
IT
• Endpoints switched on/off depending
on spring profile
This slide intentionally left blank
No Downtime Deployment
• Maintain an application mode
– LIVE
– MAINTAINCE
• Application mode is used by LB
• Switch mode to MAINTAINCE before
updating the application
• Switch mode to LIVE after re-
deployment
No Downtime Deployment
This slide intentionally left blank
TechStack– Before
Tools
• Java 6
• Jenkins
• Maven 3
• MySQL 5.1
• Jetty 6
• RabbitMQ
• Git/Github
Libs and Frameworks
• Spring
– IOC
– AMQP
– MVC
– JDBC Template
– Aspects
• JUnit
• slf4j
• Mockito, Powermock
• Joda Time
Database Schema Migrations
• Startup
• Patch Level (patch0001_test_patch.sql)
• Patch Loading
• Patch Ordering
• Patch Execution
• Post-Patch Tasks
Newest DB Schema
• Shell script generating DB dump
• Runs on Jenkins
• Commits newest dump to git repo
• Used to create empty test DBs
ATDD:
http://wakaleo.com/thucydides-sample-reports/
http://thucydides.info/
BDDandJBehave
BDD:
• is an evolution of test-driven development (TDD)
and acceptance-test driven design…
• shifts the vocabulary from being test-based to
behaviour-based
• Was developed as a response to the issues
encountered teaching test-driven development
Author: Dan North
http://en.wikipedia.org/wiki/Behavior-driven_development
JBehave:exampleof story
This slide intentionally left blank
CentralizedLogging
A tool for managing events
and logs. You can use it to
• collect logs
• parse them
• and store them for later
use (like, for searching)
http://logstash.net/
Kibana– ES data visualization engine
PerformanceMonitoring
• Reporting Via JMX, HTTP,
slf4j, CSV, Graphite etc
• The Registry
• Counters
• Meters
• Histograms
• Timer (@Timed)
http://metrics.codahale.com/
Java7
• Java 6 is dead
• Java 8 is baby
This slide intentionally left blank
Persistence
• JDBC was not OK
• JPA/Hibernate was not enough
• MyBatis – good for complex
queries
WebUI
http://www.thymeleaf.org/
TSVGeneration
ReportPrinter
PDF Validation
Impossible to:
• reliably verify absence of an element
• associate present text to particular
report attribute
– there may be false positives
• verify correct order of the text elements
• parse/verify graphs
http://pdfbox.apache.org/
Limitations
Verify:
• number of missing files
• number of unexpected files
• file content
Scope
Web UI (Admin)
https://angularjs.org/
TechStack–After
Initial tool kit
• Java 6
• Jenkins
• Maven 3
• MySQL 5.1
• Jetty 6
• RabbitMQ
Added/updated tools
• Autopatch
• Configuration Service
• JBehave + Thucydides
• Java 7
• Logstash
• Metrics library
• Amazon S3
TechStack–After
Original tech stack
• Spring
– IOC
– AMQP
– MVC
– JDBC Template
– Aspects
• JUnit
• slf4j
• Mockito, Powermock
• Joda Time
Added later
• JPA/Hibernate
• MyBatis
• Freemarker
• Thymeleaf
• Js:
– AngularJS
– RequireJS
– Bower
• Apache PDFBox
Mistakes
• JDBC code is poorly maintainable
• JUnit – @Parameterized tests
From Pilot to Product - Morning@Lohika
DevelopmentProcess
• Methodology
• Techniques and Tools
• Mistakes and Lessons
Scrum Rules
Principle Is Adopted
Product Owner +
Development Team +
Scrum Master +/-
Sprint planning meeting +
Daily Scrum meeting +
Sprint Review +
Sprint Retrospective +
Product backlog +
Sprint backlog +
Burndown chart -
SprintStructure
Planningprocess
• Preplanning (30-60 min)
– high level plans and priorities
• Grooming (up to 3 hours)
– discussion
– estimation
– Poker
– Story points (0-8)
• Planning (30-60 min)
– final estimation
– defining sprint scope (commitment)
Scrum Board
Task Workflow
• Analysis and writing AC
• AC Review
• Coding
– Unit tests
– Functionality
– Integration tests
– Writing resolution comments
• Code review
• Verification on QA env
Daily Scrum
Daily Scrum
Daily Scrum
Daily scrum
AdvertisingUkraine
Demo
Retrospective
• What is working
• What’s not working
• To improve next
time
CodeReviewTypes
Ad hoc
Passaround
Pair programming
Walkthrough
Team review
Inspection
Code ReviewTools
– Github
• Pros
– inline comments
– no extra software required
• Cons
– emails per comment (it's even not configurable)
– inline diffs, no side-by-side comparison
– small diff context
– directory structure is not visible
– Side-by-side comparison (Beyond compare)
• lines comparison in the bottom (above status bar)
• compare different files
• compare with clipboard content
– Reviewer checklist
BeyondCompare
BeyondCompare
ReviewChecklist
• The Obvious
– Code compiles, it’s
clear, test cases
etc.
• Design
• Coding Standards
• Error Handling
• Security
• Thread Safety
• Control Structures
• Functions
• Performance
• Bug Fixes
• Math
• Maintainability
• Documentation
• Requirements Tracing
This slide intentionally left blank
PeopleManagement
• Let people choose the task they like (if
possible)
• Avoid knowledge islands
• Let people be initiative
• Minimize micromanagement
• Semi-periodic face to face meetings
Peoplemanagement
• Dealing with highly
motivated/enthusiastic people
• Trainings
– Agile (scrum)
– Code review
– Thucydides
– Virtualization
– TDD and Refactoring
• Dinners
TeamBuildings
Thursdays Dinner
Cross functionalteam
TeamComposition
Ichak Adizes:
• Producers
• Administrators
• Entrepreneurs
• Integrators
This slide intentionally left blank
Mistakes
From Pilot to Product - Morning@Lohika
WeDon’t NeedTesters
WeManagedto AvoidThis
Code Review(ers)
• Start with seniors only
• Establish code review process before
extending reviewers list
• Extend reviewers list gradually
• “Code Reviewer” title should be
honorable
Learnings
• Testers are essential
• Sometimes micromanagement is a
must
• Establish code reviews right
This slide intentionally left blank
CustomerManagement
Why Customer NeedReportsTrust
LowHigh
Low High
Transparency
This slide intentionally left blank
From PilotToProduct
• Successful production launch
• Unexpectedly big demand
• Extended contract
• Rewriting on core platform
– New set of exciting technologies
Resources
SaaS Maturity Model:
• http://www.avuetech.com/wp-content/uploads/sites/7/2012/03/SaaS-Maturity-Levels.pdf
• http://blogs.msdn.com/b/architectsrule/archive/2008/08/18/saas-maturity-model-
according-to-forrester.aspx
• http://www.infoq.com/news/2008/02/saas-architecture-maturity-model
Code review:
• http://en.wikipedia.org/wiki/Code_review#Types
• http://www.softwaretestinggenius.com/istqb-foundation-level-exam-crash-course-part-6
• https://www.owasp.org/images/2/2e/OWASP_Code_Review_Guide-V1_1.pdf
• http://smartbear.com/SmartBear/media/pdfs/WP-CC-11-Best-Practices-of-Peer-Code-
Review.pdf
Cross functional team
• http://en.wikipedia.org/wiki/Cross-functional_team
Customer management
• http://habrahabr.ru/company/stratoplan/blog/202028/
Picture Sources
• http://en.wikipedia.org/wiki/File:American_truck.JPG
• http://www.expresstaxiinc.us/
• http://www.dreamstime.com/
• http://www.durhamcharterservices.com/
• http://www.galingas.com.ua/
• http://provse.te.ua/
• http://www.huffingtonpost.com/
• https://en.wikipedia.org/wiki/Scrum_(software_development)#mediaviewer/File:Scrum_process.svg
From Pilot to Product - Morning@Lohika

More Related Content

From Pilot to Product - Morning@Lohika

Editor's Notes

  1. Explain the SMB term Explain difference between Individual and SMB products Gas stations
  2. Gas station example Tips – pick up few passengers
  3. Similar businesses Business segment Schedule Length of the route Tips – pick up few passengers – doesn’t work
  4. Explain “Business pilot” term Reasons for isolation Level of isolation
  5. Change slide template
  6. SMB services: web app running on Jetty. Every service contains “similar” functionality There are 2 instances of a service Started with smb-dataimport and smb-reportgeneration services
  7. Multitenancy with multiple app instances (SaaS Level 4) Pros Support costs Ability to trigger a job for multiple clients in a single click Cons Integration with lot’s of single-tenant data sources Data isolation between clients
  8. Rule engine Separate business logic from system logic Created custom RE not based on JSR94 (JavaTM Rule Engine API) Used in Eligibility Pre-report checks Profile completeness %
  9. Connects to multiple Legacy DBs for data extraction Just a restart is needed to add new DB (no build required) Used org.springframework…AbstractRoutingDataSource Abstract {@link javax.sql.DataSource} implementation that routes {@link #getConnection()} calls to one of various target DataSources based on a lookup key. Created custom BeanFactoryPostProcessor to dynamically determine list of DBs
  10. Explain purpose of every file TSV file generation: Third parties data providers TSV Report data file Freearker vs Velocity vs Tiles
  11. Verify Seed files How PDFs are validated? Limitations Generally impossible to verify absence of an element There may be false positives – PDF may contain a text (e.g. month name) in some place, but this is not the place we expect Impossible to associate present text to particular report attribute Impossible to verify correct order of the text elements Impossible to parse/verify graphs PDF generator for integration testing No need to wait until printer provides generated seed reports
  12. What is Angular Client side MVC framework How it works reads in HTML that contains additional custom tag attributes; it then obeys the directives in those custom attributes, and binds input or output parts of the page to a model represented by standard JavaScript variables Angular teaches the browser new syntax through a construct we call directives. Examples include: Data binding, as in {{}}. DOM control structures for repeating/hiding DOM fragments. Support for forms and form validation. Attaching code-behind to DOM elements. Grouping of HTML into reusable components. Why Angular
  13. Tell why we have chosen every technology AMQP load balancing Consuming Stargate messages Spring MVC: to make it easier to trigger a process, we used HTTP endpoints to send an AMQP message later, we added a “synchronous” implementation of the HTTP endpoints for IT JDBC Initially used everywhere Now, used only in “Reading” part of legacy data extraction Aspects Refresh properties Tracking job start/end time, input parameters and status
  14. SQLs testing Dev Tools testing
  15. Few sentences about process: Scrum etc.
  16. Scrum master – removing impediments ensures that the Scrum process is used as intended.
  17. How branching is dictated by code review? Why do we use forks Types of code review
  18. https://wiki/display/PD/Code+Review+Checklist
  19. Team lead is between two fires – customers on one side and motivated people on the other Goal: ensure the customer is happy and people do what they like to do.
  20. Team lead is between two fires – customers on one side and motivated people on the other Goal: ensure the customer is happy and people do what they like to do.
  21. Dinners
  22. A cross-functional team is a group of people with different functional expertise working toward a common goal
  23. Іцхак Калдерон Адізес — один з провідних світових експертів в області підвищення ефективності ведення бізнесу і урядової діяльності шляхом внесення кардинальних змін, які не породжують хаосу і деструктивних конфліктів, що перешкоджають змінам. producers - those who work, like developers administrators - those who introduce processes entrepreneurs - new ideas, lets change everything - disruptions integrators - communicate, social glue no 4 qualities can be present, you should find others jobs - entrepreneur,
  24. We don’t need testers Sometimes micromanagement is a must
  25. Only senior and motivated person should review the code at first stage Establish code review process before extending reviewers list Extend reviewers list gradually “Code reviewer” title should be honorable