SlideShare a Scribd company logo

SaaS Testing on an Agile World

•
•
PractiTest
PractiTest

SoftTest Webinar Presentation: SaaS Testing on an Agile World (April 2010), with preview of PractiTest - test management solution in the beginning.

Related slideshows

1° Sessione - Tecnologie hyperconvergenti e di virtualizzazione storage: VMwa...
1° Sessione - Tecnologie hyperconvergenti e di virtualizzazione storage: VMwa...1° Sessione - Tecnologie hyperconvergenti e di virtualizzazione storage: VMwa...
1° Sessione - Tecnologie hyperconvergenti e di virtualizzazione storage: VMwa...
 
Hpe service virtualization 3.8 what's new chicago adm
Hpe service virtualization 3.8 what's new chicago admHpe service virtualization 3.8 what's new chicago adm
Hpe service virtualization 3.8 what's new chicago adm
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computing
 
1 of 22
Download to read offline
SaaS Testing on an Agile World
About me - Joel Montvelisky PractiTest – Co Founder & Prod. Architect QA Instructor & Consultant Mercury Interactive - QA Manager (retired ) TD, QC, WR, QTP, etc ITCB (IL) Advisory Board QABlog.practitest.com © 2010 -
Today’s Agenda   SaaS - Software as a Service   Things we Can Stop Testing   Non-trivial Stuff to Think About & TEST   Leveraging Live Feedback into the Testing Lifecycle   Open Season for Questions… © 2010 -
What is SaaS? © 2010 -
Software as a Service   Hosted Platforms – Total Control – No Maintenance   No Client Installations   Customers buys each month – keep them happy  Users don’t care it is SaaS, it needs to be better than the alternative © 2010 -
SaaS QA Management   End-to-End QA Management (Reqs – Tests – Bugs)   Enterprise Level SaaS Solution - Methodology - Customizations & Flexibility   Global platform - Communication - Languages - Available 24x7 Worldwide   Intuitive & Simple   Agile Dev, RoR, on Amazon EC2 © 2010 -
What does SaaS mean to the Testing Process? © 2010 -
Things we can STOP Testing ✘  Client or Server installations ✘  Multiplatform backend support ✘  Multiple version upgrades “Savings” of approx. 1/3 ✘  Backwards compatibility of the Traditional Enterprise Application Testing Cycle © 2010 -
The regular testing scenarios   Functionality - Manual scripts - Exploratory sessions Agile Sprints & Updates - Checklists of 3 – 4 weeks - End-2-End Scenarios - Selenium Each sprint’s cycle is - Cucumber & RSpec based on its content   Load & Stress   Multi-platform & Multi-browser support   I18N English, Swedish, German, Chinese Simplified, Hebrew © 2010 -
Non-trivial stuff to think about & TEST 1. Remote accessibility & usage   Staging Servers in USA - Amazon EC2 Development & Testing teams in Israel (& WW collaborators)   Globally distributed monitoring services (www.pingdom.com) © 2010 -
Non-trivial stuff to think about & TEST 2. Security Application-related: Infrastructure-based:   Cross-site scripting   Secured communication   SQL injections   Backups & storage policies   HTTP header injections   Controlled Access to the site & servers etc… © 2010 -
Non-trivial stuff to think about & TEST 3. Live updates & deployments (3 to 4 weeks)   No concrete separation between Dev & IT   Minimal service disruption   Dressed rehearsals (including rollbacks!)   Automated sanity & manual verification   Patches & hot-fixes (extraordinary occasions )   Prove of Concepts & Beta Releases QA & Testing as a facilitator for Balance: Stable & Professional, yet Flexible Environment © 2010 -
Non-trivial stuff to think about & TEST 4. Disaster recovery procedures Two Main scenarios: (a) System down to be brought up quickly   Configure machines   Install & deploy software   Restore data Define a policy & schedule regular drills to ensure you can achieve it! (b) Rollback to last known stable data (with or without data restore) © 2010 -
Leveraging Real Data into the Testing Process © 2010 -
Levering live feedback into the Testing Process (1) Internally developed monitoring tools - Check usage patterns No need to - Validate need & usability assumptions continue guessing!!! - Analyze patterns and create realistic & prioritized testing scenarios © 2010 -
Levering live feedback into the Testing Process (2) Site Monitoring systems   Real-time usage to monitor traffic   Allow users to contact the company with questions or issues Pingdom (www.pingdom.com)   Up to date response times for real projects and data.   Proactive alerts when things start slowing down © 2010 -
Levering live feedback into the Testing Process (3) Proactive mail notifications on issues (www.hoptoad.com)   Real-time notifications on issues on the system.   Information about the issue including system parameters that allow you to do an initial trouble-shoot of the issue © 2010 -
Levering live feedback into the Testing Process (4) Ask users for Real Projects   No better way to test than using real data   Most users will trust you not to abuse their information in exchange for assurance of working software   Since the data sits in your system it is easier to copy projects to your testing environment  Need to be careful with who has access to the data; making sure your testing environment is as secured as your production environment © 2010 -
Wrapping up… © 2010 -
Wrapping up   SaaS is only the delivery model, customers will not compromise on Functionality, Stability or Response Time   Since the system is hosted you can save all tests related to backend installation and support   You need to pay more attention to stuff like accessibility, security, deployments & DRPs   The lines between the R&D and IT organizations in SaaS are blurry   SaaS application allows better access to real data to be leveraged to improve the effectiveness of our testing operations. © 2010 -
Open Season for questions! © 2010 -
Special thanks to Anne-Marie Charrett  joel@practitest.com http://qablog.practitest.com © 2010 -

More Related Content

SaaS Testing on an Agile World

  • 2. About me - Joel Montvelisky PractiTest – Co Founder & Prod. Architect QA Instructor & Consultant Mercury Interactive - QA Manager (retired ) TD, QC, WR, QTP, etc ITCB (IL) Advisory Board QABlog.practitest.com © 2010 -
  • 3. Today’s Agenda   SaaS - Software as a Service   Things we Can Stop Testing   Non-trivial Stuff to Think About & TEST   Leveraging Live Feedback into the Testing Lifecycle   Open Season for Questions… © 2010 -
  • 4. What is SaaS? © 2010 -
  • 5. Software as a Service   Hosted Platforms – Total Control – No Maintenance   No Client Installations   Customers buys each month – keep them happy  Users don’t care it is SaaS, it needs to be better than the alternative © 2010 -
  • 6. SaaS QA Management   End-to-End QA Management (Reqs – Tests – Bugs)   Enterprise Level SaaS Solution - Methodology - Customizations & Flexibility   Global platform - Communication - Languages - Available 24x7 Worldwide   Intuitive & Simple   Agile Dev, RoR, on Amazon EC2 © 2010 -
  • 7. What does SaaS mean to the Testing Process? © 2010 -
  • 8. Things we can STOP Testing ✘  Client or Server installations ✘  Multiplatform backend support ✘  Multiple version upgrades “Savings” of approx. 1/3 ✘  Backwards compatibility of the Traditional Enterprise Application Testing Cycle © 2010 -
  • 9. The regular testing scenarios   Functionality - Manual scripts - Exploratory sessions Agile Sprints & Updates - Checklists of 3 – 4 weeks - End-2-End Scenarios - Selenium Each sprint’s cycle is - Cucumber & RSpec based on its content   Load & Stress   Multi-platform & Multi-browser support   I18N English, Swedish, German, Chinese Simplified, Hebrew © 2010 -
  • 10. Non-trivial stuff to think about & TEST 1. Remote accessibility & usage   Staging Servers in USA - Amazon EC2 Development & Testing teams in Israel (& WW collaborators)   Globally distributed monitoring services (www.pingdom.com) © 2010 -
  • 11. Non-trivial stuff to think about & TEST 2. Security Application-related: Infrastructure-based:   Cross-site scripting   Secured communication   SQL injections   Backups & storage policies   HTTP header injections   Controlled Access to the site & servers etc… © 2010 -
  • 12. Non-trivial stuff to think about & TEST 3. Live updates & deployments (3 to 4 weeks)   No concrete separation between Dev & IT   Minimal service disruption   Dressed rehearsals (including rollbacks!)   Automated sanity & manual verification   Patches & hot-fixes (extraordinary occasions )   Prove of Concepts & Beta Releases QA & Testing as a facilitator for Balance: Stable & Professional, yet Flexible Environment © 2010 -
  • 13. Non-trivial stuff to think about & TEST 4. Disaster recovery procedures Two Main scenarios: (a) System down to be brought up quickly   Configure machines   Install & deploy software   Restore data Define a policy & schedule regular drills to ensure you can achieve it! (b) Rollback to last known stable data (with or without data restore) © 2010 -
  • 14. Leveraging Real Data into the Testing Process © 2010 -
  • 15. Levering live feedback into the Testing Process (1) Internally developed monitoring tools - Check usage patterns No need to - Validate need & usability assumptions continue guessing!!! - Analyze patterns and create realistic & prioritized testing scenarios © 2010 -
  • 16. Levering live feedback into the Testing Process (2) Site Monitoring systems   Real-time usage to monitor traffic   Allow users to contact the company with questions or issues Pingdom (www.pingdom.com)   Up to date response times for real projects and data.   Proactive alerts when things start slowing down © 2010 -
  • 17. Levering live feedback into the Testing Process (3) Proactive mail notifications on issues (www.hoptoad.com)   Real-time notifications on issues on the system.   Information about the issue including system parameters that allow you to do an initial trouble-shoot of the issue © 2010 -
  • 18. Levering live feedback into the Testing Process (4) Ask users for Real Projects   No better way to test than using real data   Most users will trust you not to abuse their information in exchange for assurance of working software   Since the data sits in your system it is easier to copy projects to your testing environment  Need to be careful with who has access to the data; making sure your testing environment is as secured as your production environment © 2010 -
  • 19. Wrapping up… © 2010 -
  • 20. Wrapping up   SaaS is only the delivery model, customers will not compromise on Functionality, Stability or Response Time   Since the system is hosted you can save all tests related to backend installation and support   You need to pay more attention to stuff like accessibility, security, deployments & DRPs   The lines between the R&D and IT organizations in SaaS are blurry   SaaS application allows better access to real data to be leveraged to improve the effectiveness of our testing operations. © 2010 -
  • 22. Special thanks to Anne-Marie Charrett  joel@practitest.com http://qablog.practitest.com © 2010 -