SlideShare a Scribd company logo

Large Scale Drupal - Behind the Scenes

•Download as PPTX, PDF•
•
Boyan Borisov
Boyan Borisov

This document discusses the large-scale Drupal implementation for the Municipality of Copenhagen. It describes the client's requirements including handling thousands of websites on a multisite platform, scalability, reusability, content sharing between sites, and accessibility. It then explains the technical solutions used such as Aegir for the multisite platform, Gitlab for module development, and various Drupal modules for features like content sharing, editing interface, and performance optimization. Continuous integration, testing, and contributions back to the open source community are also covered.

1 of 23
Large Scale Drupal in the Municipality of Copenhagen - Behind the Scenes Drupal Camp Berlin 2014
About me • Boyan Borisov • Open Source Tech Lead and Architect @ Propeople • boyanb@propeople.dk • @boyan_borisov • Skype: boian.borisov • https://drupal.org/u/bo yan.borisov
What were the client’s requirements? • Multisite platform able to handle a thousand web sites • Scalable infrastructure able to handle a thousand sites available 24/7 • Multi vendor environment which allows every module to be reusable • Content sharing between all the sites on all the profiles • Centralized Digital Asset Management (DAM) system
What were the client’s requirements? • Editorial friendly UI • Hierarchy structure of the site pages • Web Content Accessibility Guidelines (WCAG) – WCAG 2.0 AA • Performance • Automated tests • Continuous integration
Multisite platform able to handle a thousand web sites One Aegir to rule them all! • What is Aegir? • Why did we choose it? • What did we do in order to extend Aegir?
Scalable infrastructure able to handle a thousand sites available 24/7
Monitoring and logging • Graphite - Scalable Realtime Graphing • Elasticsearch - http://www.elasticsearch.org • Logstash - http://logstash.net • JSONlog - https://www.drupal.org/project/jsonlog • Elasticsearch Connector - https://www.drupal.org/project/elasticsearch_con nector
Multi vendor environment which allows every module to be reusable • Gitlab - https://about.gitlab.com • Mini “drupal.org” for all municipality custom modules • Every module into its own repository • Drush make file to build a platform • Reusable drush make files • Phing build tool • Jira
Reusable make files core = 7.x api = 2 ; KKMS inheritance includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/kkms_profile/raw/7.x- 1.26/release.make includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/ kkdk_profile/raw/dev/contrib.make" includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/ kkdk_profile/raw/dev/features.make" includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/ kkdk_profile/raw/dev/custom.make"
Content sharing between all the sites on all the profiles • Push content to a site(s) • Push content to a channel • Pull content from a site • Pull content from a channel • Centralized Digital Asset Management (DAM) system • Central content management system
Content sharing architecture
Centralized Digital Asset Management (DAM) system
Central Localization Distribution (CLD) Site1 Site2 SiteN CLD
Editorial friendly UI • Field group • Media and Media browser plus • Admin views and Views bulk operations • Workbench • Entity reference and References dialog • Panelizer • Node clone • Ckeditor and Better formats • Chosen, Multiselect and Hierarchical Select • Taxonomy manager
Hierarchy structure of the site pages • Be aware not to use a module which will make a hierarchy on top of the menu_router table! • Solution is Menu Reference - https://www.drupal.org/project/menu_refe rence
The German trace
Web Content Accessibility Guidelines (WCAG) – WCAG 2.0 AA ● Drupal and accessibility https://groups.drupal.org/node/18595 https://www.drupal.org/project/node_accessi bility ● Test tools http://siteimprove.com/ http://wave.webaim.org/ Accessibility Developer Tools - Chrome plugin
Performance • Varnish with a custom vcl file • Memcache • Apache Solr views • APC cache • Horizontal and vertical scaling
Automated tests
Continuous integration • Continuous integration with Aegir • Gitlab + webhooks + merge request • Jira + Gitlab integration • Phing build tool for local development • Release branches • Jenkins • Fabric
The best part! What did we give back to the community?
Contributions ● 90+ patches ● 300+ comments ● 7 contributed modules
Questions?

More Related Content

Large Scale Drupal - Behind the Scenes

  • 1. Large Scale Drupal in the Municipality of Copenhagen - Behind the Scenes Drupal Camp Berlin 2014
  • 2. About me • Boyan Borisov • Open Source Tech Lead and Architect @ Propeople • boyanb@propeople.dk • @boyan_borisov • Skype: boian.borisov • https://drupal.org/u/bo yan.borisov
  • 3. What were the client’s requirements? • Multisite platform able to handle a thousand web sites • Scalable infrastructure able to handle a thousand sites available 24/7 • Multi vendor environment which allows every module to be reusable • Content sharing between all the sites on all the profiles • Centralized Digital Asset Management (DAM) system
  • 4. What were the client’s requirements? • Editorial friendly UI • Hierarchy structure of the site pages • Web Content Accessibility Guidelines (WCAG) – WCAG 2.0 AA • Performance • Automated tests • Continuous integration
  • 5. Multisite platform able to handle a thousand web sites One Aegir to rule them all! • What is Aegir? • Why did we choose it? • What did we do in order to extend Aegir?
  • 6. Scalable infrastructure able to handle a thousand sites available 24/7
  • 7. Monitoring and logging • Graphite - Scalable Realtime Graphing • Elasticsearch - http://www.elasticsearch.org • Logstash - http://logstash.net • JSONlog - https://www.drupal.org/project/jsonlog • Elasticsearch Connector - https://www.drupal.org/project/elasticsearch_con nector
  • 8. Multi vendor environment which allows every module to be reusable • Gitlab - https://about.gitlab.com • Mini “drupal.org” for all municipality custom modules • Every module into its own repository • Drush make file to build a platform • Reusable drush make files • Phing build tool • Jira
  • 9. Reusable make files core = 7.x api = 2 ; KKMS inheritance includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/kkms_profile/raw/7.x- 1.26/release.make includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/ kkdk_profile/raw/dev/contrib.make" includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/ kkdk_profile/raw/dev/features.make" includes[] = "https://kkgit.kk.dk/drupal-7-kkms-modules/ kkdk_profile/raw/dev/custom.make"
  • 10. Content sharing between all the sites on all the profiles • Push content to a site(s) • Push content to a channel • Pull content from a site • Pull content from a channel • Centralized Digital Asset Management (DAM) system • Central content management system
  • 12. Centralized Digital Asset Management (DAM) system
  • 13. Central Localization Distribution (CLD) Site1 Site2 SiteN CLD
  • 14. Editorial friendly UI • Field group • Media and Media browser plus • Admin views and Views bulk operations • Workbench • Entity reference and References dialog • Panelizer • Node clone • Ckeditor and Better formats • Chosen, Multiselect and Hierarchical Select • Taxonomy manager
  • 15. Hierarchy structure of the site pages • Be aware not to use a module which will make a hierarchy on top of the menu_router table! • Solution is Menu Reference - https://www.drupal.org/project/menu_refe rence
  • 17. Web Content Accessibility Guidelines (WCAG) – WCAG 2.0 AA ● Drupal and accessibility https://groups.drupal.org/node/18595 https://www.drupal.org/project/node_accessi bility ● Test tools http://siteimprove.com/ http://wave.webaim.org/ Accessibility Developer Tools - Chrome plugin
  • 18. Performance • Varnish with a custom vcl file • Memcache • Apache Solr views • APC cache • Horizontal and vertical scaling
  • 20. Continuous integration • Continuous integration with Aegir • Gitlab + webhooks + merge request • Jira + Gitlab integration • Phing build tool for local development • Release branches • Jenkins • Fabric
  • 21. The best part! What did we give back to the community?
  • 22. Contributions ● 90+ patches ● 300+ comments ● 7 contributed modules