SlideShare a Scribd company logo

Content delivery Plone Symposium East 2010

•
•
alan runyan
alan runyan

ContentMirror allows Plone content to be delivered out-of-band by synchronizing content from the Plone database into a relational database management system (RDBMS). This decouples content delivery from Plone, allowing other applications and frameworks to serve the content dynamically or statically. ContentMirror provides a simple, extensible, and fast way to mirror Plone content and has been used successfully in several large projects. While atypical for Plone, out-of-band content delivery can help overcome constraints for content management when organizational structures or technical requirements differ from Plone's default approach.

Related slideshows

JNUC 2017: Open Distribution Server
JNUC 2017: Open Distribution ServerJNUC 2017: Open Distribution Server
JNUC 2017: Open Distribution Server
 
Succeding with the Apache SOA stack
Succeding with the Apache SOA stackSucceding with the Apache SOA stack
Succeding with the Apache SOA stack
 
ScalabilityAvailability
ScalabilityAvailabilityScalabilityAvailability
ScalabilityAvailability
 
1 of 35
Alt. Content DeliveryAlan RunyanPlone Symposium EastState College, Pennsylvania5/29/09
OverviewReview modes of content deliveryWhy and what purpose Existing solutions for PloneStrengths and WeaknessContentMirror in detailConclusion
What is itTwo modes in-band & out-of-bandPlone Default is “in band”Req to Plone; Plone computes Respe.g. XDV, mod_wsgi, mod_proxy, varnishAtypical usage of Plone, “out of band”Plone emits content into intermediary storagePlone is not involved in responding to ReqNo other FOSS CMS has such feature
How Others Do ItRender pages to filesystem + syncCustom programming to iterate over repository rendering views to diskProvide an API into repository for web-tier application to source contentPromote the database schema as API
Out of BandStatic DeliveryGenerates static assets from PloneAssets transferred to web serverStatically rendered content servedDynamic DeliveryPlone pushes content into 3rd party data storeSeparate web application to display content
Why?Static DeliveryConstraints in delivery mode of websiteSecurity requires no moving partse.g. Kiosk, updated every X hoursDynamic DeliveryWeb application is vastly different than CMSDifferent organizational / skill-sets being usede.g. existing website is .NET; want new section to be driven by Plone
Current OptionsStatic DeliveryCMFDeploymentenpraxis.staticsiteDynamic DeliveryEntransitContentMirror
Plone ApproachRender pages to filesystem + syncCMFDeployment, enpraxis.staticsitePromote the database schema as APIEntransit, contentmirrorProvide API into the Repositorywsapi4plone.parfait
CMFDeploymentStatic deliveryStrengthsVarious large projects have used itVery flexible configuration modesSupports transportWeaknessFlexibility brings complexity / lacks testingLast release 2006
In the Wild
enpraxis.staticsiteStatic deliveryStrengthsVery simple and approachableSerializes Plone to file systemWeaknessRequires programming & deploymentLack of documentation and testing
In the Wild
EntransitDynamic DeliveryStrengthsSupports staging of entire changesetsConceptually simpleWeaknessRequires too much programmingDocumentation and testing is lackingToo many moving partsDoes not have connection to CMS during load into data store. Significantly complicates state mgmt
In the Wild
ContentMirrorDynamic deliveryStrengthsSimple, Extensible, and FastTight coupling with Plone event modelWeaknessEvolving data store & Plone models requires disciplineData model currently exposes impl details
In the Wild
Wrap UpAlt delivery is atypical use case for PloneSystems with multiple ORG actorsStatic Delivery vs. Dynamic DeliveryBuilding up is more approachableDocumentation / Testing criticalDynamic delivery: CM is best bet.Entransit 2 plans on layering staging on CMNow lets go into details of ContentMirror
ContentMirror Pitch Sync Serialize content from Plone RDBMSJust Works™Small code base < 1k LOC; 99% coverageLet SQLAlchemy do heavy liftingRDBMS query-ability of content attributesWell tested, 99% coverage + mocked
Evaluate TodayBuildout: Plone, ContentMirror, and SOLREstimate: 30 minutesConfiguration of Relational DatabaseEstimate: 20 minutesBootstrap database with ddl.pyEstimate: 5 minutesHave real time Plone Content (RDBMS) and Search (SOLR); Now pick frontend.
UsagesBusiness reports based on content repository (i.e. Crystal Reports, iReport)Provide richer queries (using SQL)Store data in multiple back endsRDBMSGoogle App Engine
You already knowIf you do not customize Plone heavily; it migrates really well.Plone lots of neat features: WebDAV (Enfold Desktop;), LDAP, Content Types, Cache Fu (Enfold Proxy;), etc.So the authoring experience is flexible, configurable, and enjoyableYou can have multiple “sub-sites”
ObservationsPresentation tiers have different requirements than content repositorySimplicity is key. Maintaining presentation tiers have to be simple; few abstractionsPHP or Django have few abstractions to display dynamic data on web pageIsolate complexity; never will go awayPlone seen as self-licking ice cream cone
The StoryBig university has 200 IT staff; all busy managing HR, ERP, printers, etc.Individual departments are independent and have transient staff who manage webDepartments need way to use transient staff to maintain web presence; if CMS exists. Transient staff can use PHP.PHP, Django, .NET developers are cheap
More StoryThere is no way IT can:Manage all department changes centrallyMaintain system that has software changing daily or hourlyAllow people to make a change that impacts campus wide systemNoam in Linguistics should not be able write code which DoS the campus CMSIT does best with focus: keep systems up.
Even More Story?IT offer primitives: LDAP (users), RDBMS (content), SOLR (search). Commodity servicesDepartments implement however they likeContentMirror is a lightweight step towards decoupling the CMS from presentationMake presentation as flexible as possible
InstallationBuildout the recipe.cfgPut SQLAlchemy URI in .zcml or ENVbin/instance run ddl.py $DB > out.ddlExecute out.ddlStart ZopeCreate Plone site and add contentQuery your RDBMS
FeaturesAuto-generate DDL for registered typesAll default Plone content types workCurrently focused on Archetypes; z3 easyBulk import of Plone content into RDBMSRe-entrant (run multiple times)Schema available in JSON (vocabularies)Fast. Serialized 100k content + references in < 20m
Your content typeCreate new zcml that is included<configure xmlns="http://namespaces.zope.org/zope"xmlns:ore="http://namespaces.objectrealms.net/mirror"> <ore:engineurl="mysql://user:passwd@host/database" name="mirror-db" encoding='utf8' echo="False"/> <ore:mirror content="Products.AudioholicsContent.AHHomePage.AHHomePage"/></ore:engine></configure>Re-run ddl.py; execute ddl.py in RDBMSAll default Plone content types work OOTB
Field not available?ore.contentmirror/transform.zcmlcontains adapters for how AT Fields are mapped to SQLAlchemyore.contentmirror/transform.pycontains the adapter source code <adapter for=".interfaces.ITextField .interfaces.ISchemaTransformer" provides=".interfaces.IFieldTransformer" factory=".transform.TextTransform" />
Escape HatchIf you don’t want field level serializationWrite custom transformer for your classYou take full responsibility forCopying state from instance to RDBMSMay also have to do some extra work
Of Interestore.contentmirror/operation.pyProvide buffer to filter out duplicate operationsSimple pattern for transaction integrationore.contentmirror/jsonschema.pySerializes AT schemas into RDBMS via a JSON structurePresentation can easily find out what the original schema (including vocabularies)Not used by default
DynamismWhat about PloneFormGen or interaction?ContentMirror would serialize the Form elements as database rows; FE has to draw and validate formsCollage would generate data representing rows, columns pointing to content UIDPresentation tier does re-implement common functionality (RSS, Login, etc)
ConclusionSignificant experience in communityStarting simple is more approachableStatic vs. DynamicCM has a great experience OOTBrc2 has minor OOTB issue; all fixed in trunkStaging not implemented across multiplecontent types. DEV, STAGE, PRODAlt delivery is a minority use case.
ConclusionYou would never know that Plone fits these use casesLittle visibility in broader communityHard to separate the add-on pilecollective simply has too much activityCostly to maintain; advocateWithout formal support in Plone support expensiveKapilThangavelu wrote ContentMirror & CMFDeployment – awesome software designer!
FinContent Mirror (ore.contentmirror) http://code.google.com/p/contentmirror/SOLR (collective.solr)http://lucene.apache.org/solr/Enfold Systemshttp://www.enfoldsystems.com/My email addressalan@enfoldsystems.com

More Related Content

Content delivery Plone Symposium East 2010

  • 1. Alt. Content DeliveryAlan RunyanPlone Symposium EastState College, Pennsylvania5/29/09
  • 2. OverviewReview modes of content deliveryWhy and what purpose Existing solutions for PloneStrengths and WeaknessContentMirror in detailConclusion
  • 3. What is itTwo modes in-band & out-of-bandPlone Default is “in band”Req to Plone; Plone computes Respe.g. XDV, mod_wsgi, mod_proxy, varnishAtypical usage of Plone, “out of band”Plone emits content into intermediary storagePlone is not involved in responding to ReqNo other FOSS CMS has such feature
  • 4. How Others Do ItRender pages to filesystem + syncCustom programming to iterate over repository rendering views to diskProvide an API into repository for web-tier application to source contentPromote the database schema as API
  • 5. Out of BandStatic DeliveryGenerates static assets from PloneAssets transferred to web serverStatically rendered content servedDynamic DeliveryPlone pushes content into 3rd party data storeSeparate web application to display content
  • 6. Why?Static DeliveryConstraints in delivery mode of websiteSecurity requires no moving partse.g. Kiosk, updated every X hoursDynamic DeliveryWeb application is vastly different than CMSDifferent organizational / skill-sets being usede.g. existing website is .NET; want new section to be driven by Plone
  • 8. Plone ApproachRender pages to filesystem + syncCMFDeployment, enpraxis.staticsitePromote the database schema as APIEntransit, contentmirrorProvide API into the Repositorywsapi4plone.parfait
  • 9. CMFDeploymentStatic deliveryStrengthsVarious large projects have used itVery flexible configuration modesSupports transportWeaknessFlexibility brings complexity / lacks testingLast release 2006
  • 11. enpraxis.staticsiteStatic deliveryStrengthsVery simple and approachableSerializes Plone to file systemWeaknessRequires programming & deploymentLack of documentation and testing
  • 13. EntransitDynamic DeliveryStrengthsSupports staging of entire changesetsConceptually simpleWeaknessRequires too much programmingDocumentation and testing is lackingToo many moving partsDoes not have connection to CMS during load into data store. Significantly complicates state mgmt
  • 15. ContentMirrorDynamic deliveryStrengthsSimple, Extensible, and FastTight coupling with Plone event modelWeaknessEvolving data store & Plone models requires disciplineData model currently exposes impl details
  • 17. Wrap UpAlt delivery is atypical use case for PloneSystems with multiple ORG actorsStatic Delivery vs. Dynamic DeliveryBuilding up is more approachableDocumentation / Testing criticalDynamic delivery: CM is best bet.Entransit 2 plans on layering staging on CMNow lets go into details of ContentMirror
  • 18. ContentMirror Pitch Sync Serialize content from Plone RDBMSJust Works™Small code base < 1k LOC; 99% coverageLet SQLAlchemy do heavy liftingRDBMS query-ability of content attributesWell tested, 99% coverage + mocked
  • 19. Evaluate TodayBuildout: Plone, ContentMirror, and SOLREstimate: 30 minutesConfiguration of Relational DatabaseEstimate: 20 minutesBootstrap database with ddl.pyEstimate: 5 minutesHave real time Plone Content (RDBMS) and Search (SOLR); Now pick frontend.
  • 20. UsagesBusiness reports based on content repository (i.e. Crystal Reports, iReport)Provide richer queries (using SQL)Store data in multiple back endsRDBMSGoogle App Engine
  • 21. You already knowIf you do not customize Plone heavily; it migrates really well.Plone lots of neat features: WebDAV (Enfold Desktop;), LDAP, Content Types, Cache Fu (Enfold Proxy;), etc.So the authoring experience is flexible, configurable, and enjoyableYou can have multiple “sub-sites”
  • 22. ObservationsPresentation tiers have different requirements than content repositorySimplicity is key. Maintaining presentation tiers have to be simple; few abstractionsPHP or Django have few abstractions to display dynamic data on web pageIsolate complexity; never will go awayPlone seen as self-licking ice cream cone
  • 23. The StoryBig university has 200 IT staff; all busy managing HR, ERP, printers, etc.Individual departments are independent and have transient staff who manage webDepartments need way to use transient staff to maintain web presence; if CMS exists. Transient staff can use PHP.PHP, Django, .NET developers are cheap
  • 24. More StoryThere is no way IT can:Manage all department changes centrallyMaintain system that has software changing daily or hourlyAllow people to make a change that impacts campus wide systemNoam in Linguistics should not be able write code which DoS the campus CMSIT does best with focus: keep systems up.
  • 25. Even More Story?IT offer primitives: LDAP (users), RDBMS (content), SOLR (search). Commodity servicesDepartments implement however they likeContentMirror is a lightweight step towards decoupling the CMS from presentationMake presentation as flexible as possible
  • 26. InstallationBuildout the recipe.cfgPut SQLAlchemy URI in .zcml or ENVbin/instance run ddl.py $DB > out.ddlExecute out.ddlStart ZopeCreate Plone site and add contentQuery your RDBMS
  • 27. FeaturesAuto-generate DDL for registered typesAll default Plone content types workCurrently focused on Archetypes; z3 easyBulk import of Plone content into RDBMSRe-entrant (run multiple times)Schema available in JSON (vocabularies)Fast. Serialized 100k content + references in < 20m
  • 28. Your content typeCreate new zcml that is included<configure xmlns="http://namespaces.zope.org/zope"xmlns:ore="http://namespaces.objectrealms.net/mirror"> <ore:engineurl="mysql://user:passwd@host/database" name="mirror-db" encoding='utf8' echo="False"/> <ore:mirror content="Products.AudioholicsContent.AHHomePage.AHHomePage"/></ore:engine></configure>Re-run ddl.py; execute ddl.py in RDBMSAll default Plone content types work OOTB
  • 29. Field not available?ore.contentmirror/transform.zcmlcontains adapters for how AT Fields are mapped to SQLAlchemyore.contentmirror/transform.pycontains the adapter source code <adapter for=".interfaces.ITextField .interfaces.ISchemaTransformer" provides=".interfaces.IFieldTransformer" factory=".transform.TextTransform" />
  • 30. Escape HatchIf you don’t want field level serializationWrite custom transformer for your classYou take full responsibility forCopying state from instance to RDBMSMay also have to do some extra work
  • 31. Of Interestore.contentmirror/operation.pyProvide buffer to filter out duplicate operationsSimple pattern for transaction integrationore.contentmirror/jsonschema.pySerializes AT schemas into RDBMS via a JSON structurePresentation can easily find out what the original schema (including vocabularies)Not used by default
  • 32. DynamismWhat about PloneFormGen or interaction?ContentMirror would serialize the Form elements as database rows; FE has to draw and validate formsCollage would generate data representing rows, columns pointing to content UIDPresentation tier does re-implement common functionality (RSS, Login, etc)
  • 33. ConclusionSignificant experience in communityStarting simple is more approachableStatic vs. DynamicCM has a great experience OOTBrc2 has minor OOTB issue; all fixed in trunkStaging not implemented across multiplecontent types. DEV, STAGE, PRODAlt delivery is a minority use case.
  • 34. ConclusionYou would never know that Plone fits these use casesLittle visibility in broader communityHard to separate the add-on pilecollective simply has too much activityCostly to maintain; advocateWithout formal support in Plone support expensiveKapilThangavelu wrote ContentMirror & CMFDeployment – awesome software designer!
  • 35. FinContent Mirror (ore.contentmirror) http://code.google.com/p/contentmirror/SOLR (collective.solr)http://lucene.apache.org/solr/Enfold Systemshttp://www.enfoldsystems.com/My email addressalan@enfoldsystems.com