Facets of applied smw

Facets of Applied Semantic MediaWiki ++SMWCon Fall 2011 Tutorial #42011.09.21

AboutThis tutorial is intended to be used for people who already had the basic knowledge about Semantic MediaWiki.Me: http://semantic-mediawiki.org/wiki/User:JesseSMW++ : Ontoprise, TeamMersion, Vulcan, etc.Project Halo: http://projecthalo.com/Dr. Mark GreavesVulcan Inc.Paul Allen, Idea Man

Using a BundleSemantic Bundle http://www.mediawiki.org/wiki/Semantic_BundleFeatures: (Pros and Cons)A set of author-selected extensionsA fixed set of extensions, few customization optionsSimplified download and configuration processStill need manual configurationUsually up-to-date and work well togetherCommunity tested and supported

Installation PackageSMW+ Package http://smwforum.ontoprise.com/smwforum/index.php/Help:SMW%2BFeatures:A set of (fixed) company selected extensionsProfessionally documented and testedUsually weeks after major release of latest SMWSimplified installation and configurationWindows installer and VM images are greatYou may be curious about what it does exactly…

Deployment FrameworkReally easy to use and flexibleFunctionality (extensions) growing fastSaves lots of time of maintenanceClicks rather than lots of commandsWith the benefit of efficiency, quality and consistencyLimited by the extensions and packages in itSupported extensions and packages are growing fast

Developer DeploymentWhen you want a set of custom extensions, you wantYour OWN deployment mechanism!

Deployment PracticesYour own codebase, with checkpointsSVN or Git to get the base, and thenCustomize the wikiScript or Manual or MixedA third-party deployment framework and repositoryOntoprise: http://smwforum.ontoprise.com/smwforum/index.php/Help:Deployment_FrameworkRPI: http://code.google.com/p/smwbp/wiki/setup_wikiReferata: http://smw.referata.com/wiki/Category:Packages

Build SystemYou own it!Your own selection of extensionsAnd versions of themOwn patches, scripts and templatesVulcan: Internal GIT repositories of completed buildVersioning and branches between projectsOther examples:RPI Best Practice (Google code)Hudson build (PNNL)

Future TrendA public wiki installation repositorywith lots of packagesIncluding necessary dataSemantically marked upWell documented (or even reviewed)Easy to find and useFederated wiki package store (Wiki AppStore)Call for Participation

What to CustomizeSkinsExtensionsSchema and DataTemplates and formsAnything else you can code… 

SkinsHow to customize a skinCreate your skin file in /skins folder within your wiki directory, like Ontoskin.phpFinish this skin files following Ontoskin.php skin filesThere are two classes in this skin files:Inherit from SkinTemplate, set the CSS and template filter.Inherit from QuickTemplate, set the UI styleCreate skin folder with your skin name in /skinsCopy image and CSS files to your skin folderUse your own skinIn LocalSettings.php set$wgDefaultSkin = 'wiking_skin'

Skin Customization ExampleVulcan Development Wiki and Seahawk wiki. We have customized these two skins for our projects and the key is the method of QuickTemplate. In this method, you can decide what to show and how it shows on your wikiYou can customize wiki site like menu links, page titles, CSS and JavaScript files in the data properties of QuickTemplateFor example, show wiki side bar in your skin<?phpforeach ($this->data['sidebar'] as $bar => $cont) { ?> <li> <a href="#"><h5><?php $out = wfMsg( $bar );></h5></a> <ul> <?phpforeach($cont as $key => $val) { ?> <li id="<?php echo $val['id'] ?>" > <a href="<?php echo $val['href'] ?>"><?php echo $val['text'] ?></a> </li> <?php } ?> </ul> </li><?php } ?>

Personalized SkinSkin supports customization via __USERNAME__User can have personalized styles within a skinhttp://www.mediawiki.org/wiki/Manual:Skin_configuration#User_CSSTip: A way to turn off Wikipedia banner ad:http://en.wikipedia.org/wiki/User:Jesseone/vector.css

“Just Do It!”Easy to startFlat, large ontology, at beginningMany properties only valid in certain casesMay have similar/duplicate itemsBut it’ll get better over time!Seahawks Video Wiki

Designer OntologyProfessionally generated after rounds of discussionsTime consumingMore comprehensive and thoroughly designedMore data (restriction) than requiredData (ontology) importer isn’t powerful to fully support all features in the designing tool (Protégé)Biology Ontology Wiki

The Middle WayPractical schema design Plan and ExploreThink, adapt, and evolveOur Agile Project Management Wiki

N-ary RelationsType:Record in SMWSemantic Internal Object extensionAdding an object relation in the middle (Reification)Using parameterized properties (Example: the third turn wiki)Using a triple store (external)

Find ExtensionsWhere to find extensionshttp://www.mediawiki.org/wiki/Extension_MatrixBy supported version (1.16, 1.17, …)By status (beta, stable, …)By type (API, AJAX, hook, user rights…)By new-ness (sort by most recently created time)By talk page new-ness…By most updated versions…And of course using search tools from G.Y.M…

Get the Extensions!Some essential MediaWiki extensions!http://semantic-mediawiki.org/wiki/Help:MediaWiki_extensionsAnd quite a few Semantic MediaWiki extensions:http://semantic-mediawiki.org/wiki/Help:SMW_extensionsFind what you want Not exactly? Then patch it!Nothing close? Then build it!Or find people to build it 

Our ExperiencesWe built a few semantic wikis (2008-2011)Sci-Fi movie demoSemantic Football (Entertainment) NGT – a work-order handling systemUltrapedia – an Analytical EncylopediaWiking agile project managementBiology ontologyStanford class wikiNeurowiki …

Patches Often NeededMany features are sort of there, some still not.Popular extensions (e.g. SF, SRF, etc.) often need patches or tuningExample: SF Auto-completion (over the years)Fixed set of allowed valuesAll values in a categoryAll values having a specific propertyAll values from a custom query

New Features AddedReal-time notification (Semantic NotifyMe, 2008)Aggregation (average, max, sum, etc.) (2009)Video merging Open Flash Charts (OFC) SRF (2009-2010)Strict > and < comparison (2010)Exhibit enhancements (webit) (2009)New formats: Runway, Time Plot, etc.Mapping: unique ID requirementsGeocodingSemantic WikiTags (2009)Semantic Connector (2010)

Two of Our Latest ExtensionsWiki Object ModelEnable third parties to consume wiki data via APIsAPIs (and URLs) for easy and flexible manipulation of wiki page (and data) At a finer granularityWidget Designer / Page Creator (prototype)To lower structured page creation dramaticallyTo enable data-driven application buildingCome to my Semantic Page Creator talk on Friday

Why Wiki Object ModelStrong needs for access the structured data and unstructured data in the wiki as a data storeFrom internal and external applications’ point of view A DOM-like approach is good choiceXpath is a reasonable tool and standardGranularity offers flexibility and powerEasier to operate on known objects and calling REST APIs rather than parsing and handling wiki textsCommon gateway for both internal and external apps

Wiki Object Model ObjectsCategoryLinkPropertyTextMagic wordHTML tagSentenceWord*Image31PageSectionParameterParameter valueTemplateTemplate fieldParser functionList itemTableTable cell* Not yet implemented as of September 2011

Demo of WOMDemo at http://wiking.vulcan.com/dev_sandbox/

The API is part of MediaWiki API, with our additions to allow get and set (read/write) the content of a wiki page

http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Apis

A simple inline editor (AJAX styled) to let user modify a specific portion of the wiki page

Based on http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Functions32

Security ExtensionsMany choices = nothing is perfectOptions: from very basic to very advanced:No SecurityMediaWiki Built-in GroupsSimple Security (Page Security)HaloACLFeaturesRestrictions

A Short Case StudyProposal Management System (Wiki)Users (including privileged users) submit proposalsReviewers (privileged users) rate and review themReviewers discuss and make decision and assignWhat security extension should it employ?HaloACL? – the most powerful ACL out thereSimpleSecurity? – simple and effectiveOr MediaWiki built-in groups? – simplestMediaWiki Groups

VisualizationsSemantic Result FormatterFormat = ???{Param} = ???

Visualization: Our Path2007: SRF used to only have few options 2008-2009: Exhibit seemed coolPorted webitFixed some bugs and enhanced features Runway, TimePlot, multiple rows in Map view etc.2009-2010: Open Flash ChartMore bars, pies, lines, scatter plot…InteractionRicher information display2010-11: TabbedHighChart, OFC/ScatterPlot

Comparing SRF OptionsGoogle Charts (pie and bar)Google does it! (G gets your data, no offline choice…)Static bar and pie, no color options…ExhibitMore choices (timeline, map, facet, runway…)Size limit; Browser compatibility issuesFlash-based: OFC (Open Flash Chart), High ChartMany bars, charts, plots, even scattered plotsFlash required

Sidebar CustomizationBasic: MediaWiki:SidebarEnhanced: semantic queries, and tree viewsAdvanced: use variables together with queriesCustom: Using an extension: DynamicSidebarCustomize your own sidebar at User:<username>/Sidebarhttp://www.mediawiki.org/wiki/Extension:DynamicSidebarExample: http://wiking.vulcan.com/dev/

Current Sprint in Sidebar*Current sprint{{#ask: [[Category:Project sprints]][[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]][[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]|format=template|template=Sidebar query template|link=none|}}

My Active Sprint Tasks in Sidebar*My tasks{{#ask: [[Category:Project tasks]][[Project task owner::<q>[[User alias::{{CURRENTUSER}}]]</q>]][[Project story::<q>[[Project sprint::<q>[[Category:Project sprints]][[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]][[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]</q>]]</q>]][[Project task status::In progress||Not started]]|format=template|template=Sidebar query template|link=none|}}

More Extension ExamplesTwitter Feedhttp://www.mediawikiwidgets.org/Twitter_Search (example)Flickr Feedhttp://www.mediawikiwidgets.org/FlickrAmazon Carouselhttps://widgets.amazon.com/Amazon-Carousel-Widget/Google Analyticshttp://www.mediawiki.org/wiki/Extension:Google_AnalyticsOptify Analyticshttp://www.optify.net/

Templates and FormsMediaWiki template is essential for its structured data handlingEditing, especially Template editing, is not trivialSemantic Forms extension greatly lowers the barCustomizing templates and forms can provide very visually appealing pages

Form with a Stylehttp://www.thethirdturn.com/w/index.php?title=Form:Driver&action=edit

Forms with autocompletionAdvanced Auto-Completion on Customized Query ResultsBasic Auto-Completion on Category Values

StepsKnow what to buildCustomers, customers, customers!Get your collection of extensions…Customize your wiki (template, forms)Handle the data (schema, query, API, …)

Case 1. Proper WikiFocus on Skin, Form and TemplateSteps:Pick a (default) skinDesign some formsAnd templates (queries) behind the formsBuild starter pages and Go!

Biology Ontology WikiPeople need discuss about the details related to construct a biology ontologyClasses (categories)PropertiesRelationsTextbook analysisDiscussionForms necessary to keep content tidy

Forms on Namespacehttp://www.mediawiki.org/wiki/Extension:Semantic_Forms#Based_on_namespace

Case 2: Simple Workflow AppIdea: go from one form to another formNeed form links in the templateMay need user rights management

Proposal Review Workflow ExampleUser submits a proposal via a formFill in basic information about the proposalSet a field in the form to be, say, “New”User doesn’t see further information about “Reviewer” or “Owner”, etc.Semantic notification (email and RSS feed) facilitates communicationReviewer finds the form in a canned queryReviewer changes the status to, say, “Approved”Reviewer field (say “Reviewed by”) is (automatically) setThen reviewer assigns the proposal to a owner, say, “Alice”The owner (“Alice”) now sees it and can start work on itChanges the status to “Active”Adds a start date or maybe an estimated end date too

Ways to Handle WorkflowAdd a link to open another formForce edit “review” using another form (e.g. reviewer form) When saving the new form, the category is changedNow we have a different category  a different default formChange a value to include or exclude to another templateMay need #if (or #switch) statement in template to change (or include/exclude) template valuesUse “Page has default form” property (Semantic Forms extension)

More WorkflowGood summary by YaronKorenSMWCon Spring 2011 TalkSMW and the Workflow PuzzleMy Semantic Page Creator Talk on Friday 9/23Example: A simple HelpDesk wiki

Case 3: WikingDev WikiOur project management wikiWe use it for *all* things it can do:Proposals, ideas, email messages, features, bugsAgile development: milestones, sprints, stories, tasksCode/Feature association: SVN mapped to tasks/bugsProgress analysis: burn-down charts, work calendarCollaboration, Documentation, Demonstration

Examples of Data I/OWiking Development Wiki An Imported emailhttp://wiking.vulcan.com/dev/index.php/Demo_scenariosAn email imported as a Project Bughttp://wiking.vulcan.com/dev/index.php/Issue_11886_Insert_a_single_propertyA task uploaded via Outlook with multiple commitshttp://wiking.vulcan.com/dev/index.php/WikiTags_Release_Structure

Microsoft Office ConnectorLeverage Microsoft Office applications and technologyBring SMW info to Office applications on-demandAPI for data I/O: add and modify wiki data within Microsoft Office Utilize semantics to improve relevanceSmart actions for semantic propertiesAPIAPIConnectionsSmartJesse Wang | SemTech 2010

Our Agile Project Management (Scrum)60WikingJesse Wang | SMWCon Amsterdam 2010

Wiki Data I/OWiking development wiki has two external applications Subversion integrationMicrosoft Office integrationExtensions to support itSemantic WikitagsSemantic Connector

Data I/O ExtensionsMediaWiki API – very basic read/writePage Object ModelFirst Data I/O extension to allow access some wiki structure data (links, title, basic template field)SMWWriterBased on POM, support annotations (semantic properties)Data API extension in SMW+Semantic WikiTags and Semantic ConnectorWiki Object ModelA powerful and comprehensive object model (Data API)Read/write at finer object level (sentences, template parameters)

Case Study: Battle-space Luminary System Discover when New Information represents a change in understanding of entitiesDiscovery of explicit entity links, implicit relationshipsLarge Volumes of Data in various formatsUnstructured news articlesTactical Reports, Field IntelligenceStructured Database InformationUse Wiki Pages to represent current knowledge about an entity – “what we know”Domain Ontology to represent domain of information – “what we want to know”Issue Alerts when Significant Events occurNew information according to categoryChanging information on topics of interestNeed to send information to various devices – cell phones, email, etc.63

System DesignWiki ConfigurationSemantic MediaWiki: Large developer community, active development, open source. Wikipedia uses MediaWiki, so scalability and performance are important.Semantic Results Format: Provides various rich media displays of semantic information, including graphs, timelines, mapsSemantic Forms: Provides convenient user interface for entering semantic data into wiki, avoiding cumbersome wikitextSemantic Notifications: Enables sending of notifications when results of semantic query change.Domain OntologyCreated OWL Ontology for TerrorismSemantic Parsing, Extraction, ReasoningJava Process using various Open-Source ToolkitsRapid plugin of new technologiesMultiple Data Sources supported64

Wiki Content DesignUse Templates to Ensure Consistent Look-and-FeelTemplates Correspond to Ontology ClassesFields within Templates correspond to Properties within OntologyRich Content Visualizations derived in consistent wayHierarchical Categories match Class Hierarchy within OntologyEnsures Validity for PropertiesCategory included on each Template page to ensure consistencyFormsProvide ability for users to enter data directly into wiki without knowing Wiki TextEach form corresponds to a TemplateFields within forms correspond to the fields/properties within the TemplateGUI can include auto-completionCreated Page immediately linked semantically to rest of Wiki66

Sample Visualizations67UI enables notifications based on results of query – message sent when visualization changesVisualizations automatically created w/o user edit(tables, timelines, maps, social networks…)

Facets of applied smw

Related slideshows

More Related Content

Facets of applied smw

Editor's Notes