Writing Pluggable Software
•
11 likes•3,867 views
"How to write pluggable software" presented by Tatsuhiko Miyagawa at YAPC::Asia 2007 in Tokyo on April 5th 2007.
Report
Share
![Writing Pluggable Software Tatsuhiko Miyagawa [email_address] Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo](https://cdn.statically.io/img/image.slidesharecdn.com/writing-pluggable-software-18358/85/Writing-Pluggable-Software-1-320.jpg)
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
Recommended for you
More Related Content
What's hot
What's hot (20)
Similar to Writing Pluggable Software
Similar to Writing Pluggable Software (20)
More from Tatsuhiko Miyagawa
More from Tatsuhiko Miyagawa (20)
Recently uploaded
Recently uploaded (20)
Writing Pluggable Software
- 1. Writing Pluggable Software Tatsuhiko Miyagawa [email_address] Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo
- 2. For non-JP attendees … If you find in the code, Replace that with backslash. (This is MS' fault)
- 5. Pl u ggable Software
- 6. Agenda
- 7. #1 How to make your app pluggable
- 8. #2 TMTOWTDP There's More Than One Way To Deploy Plugins Pros/Cons by examples
- 10. Benefits
- 11. #1 Keep the app design and code simple
- 12. #2 Let the app users customize the behavior (without hacking the internals)
- 13. #3 It's fun to write plugins for most hackers (see: Plagger and Kwiki)
- 14. "Can your app do XXX?" "Yes, by plugins."
- 15. "Your app has a bug in YYY" "No, it's the bug in plugin YYY, Not my fault." (Chain Of Responsibilities)
- 16. Good Enough Reasons, huh?
- 17. #1 Make your app pluggable
- 18. Example
- 19. ack (App::Ack)
- 20. grep –r for programmers
- 21. Ack is a "full-stack" software now.
- 22. By "full-stack" I mean: Easy install No configuration No way to extend
- 23. Specifically: These are hardcoded Ignored directories Filenames and types
- 24. Ignored Directories @ignore_dirs = qw( blib CVS RCS SCCS .svn _darcs .git );
- 25. Filenames and languages mapping %mappings = ( asm => [qw( s S )], binary => …, cc => [qw( c h xs )], cpp => [qw( cpp m h C H )], csharp => [qw( cs )], … perl => [qw( pl pm pod tt ttml t )], … );
- 26. What if making these pluggable?
- 27. DISCLAIMER
- 28. Don't get me wrong Andy, I love ack the way it is… Just thought it can be a very good example for the tutorial.
- 29. Quickstart: Class::Trigger Module::Pluggable © Six Apart Ltd. Employees
- 30. Class::Trigger SYNOPSIS package Foo; use Class::Trigger; sub foo { my $self = shift; $self->call_trigger('before_foo'); # some code ... $self->call_trigger('after_foo'); } package main; Foo->add_trigger(before_foo => amp;sub1); Foo->add_trigger(after_foo => amp;sub2);
- 31. Class::Trigger Helps you to implement Observer Pattern. (Rails calls this Observer)
- 32. Module::Pluggable SYNOPSIS package MyClass; use Module::Pluggable; use MyClass; my $mc = MyClass->new(); # returns the names of all plugins installed under MyClass::Plugin::* my @plugins = $mc->plugins(); package MyClass::Plugin::Foo; sub new { … } 1;
- 33. Setup plugins in App::Ack package App::Ack; use Class::Trigger; use Module::Pluggable require => 1; __PACKAGE__->plugins;
- 34. Setup plugins in App::Ack package App::Ack; use Class::Trigger; use Module::Pluggable require => 1 ; __PACKAGE__->plugins; # "requires" modules
- 35. Ignored Directories (Before) @ignore_dirs = qw( blib CVS RCS SCCS .svn _darcs .git );
- 36. Ignored Directories (After) # lib/App/Ack.pm __PACKAGE__->call_trigger(' ignore_dirs.add ', ignore_dirs);
- 37. Ignored Directories (plugins) # lib/App/Ack/Plugin/IgnorePerlBuildDir.pm package App::Ack::Plugin::IgnorePerlBuildDir; App::Ack->add_trigger( " ignore_dirs.add " => sub { my($class, $ignore_dirs) = @_; push @$ignore_dirs, qw( blib ); }, ); 1;
- 38. Ignored Directories (plugins) # lib/App/Ack/Plugin/IgnoreSourceControlDir.pm package App::Ack::Plugin::IgnoreSourcdeControlDir; App::Ack->add_trigger( " ignore_dirs.add " => sub { my($class, $ignore_dirs) = @_; push @$ignore_dirs, qw( CVS RCS .svn _darcs .git ); }, ); 1;
- 39. Filenames and languages (before) %mappings = ( asm => [qw( s S )], binary => …, cc => [qw( c h xs )], cpp => [qw( cpp m h C H )], csharp => [qw( cs )], … perl => [qw( pl pm pod tt ttml t )], … );
- 40. Filenames and languages (after) # lib/App/Ack.pm __PACKAGE__->call_trigger('mappings.add', mappings);
- 41. Filenames and languages (plugins) package App::Ack::Plugin::MappingCFamily; use strict; App::Ack->add_trigger( "mappings.add" => sub { my($class, $mappings) = @_; $mappings->{asm} = [qw( s S )]; $mappings->{cc} = [qw( c h xs )]; $mappings->{cpp} = [qw( cpp m h C H )]; $mappings->{csharp} = [qw( cs )]; $mappings->{css} = [qw( css )]; }, ); 1;
- 42. Works great with few lines of code!
- 43. Now it's time to add Some useful stuff.
- 44. Example Plugin: Content Filter
- 45. sub _search { my $fh = shift; my $is_binary = shift; my $filename = shift; my $regex = shift; my %opt = @_; if ($is_binary) { my $new_fh; App::Ack->call_trigger('filter.binary', $filename, new_fh); if ($new_fh) { return _search($new_fh, 0, $filename, $regex, @_); } }
- 46. Example: Search PDF content with ack
- 47. PDF filter plugin package App::Ack::Plugin::ExtractContentPDF; use strict; use CAM::PDF; use File::Temp; App::Ack->add_trigger( 'mappings.add' => sub { my($class, $mappings) = @_; $mappings->{pdf} = [qw(pdf)]; }, );
- 48. PDF filter plugin (cont.) App::Ack->add_trigger( 'filter.binary' => sub { my($class, $filename, $fh_ref) = @_; if ($filename =~ /pdf$/) { my $fh = File::Temp::tempfile; my $doc = CAM::PDF->new($file); my $text; for my $page (1..$doc->numPages){ $text .= $doc->getPageText($page); } print $fh $text; seek $$fh, 0, 0; $$fh_ref = $fh; } }, );
- 49. PDF search > ack --type=pdf Audrey yapcasia2007-pugs.pdf:3:Audrey Tang
- 50. Homework Use File::Extract To handle arbitrary media files
- 51. Homework 2: Search non UTF-8 files (hint: use Encode::Guess) You'll need another hook.
- 52. Summary Class::Trigger + Module::Pluggable = Pluggable app easy
- 53. #2 TMTOWTDP There's More Than One Way To Deploy Plugins
- 54. Module::Pluggable + Class::Trigger = Simple and Nice but has limitations
- 55. In Reality, we need more control over how plugins behave
- 56. 1) The order of plugin executions
- 57. 2) Per user configurations for plugins
- 58. 3) Temporarily Disable plugins Should be easy
- 59. 4) How to install & upgrade plugins
- 60. 5) Let plugins have storage area
- 61. Etc, etc.
- 62. Examples: Kwiki Plagger qpsmtpd Movable Type
- 63. I won't talk about Catalyst plugins (and other framework thingy)
- 64. Because they're NOT "plug-ins"
- 65. Install plugins And now you write MORE CODE
- 66. 95% of Catalyst plugins Are NOT "plugins" But "components" 95% of these statistics is made up by the speakers.
- 67. Kwiki 1.0
- 68. Kwiki Plugin code package Kwiki::URLBL; use Kwiki::Plugin -Base ; use Kwiki::Installer -base; const class_id => 'urlbl'; const class_title => 'URL Blacklist DNS'; const config_file => 'urlbl.yaml'; sub register { require URI::Find; my $registry = shift; $registry->add(hook => 'edit:save', pre => 'urlbl_hook'); $registry->add(action => 'blacklisted_url'); }
- 69. Kwiki Plugin (cont.) sub urlbl_hook { my $hook = pop; my $old_page = $self->hub->pages->new_page($self->pages->current->id); my $this = $self->hub->urlbl; my @old_urls = $this->get_urls($old_page->content); my @urls = $this->get_urls($self->cgi->page_content); my @new_urls = $this->get_new_urls(old_urls, urls); if (@new_urls && $this->is_blocked(new_urls)) { $hook->cancel(); return $self->redirect("action=blacklisted_url"); } }
- 70. Magic implemented in Spoon(::Hooks)
- 71. "Install" Kwiki Plugins # order doesn't matter here (according to Ingy) Kwiki::Display Kwiki::Edit Kwiki::Theme::Basic Kwiki::Toolbar Kwiki::Status Kwiki::Widgets # Comment out (or entirely remove) to disable # Kwiki::UnnecessaryStuff
- 72. Kwiki plugin config # in Kwiki::URLBL plugin __config/urlbl.yaml__ urlbl_dns: sc.surbl.org, bsb.spamlookup.net, rbl.bulkfeeds.jp # config.yaml urlbl_dns: myowndns.example.org
- 73. Kwiki plugins are CPAN modules
- 74. Install and Upgrade plugins cpan> install Kwiki::SomeStuff
- 75. Using CPAN as a repository Pros #1: reuse most of current CPAN infrastructure.
- 76. Using CPAN as a repository Pros #2: Increasing # of modules = good motivation for Perl hackers
- 77. Cons #1: Installing CPAN deps could be a mess (especially for Win32)
- 78. Cons #2: Whenever Ingy releases new Kwiki, lots of plugins just break.
- 79. Kwiki plugin storage return if grep {$page->id} @{$self-> config->cached_display_ignore }; my $html = io->catfile( $self-> plugin_directory ,$page->id )->utf8;
- 80. Kwiki 2.0
- 81. Same as Kwiki 1.0
- 82. Except: plugins are now in SVN repository
- 84. Plagger plugin package Plagger::Plugin::Publish::iCal; use strict; use base qw( Plagger::Plugin ); use Data::ICal; use Data::ICal::Entry::Event; use DateTime::Duration; use DateTime::Format::ICal; sub register { my($self, $context) = @_; $context-> register_hook ( $self, ' publish.feed ' => amp;publish_feed, ' plugin.init ' => amp;plugin_init, ); }
- 85. Plagger plugin (cont) sub plugin_init { my($self, $context) = @_; my $dir = $self->conf->{dir}; unless (-e $dir && -d _) { mkdir $dir, 0755 or $context->error("Failed to mkdir $dir: $!"); } }
- 86. Plagger plugin storage $self->conf->{invindex} ||= $self-> cache->path_to ('invindex');
- 87. Plagger plugin config # The order matters in config.yaml # if they're in the same hooks plugins: - module: Subscription::Config config: feed: - http://www.example.com/ - module: Filter::DegradeYouTube config: dev_id: XYZXYZ - module: Publish::Gmail disable: 1
- 88. Plugins Install & Upgrade cpan> notest install Plagger # or … > svn co http://…/plagger/trunk plagger > svn update
- 89. Plagger impl. ripped off by many apps now
- 90. qpsmtpd
- 91. mod_perl for SMTP Runs with tcpserver, forkserver or Danga::Socket standalone
- 92. Plugins: Flat files rock:/home/miyagawa/svn/qpsmtpd> ls -F plugins async/ greylisting auth/ hosts_allow check_badmailfrom http_config check_badmailfromto ident/ check_badrcptto logging/ check_badrcptto_patterns milter check_basicheaders parse_addr_withhelo check_earlytalker queue/ check_loop quit_fortune check_norelay rcpt_ok check_relay relay_only check_spamhelo require_resolvable_fromhost content_log rhsbl count_unrecognized_commands sender_permitted_from dns_whitelist_soft spamassassin dnsbl tls domainkeys tls_cert* dont_require_anglebrackets virus/
- 93. qpsmtpd plugin sub hook_mail { my ($self, $transaction, $sender, %param) = @_; my @badmailfrom = $self->qp->config ("badmailfrom") or return (DECLINED); for my $bad (@badmailfrom) { my $reason = $bad; $bad =~ s/^*(+).*/$1/; next unless $bad; $transaction->notes('badmailfrom', $reason) … } return (DECLINED); }
- 94. Actually qpsmtpd Plugins are "compiled" to modules
- 95. my $eval = join("", "package $package;", 'use Qpsmtpd::Constants;', "require Qpsmtpd::Plugin;", 'use vars qw(@ISA);', 'use strict;', '@ISA = qw(Qpsmtpd::Plugin);', ($test_mode ? 'use Test::More;' : ''), "sub plugin_name { qq[$plugin] }", $line, $sub, "", # last line comment without newline? ); $eval =~ m/(.*)/s; $eval = $1; eval $eval; die "eval $@" if $@;
- 96. qpsmtpd plugin config rock:/home/miyagawa/svn/qpsmtpd> ls config.sample/ config.sample: IP logging require_resolvable_fromhost badhelo loglevel rhsbl_zones badrcptto_patterns plugins size_threshold dnsbl_zones rcpthosts tls_before_auth invalid_resolvable_fromhost relayclients tls_ciphers
- 97. config/plugins # content filters virus/klez_filter # rejects mails with a SA score higher than 2 spamassassin reject_threshold 20
- 98. config/badhelo # these domains never uses their domain when greeting us, so reject transactions aol.com yahoo.com
- 99. Install & Upgrade plugins Just use subversion
- 101. MT plugins are flat-files (or scripts that call modules)
- 102. MT plugin code package MT::Plugin::BanASCII; our $Method = "deny"; use MT; use MT::Plugin; my $plugin = MT::Plugin->new({ name => "BanASCII v$VERSION", description => "Deny or moderate ASCII or Latin-1 comment", }); MT->add_plugin($plugin); MT->add_callback('CommentFilter', 2, $plugin, amp;handler);
- 103. MT plugin code (cont) sub init_app { my $plugin = shift; $plugin->SUPER::init_app(@_); my($app) = @_; return unless $app->isa('MT::App::CMS'); $app-> add_itemset_action ({ type => 'comment', key => 'spam_submission_comment', label => 'Report SPAM Comment(s)', code => sub { $plugin->submit_spams_action('MT::Comment', @_) }, } );
- 106. MT plugin storage require MT::PluginData; my $data = MT::PluginData->load({ plugin => 'sidebar-manager', key => $blog_id }, ); unless ($data) { $data = MT::PluginData->new; $data->plugin('sidebar-manager'); $data->key($blog_id); } $data->data( modulesets ); $data->save or die $data->errstr;
- 107. Order control MT->add_callback('CMSPostEntrySave', 9 , $rightfields, amp;CMSPostEntrySave); MT->add_callback('CMSPreSave_entry', 9 , $rightfields, amp;CMSPreSave_entry); MT::Entry->add_callback('pre_remove', 9 , $rightfields, amp;entry_pre_remove); Defined in plugins. No Control on users end
- 108. Conclusion Flat-files vs. Modules
- 109. Flat-files: ☺ Easy to install (Just grab it) ☻ Hard to upgrade OK for simple plugins
- 110. Modules: ☺ Full-access to Perl OO goodness ☺ Avoid duplicate efforts of CPAN ☻ Might be hard to resolve deps. Subversion to the rescue (could be a barrier for newbies)
- 111. Nice-to-haves: Order control Temporarily disable plugins Per plugin config Per plugin storage
- 112. Resources Class::Trigger http:// search.cpan.org /dist/Class-Trigger/ Module::Pluggable http:// search.cpan.org /dist/Module-Pluggable/ Ask Bjorn Hansen: Build Easily Extensible Perl Programs http://conferences.oreillynet.com/cs/os2005/view/e_sess/6806 qpsmtpd http:// smtpd.develooper.com / MT plugins http:// www.sixapart.com/pronet/plugins / Kwiki http:// www.kwiki.org / Plagger http:// plagger.org /