"fireap" - fast task runner on consul
•
0 likes•2,464 views
"Fireap" is a consul-based rapid propagative task runner for large systems. https://github.com/key-amb/fireap
Report
Share
![“fireap” - fast task runner
on Consul
@key_amb
In 渋谷.rb[:20160420] #shibuyarb](https://cdn.statically.io/img/image.slidesharecdn.com/fireap-fasttaskrunneronconsul-160420085826/85/fireap-fast-task-runner-on-consul-1-320.jpg)
"fireap" - fast task runner on consul
- 1. “fireap” - fast task runner on Consul @key_amb In 渋谷.rb[:20160420] #shibuyarb
- 2. Who am I ? ❏ @key_amb ❏ Software Engineer ❏ Serverside, Web ❏ https://github.com/key-amb ❏ https://rubygems.org/profiles/key-amb
- 3. fireap
- 4. Overview of “fireap” ❏ https://github.com/key-amb/fireap ❏ Propagative task runner on Consul Cluster ❏ O(log N) … N = node number ❏ Consul key features used in “fireap”: ❏ Kv, Event, Watch ❏ And powered by diplomat, gem for Consul API ❏ Blog: ❏ Consul クラスタ上で動作する S3 非依存の pull 型デプロイツール "fireap" を作った
- 5. Related Products ❏ https://github.com/fujiwara/stretcher ❏ https://github.com/sorah/mamiya These above enable O(1) deployment. But both need storage with big I/O capacity (like S3) => difficult to use in private DC. Then, I came up with “fireap”.
- 6. How fast is “fireap”? ❏ Benchmark: ❏ rsync for 100 of t2.micro nodes on AWS EC2. GNU Parallel fireap real 0m46.906s 0m18.992s user 0m40.407s 0m00.527s sys 0m04.241s 0m00.046s
- 8. Overview
- 9. config/fireap.toml [task.apps.bench] exec_commands = [ "rsync -az --delete <%= @remote.address %>:<%= ENV ['HOME'] %>/<%= @app %> <%= ENV['HOME'] %>/", ]
- 10. Run “fireap fire” on Publisher node // on publisher node % fireap fire -a bench -v 0.1.0-demo
- 11. Subscriber node exec commands // on subscriber node % tailf fireap.log : 2016-04-20 17:08:08 +0900 [INFO] EXEC rsync -az --delete 172.31.0.1:/home/bob/bench /home/bob/ :
- 12. Propagates through the whole cluster
- 13. Changes in v0.1 ~ v0.4 ❏ Enable to filter target nodes by Consul “service”, “tag” and “node name”. ❏ Add config validation before every command execution. ❏ Add `task` subcommand to show tasks in config file.
- 14. Things to be done ❏ Write more tests and do more benchmarks ❏ Plan to monitor Consul ❏ seems preferable especially in bigger systems ❏ Datastore alternatives (maybe) ❏ may be needed if Consul Kv goes slow in bigger systems ❏ Clean up codes
- 15. How to get started ❏ Documentation is Available ❏ https://github.com/key-amb/fireap/wiki ❏ If it doesn’t work, let me know as GitHub issue. ❏ I’m happy if you try “fireap”.
- 16. Thank you!
- 17. ❏ Serf／Consulで管理を自動化！ ～実践的な 手法を紹介～：特集｜gihyo.jp ❏ Consulと自作OSSを活用した100台規模の Webサービス運用 // Speaker Deck Appendix