SlideShare a Scribd company logo
Flickr   : Web Services Cal Henderson
What is Flickr? Photo sharing website (flickr.com) The  place to store digital photos The centre of a big distributed system A set of open APIs
What the heck are ‘Web Services’? The future of the Internet!!!1 Really just buzzwords
Web services in a nutshell Server Business Logic Interface Client Interface UI Transport

Recommended for you

Configuring Apache Servers for Better Web Perormance
Configuring Apache Servers for Better Web PerormanceConfiguring Apache Servers for Better Web Perormance
Configuring Apache Servers for Better Web Perormance

Apache is the most popular web server in the world, yet its default configuration can't handle high traffic. Learn how to setup Apache for high performance sites and leverage many of its available modules to deliver a faster web experience for your users. Discover how Apache can max out a 1 Gbps NIC and how to serve over 140,000 pages per minute with a small Apache cluster. This presentation was given by Spark::red's founding partner Devon Hillard in March 2012 at the Boston Web Performance Meetup.

sparkred; spark::red; apache
Optimizing the performance of WordPress
Optimizing the performance of WordPressOptimizing the performance of WordPress
Optimizing the performance of WordPress

Optimizing the performance of WordPress by Josh Highland of NewLeaf Labs. Presentation was given at Orange County WordCamp 2011

wordpressoptimizationorange county wordcamp
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...

Amazon CloudFront and Amazon Route 53 can help optimize web application performance and availability. CloudFront improves performance by caching static and reusable content at edge locations and optimizing delivery of dynamic content through features like keep-alive connections and latency-based routing. Route 53 provides fast, reliable DNS services and can health check origins to improve high availability. Together, CloudFront and Route 53 provide a global network that caches content close to users and routes traffic based on network conditions to optimize performance and design for failure.

aws cloudarc309acceleration
Web services in a nutshell Server Business Logic Interface Client Interface UI Transport Web Server Web Browser HTTP
Web services in a nutshell Server Business Logic Interface Client Interface UI Transport Web Server Application XML-RPC
Web services in a nutshell Server Business Logic Interface Client Interface UI Transport Web Server Java Programmers SOAP
Why should I care? You can avoid code reuse While offering multiple services…

Recommended for you

Lamp Stack Optimization
Lamp Stack OptimizationLamp Stack Optimization
Lamp Stack Optimization

Mike Creuzer's presentation on LAMP optimization for the October, 2009 meeting of the Suburban Chicago PHP & Web Development Meetup

linuxmysqlphp
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...

Traditionally, content delivery networks (CDNs) were known to accelerate static content. Amazon CloudFront has come a long way and now supports delivery of entire websites that include dynamic and static content. In this session, we introduce you to CloudFront’s dynamic delivery features that help improve the performance, scalability, and availability of your website while helping you lower your costs. We talk about architectural patterns such as SSL termination, close proximity connection termination, origin offload with keep-alive connections, and last-mile latency improvement. Also learn how to take advantage of Amazon Route 53's health check, automatic failover, and latency-based routing to build highly available web apps on AWS.

aws cloudparviz deyhim2014 aws summit san francisco
Drupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp NorthDrupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp North

This document provides information on improving Drupal performance through various techniques including performance testing, caching, optimizing database and server configurations, using tools like Varnish, load balancers, and CDNs, and addressing inefficient code. It also discusses a case study of using scalable cloud hosting and caching strategies to handle peak traffic for a site during major awards events.

drupalperformance
Web services Server Business Logic Interface Web Browsers HTTP Interface Interface Email Clients Email Web Apps REST Clients
Web services Server Business Logic Interface Web Browsers HTTP Interface Interface Email Clients Email Web Apps REST Clients People get  very excited  about this part
Ok, I get that bit Give me a real example! Aren’t you supposed to be talking about Flickr?
Flickr’s Logical Architecture Page Logic Business/Application Logic Database Photo Storage API Logic Endpoints Templates Users 3 rd  Party Apps Flickr Apps Node Service Flickr .com Email Parser

Recommended for you

Use case for using the ElastiCache for Redis in production
Use case for using the ElastiCache for Redis in productionUse case for using the ElastiCache for Redis in production
Use case for using the ElastiCache for Redis in production

This document discusses using ElastiCache for Redis in production. It begins with an introduction to the author and their role at gumi Inc. It then provides an overview of Redis operations and data persistence methods. The document outlines gumi's migration from memcached to Redis, including their use of master-slave replication and multiple databases. It notes potential issues with ElastiCache for Redis like lack of auto failover and inability to change instance types. In summary, while some features are still missing, ElastiCache for Redis allows quick node provisioning and has proven reliable for gumi despite initial function gaps.

Building Web APIs that Scale
Building Web APIs that ScaleBuilding Web APIs that Scale
Building Web APIs that Scale

The document discusses building highly scalable web APIs. It proposes moving to an asynchronous architecture using event-driven frameworks like gevent. This allows decoupling requests from backend resources to avoid failures under load. The architecture incorporates authentication, throttling to delay requests if needed, and selectively dropping requests to prevent failures. The goal is graceful degradation, with the ability to service most requests during spikes rather than having the entire API become unavailable.

Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...

Traditionally, content delivery networks (CDNs) were known to accelerate static content. Amazon CloudFront has come a long way and now supports delivery of entire websites that include dynamic and static content. In this session, we introduce you to CloudFront dynamic delivery features that help improve the performance, scalability and availability of your website, while helping you lower your costs. In this session we will talk about architectural patterns such as SSL termination, close proximity connection termination, origin offload with keep-alive connections and last-mile latency improvement. You will also learn how to take advantage of Amazon Route 53 health check, automatic failover and latency-based routing to build highly available web apps on AWS.

awssummittel-aviv 2014
Flickr’s Physical Architecture Web Servers Database Servers Node Servers Static Servers Users Metadata Servers
But seriously… We only care about PHP! So where does Flickr use it?
PHP is at the core of Flickr Page Logic Business/Application Logic Database Photo Storage API Logic Endpoints Templates Users 3 rd  Party Apps Flickr Apps Node Service Flickr .com Email Parser
Ok, ok – what besides PHP? Smarty for templating PEAR for XML and Email parsing Java for… Controlling ImageMagick (image processing) Storage metadata The node service MySQL (4.0 / InnoDb) Perl for deployment & testing tools Apache 2, Redhat, etc. etc.

Recommended for you

2015 ZendCon - Do you queue
2015 ZendCon - Do you queue2015 ZendCon - Do you queue
2015 ZendCon - Do you queue

A document about queues discusses what queues are, why they are used, common use cases, implementation patterns, protocols, considerations when implementing queues, and how to handle issues that may arise. Queues act as buffers that allow different applications or systems to communicate asynchronously by passing messages. They help decouple components, distribute load, and improve reliability and user experience. Common examples of messages that may be queued include emails, images, videos, and IoT data.

message queue
Scale your Alfresco Solutions
Scale your Alfresco Solutions Scale your Alfresco Solutions
Scale your Alfresco Solutions

In this session, we'll discuss architectural, design and tuning best practices for building rock solid and scalable Alfresco Solutions. We'll cover the typical use cases for highly scalable Alfresco solutions, like massive injection and high concurrency, also introducing 3.3 and 3.4 Transfer / Replication services for building complex high availability enterprise architectures.

devcon 2010best practices trackalfresco
Shopzilla - Performance By Design
Shopzilla - Performance By DesignShopzilla - Performance By Design
Shopzilla - Performance By Design

Shopzilla redesigned their architecture to improve performance and scalability. The new design simplified layers, utilized caching extensively, and applied best practices for front-end performance. This led to significant business benefits including a 7-12% increase in conversion rates, 8-120% increase in search engine sessions, and a 225% increase in development velocity. Performance testing was a key part of the new approach.

javaperformanceshopzilla
Medium sized application Small team (3 programmers until recently) 1 PHP, 1 Flash/DHTML, 1 Java >60,000 lines of PHP code >80 smarty extensions >60,000 lines of templates >250,000 users >3,500,000 photos >50,000,000 page views per month Growing fast Like,  really  fast So these stats are out of date by now
Thinking outside the web app Services Atom/RSS/RDF Feeds APIs SOAP XML-RPC REST We love PEAR::XML::Tree
More services Email interface Postfix PHP PEAR::Mail::mimeDecode FTP Uploading API Authentication API Unicode (Not really a service, but common to all Flickr services)
Even more services Real time application The “node service” ‘Cool’ flash apps Which use the REST APIs Blogging APIs Blogger API (1 & 2) Metaweblog API Atom LiveJournal

Recommended for you

AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...

Whether you’re running a simple website, a mobile app, or a suite of business applications, DNS is a fundamental part of any architecture in the cloud. In this mid-level architecture session, we’ll cover everything you need to get started with Amazon Route 53, AWS’s highly-available DNS service. You’ll learn how to use public DNS, including routing techniques such as weighted round-robin, latency-based routing, and geo DNS; how to configure DNS failover using health checks; how and when to use private DNS within your Virtual Private Cloud (VPC); and how Amazon Route 53 interacts with Amazon EC2’s DNS for instance naming and DNS resolution across your network. We will conclude the session with a real-world migration example. Warner Bros. Entertainment recently completed a full DNS migration to Route 53. Vahram Sukyas, Vice President, Application Infrastructure & Operations at Warner Bros. Entertainment, will share details on his team's architecture, migration strategy, and lessons learned which are useful for enterprises and startups alike.

introductory (200 level)reinvent2016amazon web services
Effectively Deploying MongoDB on AEM
Effectively Deploying MongoDB on AEMEffectively Deploying MongoDB on AEM
Effectively Deploying MongoDB on AEM

The document discusses using MongoDB as a scalable storage solution for Adobe Experience Manager (AEM). It introduces MongoDB and the MongoMK storage component that allows AEM to use MongoDB. The rest of the document covers best practices for sizing, deploying, and operating an AEM and MongoDB configuration including considerations for availability, volume, working set, latency, deployment automation, and operational monitoring.

ops managermongodbaem
Indroduction to Web Application
Indroduction to Web ApplicationIndroduction to Web Application
Indroduction to Web Application

This document provides an introduction to web applications, outlining key concepts like client-server communication using HTTP, static websites served from web servers, dynamic content through client-side scripts, and full-fledged web applications running on application servers using server-side scripts. It explains how static files are directly served to clients, while dynamic requests trigger script interpretation or program execution to generate responses. Modern application servers like Tomcat use Java to compile scripts for high performance and prevent crashes through the virtual machine.

web application development
APIs are simple! Modeled on XML-RPC (sort of) Method calls with XML responses Named arguments (key/name pairs) Tricky in  WebServices.framework on Mac OS X SOAP, XML-RPC and REST are just transports PHP endpoints mean we can use the same application logic as the website Endpoints talk to the business logic using PHP function calls Essentially a really fast transport
XML isn’t simple :( PHP 4 doesn’t have good a XML parser PHP 5 is new and scares me (and it wasn’t out when we started) Expat is cool though (PEAR::XML::Parser) Why doesn’t PEAR have XPath? Because PEAR is stupid! PHP 4 sucks! Actually, PHPXPath rocks http://phpxpath.sourceforge.net/
Creating API methods Stateless method-call APIs are easy to extend They don’t affect each other Adding a method requires no knowledge of the transport We just get passed arguments and return XML The transport layer hides all that junk Adding a method once makes it available to all the interfaces Self documenting – method dispatch requires a list of methods Because everyone hates writing documentation
Red-Hot Unicode Action UTF-8 pages CJKV support It’s  really  cool

Recommended for you

Sizing your alfresco platform
Sizing your alfresco platformSizing your alfresco platform
Sizing your alfresco platform

Sizing an alfresco infrastructure has always been an interesting topic with lots of unrevealed questions. There is no perfect formula that can accurately define what is the perfect sizing for your architecture considering your use case. However, we can provide you with valuable guidance on how to size your Alfresco solution, by asking the right questions, collecting the right numbers, and taking the right assumptions on a very interesting sizing exercise. How many alfresco servers will you need on your alfresco cluster? How many CPUs/cores do you need on those servers to handle your estimated user concurrency? How do you estimate the sizing and growth of your storage? How much memory do you need on your Solr servers? How many Solr servers do you need to get the response times you require? What are the golden rules that can drive and maintain the success of an Alfresco project?

architecture alfrescobest practices alfrescosizing alfresco
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014

Peek behind the scenes to learn about Amazon ElastiCache's design and architecture. See common design patterns of our Memcached and Redis offerings and how customers have used them for in-memory operations and achieved improved latency and throughput for applications. During this session, we review best practices, design patterns, and anti-patterns related to Amazon ElastiCache.

aws-reinventaws cloudservices deep dive
Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009

Synchronous Reads, Asynchronous Writes refers to an architectural approach where data reads are performed synchronously through services, while data writes are performed asynchronously through a messaging system. This allows for decoupling of services, horizontal scaling of reads and writes, and loose coupling between systems. The key aspects are performing JSON RESTful reads through services like Sinatra, and pushing writes to a messaging system like RabbitMQ with routing keys to trigger downstream processing. This approach can help solve issues with monolithic Rails applications that do not scale effectively.

rubyconf ruby rabbitmq sinatra typhoeus
 
Unicode for all It’s  really  easy Don’t need PHP support Don’t need MySQL support Just need the right HTTP headers UTF-8 is 7-bit transparent Just don’t mess with high characters Don’t use HtmlEntities()! Or |escape in Smarty But bear in mind… JavaScript has patchy Unicode support People using your APIs might be stupid Some of them ARE stupid, guaranteed
Scaling the beast Why PHP is great MySQL scaling Search scaling Horizontal scaling
But first… Why do we need to scale? There are a lot of people on the Internet They all want to use our “web services” Whether they know it yet or not

Recommended for you

PHP is the king, nodejs is the prince and Lua is the fool
PHP is the king, nodejs is the prince and Lua is the foolPHP is the king, nodejs is the prince and Lua is the fool
PHP is the king, nodejs is the prince and Lua is the fool

Namshi is an e-commerce company based in Dubai that has grown to over 250 employees and serves millions of visitors per month. Originally using a monolithic PHP architecture, they migrated to a microservices architecture using PHP, Node.js, RabbitMQ, and other tools to improve scalability, deployability, and maintainability. This involved breaking the monolith into independent services, implementing an event-driven messaging system, and building new frontends using JavaScript.

nodesoaphp. nodejs
Flickr Architecture Presentation
Flickr Architecture PresentationFlickr Architecture Presentation
Flickr Architecture Presentation

1) Flickr is a photo sharing website built using PHP that allows users to upload and tag photos and share them publicly or privately. 2) The website uses a logical architecture with application logic, page logic, templates, and endpoints separating the different components. 3) While primarily built with PHP, the website also incorporates technologies like Smarty for templating, Perl for image processing, MySQL for the database, and Java for the node service.

 
by eraz
Flickr and PHP - Cal Henderson
Flickr and PHP - Cal HendersonFlickr and PHP - Cal Henderson
Flickr and PHP - Cal Henderson

This document summarizes Flickr's architecture and use of PHP. [1] Flickr uses PHP for its application logic and templating, with Smarty templating and PEAR libraries. [2] The logical architecture includes photo storage, databases, node services, application logic, page/API logic, and templates/endpoints. [3] PHP allowed Flickr to scale horizontally by making the application stateless and storing all data in databases.

flickrphp
Why PHP is great Stateless We can bounce people around servers Everything is stored in the database Even the smarty cache “Shared nothing” (so long as we avoid PHP sessions) But what this really means… …is we just have to deal with scaling elsewhere
A MySQL Scaling Haiku Database server slow Load of over two hundred Replication wins!
MySQL Replication But it only gives you more SELECT’s Else you need to partition vertically Re-architecting sucks :(
Looking at usage But really, we SELECT much more than anything else A snapshot says SELECT’s 44m INSERT’s 1.3m UPDATE’s 1.7m DELETE’s 0.3m 19 SELECT’s for each IUD

Recommended for you

Web Services Mash-Up
Web Services Mash-UpWeb Services Mash-Up
Web Services Mash-Up

The document discusses Flickr and web services. It describes Flickr as a photo-sharing website and application with an open API. It then discusses different types of web service transports including SOAP, XML-RPC, and REST. It notes that services should support multiple transports and cautions about potential performance issues from API abuse.

Etech2005
Etech2005Etech2005
Etech2005

The document discusses Flickr and web services. It describes Flickr as a photo-sharing website and application with an open API. It then discusses different types of web service transports including SOAP, XML-RPC, and REST. The document cautions that performance problems can occur if APIs are abused, such as through excessive scraping or polling, and recommends solutions like caching and monitoring.

PHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the foolPHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the fool

This document discusses Namshi's transition from a monolithic PHP architecture to a microservices architecture using PHP, Node.js, and other technologies. Some of the benefits realized include improved scalability, deployability, and maintainability. Key aspects covered include the use of service-oriented architecture with independent services, HTTP as the communication protocol, RabbitMQ for messaging, and PHP frameworks like Silex for building APIs. The new architecture also aims to be stateless, using JSON Web Tokens for authentication.

phpexpresslua
Replication is really cool A bunch of slave servers handle all the SELECT’s A single master handles IUD’s We can scale horizontally, at least for a while.
Searching A simple text search We were using RLIKE Then switched to LIKE Then disabled it all together
FULLTEXT Indexes FULLTEXT saves the day! But they’re only supported on MyISAM tables And we use InnoDb for locking We’re doomed :(
But wait! Partial replication saves the day Replicate the portion of the database we want to search  But change the table types on the slave to MyISAM It can keep up because it’s only handling IUD’s on a couple of tables And we can reduce the IUD’s with a little bit of vertical partitioning

Recommended for you

Servlet & jsp
Servlet  &  jspServlet  &  jsp
Servlet & jsp

The document discusses web servers, web browsers, and how they communicate via HTTP. It explains that the browser makes requests to the server, and the server responds by sending back content like HTML pages, images, and other resources. It also provides details on HTTP methods like GET and POST, and how browsers use HTML and CSS to render and display web pages for users.

Modern Application Stacks
Modern Application StacksModern Application Stacks
Modern Application Stacks

Slides for a presentation I gave on May 4, 2011 at the GTA PHP user's group (http://meetup.gtaphp.org) about components for modern PHP application stacks

phpapplication stacks
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystemI can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem

The Ruby ecosystem is pretty awesome when it comes to developing or consuming HTTP APIs. On the publishing front, the Rails framework is an attractive option because it supports publishing what are popularly (but inaccurately) referred to as 'RESTful' APIs quickly and effortlessly. On the consumer side, the Ruby ecosystem provides several very fluent and powerful libraries that make it easy to consume HTTP based APIs. Since a significant proportion of projects today require that APIs be both published and consumed, many of them wind up choosing Ruby as a platform for the reasons mentioned above. This talk is targeted at folks that are currently on such projects, or anticipate being on such projects in the future. We will cover: Consuming HTTP APIs: 1) The basics of making HTTP calls with Ruby 2) The strengths and weaknesses of Ruby's Net::HTTP across 1.8, 1.9 and JRuby (possibly Rubinius if we have the time to do research) 3) Popular HTTP libraries that either make it easier to do HTTP by providing better APIs, make it faster by using libCurl or both 4) Different approaches to deserializing popular encoding formats such as XML and JSON and the pitfalls thereof Producing HTTP APIs using Rails: 1) The basics of REST 2) What Rails gives you out of the box - content-type negotiation, deserialization etc. and the limitations thereof 3) What Rails fails to give you out of the box - hypermedia controls etc. 4) What Rails does wrong - wrong PUT semantics, no support for PATCH, error handling results in responses that violate the clients Accepts header constraints etc. 4) How one can achieve Level 2 on the Richardson Maturity Model of REST using Rails 5) Writing tests for all of this At the end of this, our audience will understand how you can both consume and produce HTTP APIs in the Ruby ecosystem. They will also have a clear idea of what the limitations of such systems are and what the can do to work around the limitations.

railsrubyconfindiarubyconfindia2011
JOIN’s are slow “ Normalised data is for sissies” Erm, “ Selective de-normalisation can be a big win” Keep multiple copies of data around Makes searching faster Have to ensure consistency in the application logic For instance, have a concat’d field containing a bunch of child-row data, just for searching.
Our current setup Slave Farm DB1 Master IUD’s SELECT’s Search Slave Farm Search SELECT’s DB3 Main Search slave DB2 Main Slave
Our current, current setup Main Cluster Aux Cluster Search Cluster Slave Farm DB1 Master IUD’s SELECT’s Search Slave Farm Search SELECT’s DB3 Main Search slave DB2 Main Slave Slave Farm DB4 Master IUD’s SELECT’s DB5 Main Slave
Horizontal scaling At the core of our design Just add hardware! Inexpensive Not exponential Avoid redesigns/re-architectures

Recommended for you

REST+JS - Codebits 2011
REST+JS - Codebits 2011REST+JS - Codebits 2011
REST+JS - Codebits 2011

A talk I gave at Codebits 2011. You can find the video here: https://codebits.eu/intra/s/session/204 Abstract: There's an abundance of talks about developing for the web. Most of it targeted to developing mass-market websites, worrying about CDNs, SEO, bookmarks, progressive enhancement, etc. But what if you want to build fullblown applications that just happen to be implemented in web technologies? Web apps, especially single-page, highly interactive ones, have some special characteristics. For a long time we've been implementing web apps as if the browser was a mostly dumb terminal, and instead kept most of the logic on the server. Even current favorits like RoR or GWT keep to that old paradigm. We'll see how we can take a REST architecture, combining it with a good dose of javascript MVP to create applications that are much closer, both in structure and in functionality, to desktop apps than to normal webpages. By fully embracing web technology, instead of "hiding" behind some leaky abstractions, we'll be set free to take our apps to the next level. And we'll see how much nicer they are to develop.

flickr's architecture & php
flickr's architecture & php flickr's architecture & php
flickr's architecture & php

- Flickr uses PHP for its page and application logic, along with technologies like Smarty, PEAR, Perl, and Java. The site has over 60,000 lines of PHP code and templates. - The logical architecture includes page logic, application logic, photo storage, APIs, templates, third-party apps. The physical architecture includes static servers, database servers, and node servers. - PHP is used for page and application logic. Smarty is used for templating. MySQL is used for storage. Java is used in the node service. Flickr scales horizontally by adding more hardware over time.

Lecture1 introduction by okello erick
Lecture1 introduction by okello erickLecture1 introduction by okello erick
Lecture1 introduction by okello erick

This document provides an overview of the IT3101 Web-based Database Development course. The course aims to teach theoretical and practical skills for developing web-based database systems using PHP, MYSQL and SQL. The course content is divided into three parts that cover working with MySQL, working with PHP, and building simple applications. Key topics include an introduction to client-side and server-side scripting, learning PHP syntax and functions, interacting with MySQL databases, and putting the skills to use through sample projects.

Talking to the Node Service Just another service with an API But just internal at the moment Everyone speaks XML (badly) Just TCP/IP - fsockopen() We’re issuing commands, not requesting data, so we don’t bother to parse the response Just substring search for  state=“ok” This only works for a simple protocol
Still talking to the Node Service Don’t rely on it! Check the connection was established Use a connection timeout Use an IO timeout!
RSS / Atom / RDF Different formats (all quite bad) We’re generating a lot of different feeds Abstract the difference away using templates No good way to do private feeds. Why is nobody working on this? (WSSE maybe?) Most of the feed readers (including bloglines.com) support basic HTTP Auth Easy to implement in PHP We love PHP It’s great!
Receiving email We want users to be able to email photos to Flickr Get postfix to pipe each mail to a PHP script Parse the mail and find any photos Cellular phone companies hate you Lots of mailers are retarded Photos as text/plain attachments Segments out of order No mime types UUEncoded and mime-less

Recommended for you

Mobile APIs: Optimizing APIs for Many Devices
Mobile APIs: Optimizing APIs for Many DevicesMobile APIs: Optimizing APIs for Many Devices
Mobile APIs: Optimizing APIs for Many Devices

This document discusses optimizing APIs for mobile devices. It notes that mobile devices and networks are slower than servers, so APIs need to be optimized for these constraints. It recommends transforming data to more efficient formats like JSON, offering partial responses, enabling compression, implementing caching across a content delivery network, and being judicious about the amount of data returned to improve performance for mobile clients. The overall message is that APIs need to be designed with the limitations of mobile in mind.

api strategyapiapi management
CMIS: An Open API for Managing Content
CMIS: An Open API for Managing ContentCMIS: An Open API for Managing Content
CMIS: An Open API for Managing Content

This talk gives an overview of CMIS and the Alfresco API. Originally presented at GlueCon 2013 and updated for Alfresco Day Sydney in August of 2013.

alfresco cloudcmisalfresco api
Lamp
LampLamp
Lamp

The document provides an overview of the LAMP web development stack, which consists of Linux as the operating system, Apache as the web server, MySQL as the database management system, and PHP as the programming language. It discusses each component, their history, features, and how they work together to provide a free and open-source alternative to proprietary web development platforms.

Processing email PEAR to the rescue Mail::mime_decode With some patches UUEncoding Relax the address atom parser We need to convert character sets ICONV loves you
Upload via FTP PHP isn’t so great at being a daemon PHP4, I mean. Maybe PHP 5 is great Leaks memory like a sieve No (easy) threads Java to the rescue Java just acts as an FTPd and passes all uploaded files to PHP for processing This isn’t actually public Not my idea Bricolage does this I think. Maybe Zope?
Blogs Why does everyone loves blogs so much? Only a few APIs really Blogger Metaweblog Blogger2 Movable Type Atom Live Journal
It’s all broken Lots of blog software has broken interfaces It’s a support nightmare Manila is tricky But it all works, more or less Abstracted in the application logic We just call  blogs_post_message(); And so can you, via the API

Recommended for you

PHP Basics
PHP BasicsPHP Basics
PHP Basics

The break command is used to skip the current iteration of a loop and move on to the next one.

php
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from TechlightningFrequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from Techlightning

MuleSoft Interview Questions and answers from youtube channel Techlightning https://www.youtube.com/watch?v=JqBgT2t6cCQ&list=PLfEAetjBY9s4YdBie3VSpufxNcnC3wJvK

mulesoft for beginnersmulesoft
Day03 api
Day03   apiDay03   api
Day03 api

This document discusses issues a mobile developer named Sara had with her company's backend team. Some key problems Sara faced included: changes to the backend breaking features in her app; inconsistencies in API data formats, JSON structure, and status codes; and lack of navigation links and proper documentation for the API. The document then provides recommendations for the backend team to improve the API, such as implementing versioning, following hypermedia constraints, maintaining consistency in data formats and structures, and thoroughly documenting the API.

soapphp webservicesphp xml
Back to those APIs We opened up the Flickr APIs a few months ago Programmers mainly build tools for other programmers We now have Perl, python, PHP, ActionScript, XMLHTTP, .NET, Objective-C, C++, C and Ruby interface libraries But also a few actual applications
Flickr Rainbow
Tag Wallpaper
iPhoto Plugin We developed a Mac uploader But it wasn’t great A user developed an iPhoto plugin It  was  great APIs encourage people to do your work for you

Recommended for you

Sending mail,web services
Sending mail,web servicesSending mail,web services
Sending mail,web services

The document discusses sending emails using PHP. It provides the syntax for PHP's mail() function, including required and optional parameters. It then gives an example of sending a simple text email. It also demonstrates how to create an HTML form that sends email input to a specified address. Finally, it provides an overview of different types of web services (SOAP, XML-RPC, REST), their advantages and disadvantages, and reasons for using web services.

Hadoop: Distributed data processing
Hadoop: Distributed data processingHadoop: Distributed data processing
Hadoop: Distributed data processing

The document summarizes a presentation given by Amr Awadallah of Cloudera on Hadoop. It discusses how current storage systems are unable to perform computation, and how Hadoop addresses this through its marriage of HDFS for scalable storage and MapReduce for distributed processing. It provides an overview of Hadoop's history and design principles such as managing itself, scaling performance linearly, and moving computation to data.

clouderascaleyahoo
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and HadoopFacebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop

Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop. More info here http://www.royans.net/arch/hive-facebook/

scalehigh availablescalable
Flickr Carnivore Uses Carnivore PE Sniffs AIM traffic (amongst others) from the local net Calculates the most popular words of the moment Uses the Flickr API to display photos of those words It’s like a really invasive zeitgeist
Flickr Tivo A Tivo app which uses Flickr photos Just Type in some tags And your TV becomes a “digital picture frame”
So what next? Even more scaling PHP 5? MySQL 5? or NDB? Taking over the world
Flickr   : Web Services Cal Henderson

Recommended for you

Grid – Distributed Computing at Scale
Grid – Distributed Computing at ScaleGrid – Distributed Computing at Scale
Grid – Distributed Computing at Scale

Grid computing provides a distributed computing platform that allows resources to be shared and utilized across organizational boundaries at scale. It abstracts applications and services from underlying infrastructure through virtualization, automation, and service-oriented architectures. While grid adoption began in scientific and technical fields, grids are now being used in commercial enterprises for shared infrastructures, regional collaborations, and emerging "cloud" utility providers. Widespread adoption of grid computing will require addressing social and standards barriers as grids move from early deployments to broader use across different industries.

How Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the serviceHow Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the service

Are you pushing the envelope of what your web application can handle? Do your engineers feel the impending need to overhaul and retrofit your system before it unexpectedly keels over on you? Come hear Six Apart talk about the tools, software and most importantly, the process it has been using to completely rewrite the backend of TypePad without disrupting thousands of users and paying customers.

Dmk Bo2 K7 Web
Dmk Bo2 K7 WebDmk Bo2 K7 Web
Dmk Bo2 K7 Web

This document summarizes Dan Kaminsky's 2007 talk at Black Hat about exploiting vulnerabilities in the DNS system and web browser security policies to enable arbitrary TCP and UDP access from within a web browser. Kaminsky describes how controlling DNS responses allows an attacker to bypass firewalls and access internal networks by manipulating the browser's view of domain names and IP addresses. He then outlines his proof-of-concept attack software called "Slirpie" that implements a VPN-like tunnel using only a compromised web browser.

These slides are online http://ludicorp.com/flickr/
Any  Questions?

More Related Content

What's hot

Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
royans
 
Building a Scalable Architecture for web apps
Building a Scalable Architecture for web appsBuilding a Scalable Architecture for web apps
Building a Scalable Architecture for web apps
Directi Group
 
Threading through InterBase, Firebird, and beyond
Threading through InterBase, Firebird, and beyondThreading through InterBase, Firebird, and beyond
Threading through InterBase, Firebird, and beyond
Mind The Firebird
 
Configuring Apache Servers for Better Web Perormance
Configuring Apache Servers for Better Web PerormanceConfiguring Apache Servers for Better Web Perormance
Configuring Apache Servers for Better Web Perormance
Spark::red
 
Optimizing the performance of WordPress
Optimizing the performance of WordPressOptimizing the performance of WordPress
Optimizing the performance of WordPress
Josh Highland Giese
 
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
Amazon Web Services
 
Lamp Stack Optimization
Lamp Stack OptimizationLamp Stack Optimization
Lamp Stack Optimization
Dave Ross
 
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
Amazon Web Services
 
Drupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp NorthDrupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp North
Philip Norton
 
Use case for using the ElastiCache for Redis in production
Use case for using the ElastiCache for Redis in productionUse case for using the ElastiCache for Redis in production
Use case for using the ElastiCache for Redis in production
知教 本間
 
Building Web APIs that Scale
Building Web APIs that ScaleBuilding Web APIs that Scale
Building Web APIs that Scale
Salesforce Developers
 
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
Amazon Web Services
 
2015 ZendCon - Do you queue
2015 ZendCon - Do you queue2015 ZendCon - Do you queue
2015 ZendCon - Do you queue
Mike Willbanks
 
Scale your Alfresco Solutions
Scale your Alfresco Solutions Scale your Alfresco Solutions
Scale your Alfresco Solutions
Alfresco Software
 
Shopzilla - Performance By Design
Shopzilla - Performance By DesignShopzilla - Performance By Design
Shopzilla - Performance By Design
Tim Morrow
 
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
Amazon Web Services
 
Effectively Deploying MongoDB on AEM
Effectively Deploying MongoDB on AEMEffectively Deploying MongoDB on AEM
Effectively Deploying MongoDB on AEM
Norberto Leite
 
Indroduction to Web Application
Indroduction to Web ApplicationIndroduction to Web Application
Indroduction to Web Application
torny10
 
Sizing your alfresco platform
Sizing your alfresco platformSizing your alfresco platform
Sizing your alfresco platform
Luis Cabaceira
 
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
Amazon Web Services
 

What's hot (20)

Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Building a Scalable Architecture for web apps
Building a Scalable Architecture for web appsBuilding a Scalable Architecture for web apps
Building a Scalable Architecture for web apps
 
Threading through InterBase, Firebird, and beyond
Threading through InterBase, Firebird, and beyondThreading through InterBase, Firebird, and beyond
Threading through InterBase, Firebird, and beyond
 
Configuring Apache Servers for Better Web Perormance
Configuring Apache Servers for Better Web PerormanceConfiguring Apache Servers for Better Web Perormance
Configuring Apache Servers for Better Web Perormance
 
Optimizing the performance of WordPress
Optimizing the performance of WordPressOptimizing the performance of WordPress
Optimizing the performance of WordPress
 
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) ...
 
Lamp Stack Optimization
Lamp Stack OptimizationLamp Stack Optimization
Lamp Stack Optimization
 
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront ...
 
Drupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp NorthDrupal Performance : DrupalCamp North
Drupal Performance : DrupalCamp North
 
Use case for using the ElastiCache for Redis in production
Use case for using the ElastiCache for Redis in productionUse case for using the ElastiCache for Redis in production
Use case for using the ElastiCache for Redis in production
 
Building Web APIs that Scale
Building Web APIs that ScaleBuilding Web APIs that Scale
Building Web APIs that Scale
 
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
Dynamic Content Acceleration: Fast Web Apps with Amazon CloudFront and Amazon...
 
2015 ZendCon - Do you queue
2015 ZendCon - Do you queue2015 ZendCon - Do you queue
2015 ZendCon - Do you queue
 
Scale your Alfresco Solutions
Scale your Alfresco Solutions Scale your Alfresco Solutions
Scale your Alfresco Solutions
 
Shopzilla - Performance By Design
Shopzilla - Performance By DesignShopzilla - Performance By Design
Shopzilla - Performance By Design
 
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
AWS re:Invent 2016: DNS Demystified: Getting Started with Amazon Route 53, fe...
 
Effectively Deploying MongoDB on AEM
Effectively Deploying MongoDB on AEMEffectively Deploying MongoDB on AEM
Effectively Deploying MongoDB on AEM
 
Indroduction to Web Application
Indroduction to Web ApplicationIndroduction to Web Application
Indroduction to Web Application
 
Sizing your alfresco platform
Sizing your alfresco platformSizing your alfresco platform
Sizing your alfresco platform
 
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
(SDD402) Amazon ElastiCache Deep Dive | AWS re:Invent 2014
 

Similar to Flickr Services

Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009
pauldix
 
PHP is the king, nodejs is the prince and Lua is the fool
PHP is the king, nodejs is the prince and Lua is the foolPHP is the king, nodejs is the prince and Lua is the fool
PHP is the king, nodejs is the prince and Lua is the fool
Alessandro Cinelli (cirpo)
 
Flickr Architecture Presentation
Flickr Architecture PresentationFlickr Architecture Presentation
Flickr Architecture Presentation
eraz
 
Flickr and PHP - Cal Henderson
Flickr and PHP - Cal HendersonFlickr and PHP - Cal Henderson
Flickr and PHP - Cal Henderson
kangaro10a
 
Web Services Mash-Up
Web Services Mash-UpWeb Services Mash-Up
Web Services Mash-Up
Cal Henderson
 
Etech2005
Etech2005Etech2005
Etech2005
royans
 
PHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the foolPHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the fool
Alessandro Cinelli (cirpo)
 
Servlet & jsp
Servlet  &  jspServlet  &  jsp
Servlet & jsp
Subhasis Nayak
 
Modern Application Stacks
Modern Application StacksModern Application Stacks
Modern Application Stacks
chartjes
 
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystemI can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
Sidu Ponnappa
 
REST+JS - Codebits 2011
REST+JS - Codebits 2011REST+JS - Codebits 2011
REST+JS - Codebits 2011
João Nelas
 
flickr's architecture & php
flickr's architecture & php flickr's architecture & php
flickr's architecture & php
coolpics
 
Lecture1 introduction by okello erick
Lecture1 introduction by okello erickLecture1 introduction by okello erick
Lecture1 introduction by okello erick
okelloerick
 
Mobile APIs: Optimizing APIs for Many Devices
Mobile APIs: Optimizing APIs for Many DevicesMobile APIs: Optimizing APIs for Many Devices
Mobile APIs: Optimizing APIs for Many Devices
Apigee | Google Cloud
 
CMIS: An Open API for Managing Content
CMIS: An Open API for Managing ContentCMIS: An Open API for Managing Content
CMIS: An Open API for Managing Content
Jeff Potts
 
Lamp
LampLamp
Lamp
anurag1p
 
PHP Basics
PHP BasicsPHP Basics
PHP Basics
Roohul Amin
 
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from TechlightningFrequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
Arul ChristhuRaj Alphonse
 
Day03 api
Day03   apiDay03   api
Sending mail,web services
Sending mail,web servicesSending mail,web services
Sending mail,web services
Devang Patel
 

Similar to Flickr Services (20)

Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009Synchronous Reads Asynchronous Writes RubyConf 2009
Synchronous Reads Asynchronous Writes RubyConf 2009
 
PHP is the king, nodejs is the prince and Lua is the fool
PHP is the king, nodejs is the prince and Lua is the foolPHP is the king, nodejs is the prince and Lua is the fool
PHP is the king, nodejs is the prince and Lua is the fool
 
Flickr Architecture Presentation
Flickr Architecture PresentationFlickr Architecture Presentation
Flickr Architecture Presentation
 
Flickr and PHP - Cal Henderson
Flickr and PHP - Cal HendersonFlickr and PHP - Cal Henderson
Flickr and PHP - Cal Henderson
 
Web Services Mash-Up
Web Services Mash-UpWeb Services Mash-Up
Web Services Mash-Up
 
Etech2005
Etech2005Etech2005
Etech2005
 
PHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the foolPHP is the King, nodejs is the Prince and Lua is the fool
PHP is the King, nodejs is the Prince and Lua is the fool
 
Servlet & jsp
Servlet  &  jspServlet  &  jsp
Servlet & jsp
 
Modern Application Stacks
Modern Application StacksModern Application Stacks
Modern Application Stacks
 
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystemI can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem
 
REST+JS - Codebits 2011
REST+JS - Codebits 2011REST+JS - Codebits 2011
REST+JS - Codebits 2011
 
flickr's architecture & php
flickr's architecture & php flickr's architecture & php
flickr's architecture & php
 
Lecture1 introduction by okello erick
Lecture1 introduction by okello erickLecture1 introduction by okello erick
Lecture1 introduction by okello erick
 
Mobile APIs: Optimizing APIs for Many Devices
Mobile APIs: Optimizing APIs for Many DevicesMobile APIs: Optimizing APIs for Many Devices
Mobile APIs: Optimizing APIs for Many Devices
 
CMIS: An Open API for Managing Content
CMIS: An Open API for Managing ContentCMIS: An Open API for Managing Content
CMIS: An Open API for Managing Content
 
Lamp
LampLamp
Lamp
 
PHP Basics
PHP BasicsPHP Basics
PHP Basics
 
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from TechlightningFrequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
 
Day03 api
Day03   apiDay03   api
Day03 api
 
Sending mail,web services
Sending mail,web servicesSending mail,web services
Sending mail,web services
 

More from royans

Hadoop: Distributed data processing
Hadoop: Distributed data processingHadoop: Distributed data processing
Hadoop: Distributed data processing
royans
 
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and HadoopFacebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop
royans
 
Grid – Distributed Computing at Scale
Grid – Distributed Computing at ScaleGrid – Distributed Computing at Scale
Grid – Distributed Computing at Scale
royans
 
How Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the serviceHow Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the service
royans
 
Dmk Bo2 K7 Web
Dmk Bo2 K7 WebDmk Bo2 K7 Web
Dmk Bo2 K7 Web
royans
 
21 Www Web Services
21 Www Web Services21 Www Web Services
21 Www Web Services
royans
 
Web Design World Flickr
Web Design World FlickrWeb Design World Flickr
Web Design World Flickr
royans
 
Flickr Services
Flickr ServicesFlickr Services
Flickr Services
royans
 
Scalable Web Arch
Scalable Web ArchScalable Web Arch
Scalable Web Arch
royans
 
Web 2.0 Summit Flickr
Web 2.0 Summit FlickrWeb 2.0 Summit Flickr
Web 2.0 Summit Flickr
royans
 

More from royans (10)

Hadoop: Distributed data processing
Hadoop: Distributed data processingHadoop: Distributed data processing
Hadoop: Distributed data processing
 
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and HadoopFacebooks Petabyte Scale Data Warehouse using Hive and Hadoop
Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop
 
Grid – Distributed Computing at Scale
Grid – Distributed Computing at ScaleGrid – Distributed Computing at Scale
Grid – Distributed Computing at Scale
 
How Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the serviceHow Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the service
 
Dmk Bo2 K7 Web
Dmk Bo2 K7 WebDmk Bo2 K7 Web
Dmk Bo2 K7 Web
 
21 Www Web Services
21 Www Web Services21 Www Web Services
21 Www Web Services
 
Web Design World Flickr
Web Design World FlickrWeb Design World Flickr
Web Design World Flickr
 
Flickr Services
Flickr ServicesFlickr Services
Flickr Services
 
Scalable Web Arch
Scalable Web ArchScalable Web Arch
Scalable Web Arch
 
Web 2.0 Summit Flickr
Web 2.0 Summit FlickrWeb 2.0 Summit Flickr
Web 2.0 Summit Flickr
 

Recently uploaded

Entrepreneurial mindset: An Introduction to Entrepreneurship
Entrepreneurial mindset: An Introduction to EntrepreneurshipEntrepreneurial mindset: An Introduction to Entrepreneurship
Entrepreneurial mindset: An Introduction to Entrepreneurship
Sanjay Joshi
 
Destor.One - Business Presentation Dec22.pdf
Destor.One - Business Presentation Dec22.pdfDestor.One - Business Presentation Dec22.pdf
Destor.One - Business Presentation Dec22.pdf
Thales Jacobi
 
全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】
全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】
全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】
andagarcia212
 
Staffan Canback - The 18 Rays of Project Management
Staffan Canback - The 18 Rays of Project ManagementStaffan Canback - The 18 Rays of Project Management
Staffan Canback - The 18 Rays of Project Management
Tellusant, Inc.
 
TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...
TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...
TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...
David Schmidt
 
Family/Indoor Entertainment Centers Market: Regulation and Compliance Updates
Family/Indoor Entertainment Centers Market: Regulation and Compliance UpdatesFamily/Indoor Entertainment Centers Market: Regulation and Compliance Updates
Family/Indoor Entertainment Centers Market: Regulation and Compliance Updates
AishwaryaDoiphode3
 
Research Methodology, Objectives, Types and Significance of Research
Research Methodology, Objectives, Types and Significance of ResearchResearch Methodology, Objectives, Types and Significance of Research
Research Methodology, Objectives, Types and Significance of Research
indumathi967565
 
The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...
The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...
The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...
SOFTTECHHUB
 
Innovative Full Stack Developer Crafting Seamless Web Solutions
Innovative Full Stack Developer Crafting Seamless Web SolutionsInnovative Full Stack Developer Crafting Seamless Web Solutions
Innovative Full Stack Developer Crafting Seamless Web Solutions
Harwinder Singh
 
The Strengths and Weaknesses of Each Zodiac Sign
The Strengths and Weaknesses of Each Zodiac SignThe Strengths and Weaknesses of Each Zodiac Sign
The Strengths and Weaknesses of Each Zodiac Sign
my Pandit
 
Globalization strategy for Meesho with respect to Shopee
Globalization strategy for Meesho with respect to ShopeeGlobalization strategy for Meesho with respect to Shopee
Globalization strategy for Meesho with respect to Shopee
AsmitaSinghaRoy1
 
Expert International Executive Recruiters for Global Expansion
Expert International Executive Recruiters for Global ExpansionExpert International Executive Recruiters for Global Expansion
Expert International Executive Recruiters for Global Expansion
Alliance Recruitment Agency - Staffing Agency In California, Esplanade Avenue, Pacifica, CA, USA
 
DEKISH ELEVATOR INDIA PVT LTD Brochure.pdf
DEKISH ELEVATOR INDIA PVT LTD Brochure.pdfDEKISH ELEVATOR INDIA PVT LTD Brochure.pdf
DEKISH ELEVATOR INDIA PVT LTD Brochure.pdf
unosafeads
 
Mandated reporting powerpoint to help with understanding your role
Mandated reporting powerpoint to help with understanding your roleMandated reporting powerpoint to help with understanding your role
Mandated reporting powerpoint to help with understanding your role
khidalgo2
 
PETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAA
PETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAAPETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAA
PETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAA
lawrenceads01
 
TALENT ACQUISITION AND MANAGEMENT LECTURE 2
TALENT ACQUISITION AND MANAGEMENT LECTURE 2TALENT ACQUISITION AND MANAGEMENT LECTURE 2
TALENT ACQUISITION AND MANAGEMENT LECTURE 2
projectseasy
 
BeMetals Presentation_July 2 2024 .pdf
BeMetals Presentation_July 2 2024    .pdfBeMetals Presentation_July 2 2024    .pdf
BeMetals Presentation_July 2 2024 .pdf
DerekIwanaka1
 
Transforming the Future of Limo Services.pptx
Transforming the Future of Limo Services.pptxTransforming the Future of Limo Services.pptx
Transforming the Future of Limo Services.pptx
limocaptaincom
 
Credit Card Stats And Trends in 2024- Webpays
Credit Card Stats And Trends in 2024- WebpaysCredit Card Stats And Trends in 2024- Webpays
Credit Card Stats And Trends in 2024- Webpays
itio Innovex Pvt Ltv
 
Discover who your target audience is and reach them
Discover who your target audience is and reach themDiscover who your target audience is and reach them
Discover who your target audience is and reach them
Quibble
 

Recently uploaded (20)

Entrepreneurial mindset: An Introduction to Entrepreneurship
Entrepreneurial mindset: An Introduction to EntrepreneurshipEntrepreneurial mindset: An Introduction to Entrepreneurship
Entrepreneurial mindset: An Introduction to Entrepreneurship
 
Destor.One - Business Presentation Dec22.pdf
Destor.One - Business Presentation Dec22.pdfDestor.One - Business Presentation Dec22.pdf
Destor.One - Business Presentation Dec22.pdf
 
全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】
全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】
全国2024欧洲杯盘口正规平台-全球网络2024欧洲杯盘口平台 |【​网址​🎉ac10.net🎉​】
 
Staffan Canback - The 18 Rays of Project Management
Staffan Canback - The 18 Rays of Project ManagementStaffan Canback - The 18 Rays of Project Management
Staffan Canback - The 18 Rays of Project Management
 
TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...
TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...
TPH Global Solutions Overview: Successful Strategies for Selling to Mass Merc...
 
Family/Indoor Entertainment Centers Market: Regulation and Compliance Updates
Family/Indoor Entertainment Centers Market: Regulation and Compliance UpdatesFamily/Indoor Entertainment Centers Market: Regulation and Compliance Updates
Family/Indoor Entertainment Centers Market: Regulation and Compliance Updates
 
Research Methodology, Objectives, Types and Significance of Research
Research Methodology, Objectives, Types and Significance of ResearchResearch Methodology, Objectives, Types and Significance of Research
Research Methodology, Objectives, Types and Significance of Research
 
The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...
The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...
The AI-Powered Side Hustle Transforming Lives: A Dad's Journey to Financial S...
 
Innovative Full Stack Developer Crafting Seamless Web Solutions
Innovative Full Stack Developer Crafting Seamless Web SolutionsInnovative Full Stack Developer Crafting Seamless Web Solutions
Innovative Full Stack Developer Crafting Seamless Web Solutions
 
The Strengths and Weaknesses of Each Zodiac Sign
The Strengths and Weaknesses of Each Zodiac SignThe Strengths and Weaknesses of Each Zodiac Sign
The Strengths and Weaknesses of Each Zodiac Sign
 
Globalization strategy for Meesho with respect to Shopee
Globalization strategy for Meesho with respect to ShopeeGlobalization strategy for Meesho with respect to Shopee
Globalization strategy for Meesho with respect to Shopee
 
Expert International Executive Recruiters for Global Expansion
Expert International Executive Recruiters for Global ExpansionExpert International Executive Recruiters for Global Expansion
Expert International Executive Recruiters for Global Expansion
 
DEKISH ELEVATOR INDIA PVT LTD Brochure.pdf
DEKISH ELEVATOR INDIA PVT LTD Brochure.pdfDEKISH ELEVATOR INDIA PVT LTD Brochure.pdf
DEKISH ELEVATOR INDIA PVT LTD Brochure.pdf
 
Mandated reporting powerpoint to help with understanding your role
Mandated reporting powerpoint to help with understanding your roleMandated reporting powerpoint to help with understanding your role
Mandated reporting powerpoint to help with understanding your role
 
PETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAA
PETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAAPETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAA
PETAVIT MICHAEL TAY.pdfAAAAAAAAAAAAAAAAAAAA
 
TALENT ACQUISITION AND MANAGEMENT LECTURE 2
TALENT ACQUISITION AND MANAGEMENT LECTURE 2TALENT ACQUISITION AND MANAGEMENT LECTURE 2
TALENT ACQUISITION AND MANAGEMENT LECTURE 2
 
BeMetals Presentation_July 2 2024 .pdf
BeMetals Presentation_July 2 2024    .pdfBeMetals Presentation_July 2 2024    .pdf
BeMetals Presentation_July 2 2024 .pdf
 
Transforming the Future of Limo Services.pptx
Transforming the Future of Limo Services.pptxTransforming the Future of Limo Services.pptx
Transforming the Future of Limo Services.pptx
 
Credit Card Stats And Trends in 2024- Webpays
Credit Card Stats And Trends in 2024- WebpaysCredit Card Stats And Trends in 2024- Webpays
Credit Card Stats And Trends in 2024- Webpays
 
Discover who your target audience is and reach them
Discover who your target audience is and reach themDiscover who your target audience is and reach them
Discover who your target audience is and reach them
 

Flickr Services

  • 1. Flickr : Web Services Cal Henderson
  • 2. What is Flickr? Photo sharing website (flickr.com) The place to store digital photos The centre of a big distributed system A set of open APIs
  • 3. What the heck are ‘Web Services’? The future of the Internet!!!1 Really just buzzwords
  • 4. Web services in a nutshell Server Business Logic Interface Client Interface UI Transport
  • 5. Web services in a nutshell Server Business Logic Interface Client Interface UI Transport Web Server Web Browser HTTP
  • 6. Web services in a nutshell Server Business Logic Interface Client Interface UI Transport Web Server Application XML-RPC
  • 7. Web services in a nutshell Server Business Logic Interface Client Interface UI Transport Web Server Java Programmers SOAP
  • 8. Why should I care? You can avoid code reuse While offering multiple services…
  • 9. Web services Server Business Logic Interface Web Browsers HTTP Interface Interface Email Clients Email Web Apps REST Clients
  • 10. Web services Server Business Logic Interface Web Browsers HTTP Interface Interface Email Clients Email Web Apps REST Clients People get very excited about this part
  • 11. Ok, I get that bit Give me a real example! Aren’t you supposed to be talking about Flickr?
  • 12. Flickr’s Logical Architecture Page Logic Business/Application Logic Database Photo Storage API Logic Endpoints Templates Users 3 rd Party Apps Flickr Apps Node Service Flickr .com Email Parser
  • 13. Flickr’s Physical Architecture Web Servers Database Servers Node Servers Static Servers Users Metadata Servers
  • 14. But seriously… We only care about PHP! So where does Flickr use it?
  • 15. PHP is at the core of Flickr Page Logic Business/Application Logic Database Photo Storage API Logic Endpoints Templates Users 3 rd Party Apps Flickr Apps Node Service Flickr .com Email Parser
  • 16. Ok, ok – what besides PHP? Smarty for templating PEAR for XML and Email parsing Java for… Controlling ImageMagick (image processing) Storage metadata The node service MySQL (4.0 / InnoDb) Perl for deployment & testing tools Apache 2, Redhat, etc. etc.
  • 17. Medium sized application Small team (3 programmers until recently) 1 PHP, 1 Flash/DHTML, 1 Java >60,000 lines of PHP code >80 smarty extensions >60,000 lines of templates >250,000 users >3,500,000 photos >50,000,000 page views per month Growing fast Like, really fast So these stats are out of date by now
  • 18. Thinking outside the web app Services Atom/RSS/RDF Feeds APIs SOAP XML-RPC REST We love PEAR::XML::Tree
  • 19. More services Email interface Postfix PHP PEAR::Mail::mimeDecode FTP Uploading API Authentication API Unicode (Not really a service, but common to all Flickr services)
  • 20. Even more services Real time application The “node service” ‘Cool’ flash apps Which use the REST APIs Blogging APIs Blogger API (1 & 2) Metaweblog API Atom LiveJournal
  • 21. APIs are simple! Modeled on XML-RPC (sort of) Method calls with XML responses Named arguments (key/name pairs) Tricky in WebServices.framework on Mac OS X SOAP, XML-RPC and REST are just transports PHP endpoints mean we can use the same application logic as the website Endpoints talk to the business logic using PHP function calls Essentially a really fast transport
  • 22. XML isn’t simple :( PHP 4 doesn’t have good a XML parser PHP 5 is new and scares me (and it wasn’t out when we started) Expat is cool though (PEAR::XML::Parser) Why doesn’t PEAR have XPath? Because PEAR is stupid! PHP 4 sucks! Actually, PHPXPath rocks http://phpxpath.sourceforge.net/
  • 23. Creating API methods Stateless method-call APIs are easy to extend They don’t affect each other Adding a method requires no knowledge of the transport We just get passed arguments and return XML The transport layer hides all that junk Adding a method once makes it available to all the interfaces Self documenting – method dispatch requires a list of methods Because everyone hates writing documentation
  • 24. Red-Hot Unicode Action UTF-8 pages CJKV support It’s really cool
  • 25.  
  • 26. Unicode for all It’s really easy Don’t need PHP support Don’t need MySQL support Just need the right HTTP headers UTF-8 is 7-bit transparent Just don’t mess with high characters Don’t use HtmlEntities()! Or |escape in Smarty But bear in mind… JavaScript has patchy Unicode support People using your APIs might be stupid Some of them ARE stupid, guaranteed
  • 27. Scaling the beast Why PHP is great MySQL scaling Search scaling Horizontal scaling
  • 28. But first… Why do we need to scale? There are a lot of people on the Internet They all want to use our ���web services” Whether they know it yet or not
  • 29. Why PHP is great Stateless We can bounce people around servers Everything is stored in the database Even the smarty cache “Shared nothing” (so long as we avoid PHP sessions) But what this really means… …is we just have to deal with scaling elsewhere
  • 30. A MySQL Scaling Haiku Database server slow Load of over two hundred Replication wins!
  • 31. MySQL Replication But it only gives you more SELECT’s Else you need to partition vertically Re-architecting sucks :(
  • 32. Looking at usage But really, we SELECT much more than anything else A snapshot says SELECT’s 44m INSERT’s 1.3m UPDATE’s 1.7m DELETE’s 0.3m 19 SELECT’s for each IUD
  • 33. Replication is really cool A bunch of slave servers handle all the SELECT’s A single master handles IUD’s We can scale horizontally, at least for a while.
  • 34. Searching A simple text search We were using RLIKE Then switched to LIKE Then disabled it all together
  • 35. FULLTEXT Indexes FULLTEXT saves the day! But they’re only supported on MyISAM tables And we use InnoDb for locking We’re doomed :(
  • 36. But wait! Partial replication saves the day Replicate the portion of the database we want to search But change the table types on the slave to MyISAM It can keep up because it’s only handling IUD’s on a couple of tables And we can reduce the IUD’s with a little bit of vertical partitioning
  • 37. JOIN’s are slow “ Normalised data is for sissies” Erm, “ Selective de-normalisation can be a big win” Keep multiple copies of data around Makes searching faster Have to ensure consistency in the application logic For instance, have a concat’d field containing a bunch of child-row data, just for searching.
  • 38. Our current setup Slave Farm DB1 Master IUD’s SELECT’s Search Slave Farm Search SELECT’s DB3 Main Search slave DB2 Main Slave
  • 39. Our current, current setup Main Cluster Aux Cluster Search Cluster Slave Farm DB1 Master IUD’s SELECT’s Search Slave Farm Search SELECT’s DB3 Main Search slave DB2 Main Slave Slave Farm DB4 Master IUD’s SELECT’s DB5 Main Slave
  • 40. Horizontal scaling At the core of our design Just add hardware! Inexpensive Not exponential Avoid redesigns/re-architectures
  • 41. Talking to the Node Service Just another service with an API But just internal at the moment Everyone speaks XML (badly) Just TCP/IP - fsockopen() We’re issuing commands, not requesting data, so we don’t bother to parse the response Just substring search for state=“ok” This only works for a simple protocol
  • 42. Still talking to the Node Service Don’t rely on it! Check the connection was established Use a connection timeout Use an IO timeout!
  • 43. RSS / Atom / RDF Different formats (all quite bad) We’re generating a lot of different feeds Abstract the difference away using templates No good way to do private feeds. Why is nobody working on this? (WSSE maybe?) Most of the feed readers (including bloglines.com) support basic HTTP Auth Easy to implement in PHP We love PHP It’s great!
  • 44. Receiving email We want users to be able to email photos to Flickr Get postfix to pipe each mail to a PHP script Parse the mail and find any photos Cellular phone companies hate you Lots of mailers are retarded Photos as text/plain attachments Segments out of order No mime types UUEncoded and mime-less
  • 45. Processing email PEAR to the rescue Mail::mime_decode With some patches UUEncoding Relax the address atom parser We need to convert character sets ICONV loves you
  • 46. Upload via FTP PHP isn’t so great at being a daemon PHP4, I mean. Maybe PHP 5 is great Leaks memory like a sieve No (easy) threads Java to the rescue Java just acts as an FTPd and passes all uploaded files to PHP for processing This isn’t actually public Not my idea Bricolage does this I think. Maybe Zope?
  • 47. Blogs Why does everyone loves blogs so much? Only a few APIs really Blogger Metaweblog Blogger2 Movable Type Atom Live Journal
  • 48. It’s all broken Lots of blog software has broken interfaces It’s a support nightmare Manila is tricky But it all works, more or less Abstracted in the application logic We just call blogs_post_message(); And so can you, via the API
  • 49. Back to those APIs We opened up the Flickr APIs a few months ago Programmers mainly build tools for other programmers We now have Perl, python, PHP, ActionScript, XMLHTTP, .NET, Objective-C, C++, C and Ruby interface libraries But also a few actual applications
  • 52. iPhoto Plugin We developed a Mac uploader But it wasn’t great A user developed an iPhoto plugin It was great APIs encourage people to do your work for you
  • 53. Flickr Carnivore Uses Carnivore PE Sniffs AIM traffic (amongst others) from the local net Calculates the most popular words of the moment Uses the Flickr API to display photos of those words It’s like a really invasive zeitgeist
  • 54. Flickr Tivo A Tivo app which uses Flickr photos Just Type in some tags And your TV becomes a “digital picture frame”
  • 55. So what next? Even more scaling PHP 5? MySQL 5? or NDB? Taking over the world
  • 56. Flickr : Web Services Cal Henderson
  • 57. These slides are online http://ludicorp.com/flickr/