In episode 2, Douglas Crockford discovers JSON
/bb|[^b]{2}/
Never stop Grokking
Showing posts with label json. Show all posts
Showing posts with label json. Show all posts
Tuesday, July 06, 2010
Sunday, October 08, 2006
Unintrusive dynamic script nodes
There are several methods of doing remoting via javascript. Dynamic script nodes are one of the easiest to use. Unlike XHR, which requires a response handler for most cases, dynamic script nodes require no javascript action after the initial call. The response from the server contains all logic that needs to be executed, and the browser takes care of executing it without interrupting the rest of your control flow.
They do come with two caveats however.
Dynamic script nodes also allow one to do cross-domain remoting without setting off sirens and flashing lights in the browser. This opens up an inherrent security problem. If you - as the developer of this application - do not have control over the source of your script, then you cannot trust that it will do nothing malicious. I'll skirt the issue in this article which concentrates on issue number two.
Using dynamic script nodes involves adding a <script> node to the DOM. For a simple application that just makes one or two calls, this isn't much of an issue, but for a complex application, the DOM can easily grow large, which starts to push memory limits of browsers.
An application that makes use of the flickr APIs or Yahoo! APIs that return JSON data suitably wrapped in a callback of your choice could hit these limits if built entirely in Javascript.
The call would be something like this:
These script nodes can be safely removed as soon as they've called their callback function. One needn't even wait for the callback to return, which means that the removal could be done within the callback itself.
While simple on the face of it, there's a bunch of housekeeping that goes with making this possible, and this isn't logic that all developers should be required to code into all their callbacks. It really should be generic enough to be separated out.
The code I came up with looks like this:
This code is moderately simplified from what it would be if it were truly generic. Additions that would need to be made include:
This is what I did for the flickr API:
And this is how you call methods on flickr:
PS: I didn't mention it earlier, but caveat number three is the maximum size restriction on GET requests. There's nothing that can be done to get around this since the browser is going to make a GET request. Just use this for API calls that don't send too much data to the server. You can receive as much as you want.
They do come with two caveats however.
Dynamic script nodes also allow one to do cross-domain remoting without setting off sirens and flashing lights in the browser. This opens up an inherrent security problem. If you - as the developer of this application - do not have control over the source of your script, then you cannot trust that it will do nothing malicious. I'll skirt the issue in this article which concentrates on issue number two.
Using dynamic script nodes involves adding a <script> node to the DOM. For a simple application that just makes one or two calls, this isn't much of an issue, but for a complex application, the DOM can easily grow large, which starts to push memory limits of browsers.
An application that makes use of the flickr APIs or Yahoo! APIs that return JSON data suitably wrapped in a callback of your choice could hit these limits if built entirely in Javascript.
The call would be something like this:
var scr = document.createElement("script"); scr.type="text/javascript"; scr.src = "http://api.flickr.com/services/rest/?method=flickr.interestingness.getList" + "&api_key=xxxxxxxxx&format=json&jsoncallback=myfunc"; document.body.appendChild(scr);And a large number of such calls will add a lot of <script> nodes to the document, all of which have already served their purpose and are no longer needed.
These script nodes can be safely removed as soon as they've called their callback function. One needn't even wait for the callback to return, which means that the removal could be done within the callback itself.
While simple on the face of it, there's a bunch of housekeeping that goes with making this possible, and this isn't logic that all developers should be required to code into all their callbacks. It really should be generic enough to be separated out.
The code I came up with looks like this:
// The global callbacks array that stores a reference to all // active callbacks. This array is sparse. var callbacks = []; function call_api_method(url, callback) { // We create the script element first so that it will // be available to the closure var scr = document.createElement("script"); // Now add our custom callback to the callback array // so that the added script node can access it var i = callbacks.length; callbacks[i] = function(json) { // first remove the script node since we no longer // need it document.body.removeChild(scr); // On failure display a generic error message if(json.stat == 'fail') alert("Error calling method: " + json.errorString); // On success, call the callback else callback(json); // Clear out our entry in the callback array since we // don't need it anymore callbacks[i] = null; delete callbacks[i]; // and clear out all outer variables referenced in the // closure to prevent memory leaks in some browsers i = scr = callback = null; }; scr.type="text/javascript"; // add our own callback function to the script url // the resource sitting at the other end of this url // needs to know what to do with this argument scr.src = url + '&callback=' + encodeURIComponent('callbacks["' + i + '"]'); // finally, add the script node to initiate data transfer document.body.appendChild(scr); }A few things stand out:
- The reference to the script node is held by the callback since it's a closure
- A global reference to the callback is necessary so that the script can access it
- We need to clean up in a particular order to avoid memory leaks caused by circular references
This code is moderately simplified from what it would be if it were truly generic. Additions that would need to be made include:
- Encapsulate the callbacks array and the method into an object so that we don't pollute global space.
- Instead of accepting a callback function, accept a callback object with separate methods for success and failure as well as callback arguments and execution scope.
- For most applications, the url would be similar for various calls differring only in small parts, eg: the method name in the Flickr API. It would be good if the above class were an abstract base class with specialisations for different services providing the base API url.
This is what I did for the flickr API:
var apikey = "xxxxxxxxx"; var apiurl = "http://api.flickr.com/services/rest?api_key=" + apikey + "&format=json&method=flickr."; var usernamemap = {}; function call_api_method(method, uname, callback, params) { var scr = document.createElement("script"); var cb = function(json) { document.body.removeChild(scr); usernamemap[uname].callback = null; if(json.stat == 'fail') alert("Error " + json.code + " calling flickr." + method + ": " + json.message); else callback(json, uname); scr = uname = method = callback = null; }; usernamemap[uname] = usernamemap[uname] || {}; usernamemap[uname].callback = cb; scr.type="text/javascript"; var url = apiurl + method + "&jsoncallback=usernamemap" + encodeURIComponent("['" + uname + "']") + ".callback"; for(var k in params) if(params.hasOwnProperty(k)) url += '&' + encodeURIComponent(k) + '=' + encodeURIComponent(params[k]); scr.src = url; document.body.appendChild(scr); }
And this is how you call methods on flickr:
call_api_method("people.findByUsername", uname, map_user_to_nsid, {"username":uname}); call_api_method("photosets.getList", uname, show_set_list, {"user_id":nsid});
PS: I didn't mention it earlier, but caveat number three is the maximum size restriction on GET requests. There's nothing that can be done to get around this since the browser is going to make a GET request. Just use this for API calls that don't send too much data to the server. You can receive as much as you want.
Tuesday, February 21, 2006
Rich, accessible pagination with unobtrusive javascript
I've seen a few questions about doing pagination with AJAX, and I don't like the thought of it. It smells of accessibility problems. I've addressed this issue before in my own toy pages (since I don't actually write production web code), so thought I'd share it.
This is a sort of continuation of my post on progressive enhancement.
For simplicity, I'll assume that we're representing our data as a <LI>st. A table is similar, except that you need to redraw the entire table since it's read-only in IE.
Step 1: Build HTML (PHP with MySQL)
Step 2: Attach onclick handlers
Step 3: Make async call:
Step 4: Modify back end to check for
I've hilighted the code that changed, it's just a bunch of if conditions. Yeah, it's ugly, but cleaning it up is not the purpose of this article.
Step 5: Make your
Step 6: Rewrite paging urls:
Steps 5 and 6 are the same function of course, so don't split them up.
If javascript is enabled, we prevent the default href from being called with our
The back end php script still hits the database as usual, and gets back a result set, which it now builds into a PHP hash, and then converts to a JSON object. The JSON object is sent back to the client where it is converted into HTML to push into the
The
This is quite a simple implementation. You could get really creative with javascript, showing funky page transitions that keep the user busy while your
Update: json_encode is available from the PHP-JSON extension available under the LGPL.
Update 2: The cleaner way to write the PHP code that I mentioned in Improvements above is something like this:
Update: I've put up a working example on sourceforge.
This is a sort of continuation of my post on progressive enhancement.
Problem definition
Given a long list of data, display it to the user in pages to avoid scrolling. Typically you'd have a bunch of navigation links at the end with First, Last, Next, Previous links, or links to specific pages.Six steps from vanilla HTML pages to AJAX pages
It's important to note that the problem definition does not mention AJAX, but people always like to make their apps buzzword compliant. So, forget about AJAX for the moment and concentrate on solving the problem — retrieve a database resultset in limited sized pages. Once we've done that, it's five more steps to accessible AJAXified pages:- Build the page as you would for html only pagination
- When your pagination links load, attach
onclick
handlers to them. - The onclick handler makes an
asyncRequest
tothis.href + '&js=1'
(or something similar) - Modify your backend code to check for
js=1
in the query string.
If not found, then send the entire page with header and footer as before
If found, then send one of the following:- The html for the paged data
- XML for the paged data
- A JSON object representing the paged data
- In your callback for the
asyncRequest
, do one of the following:- Put the html into the innerHTML of your page's container object
- Parse the XML and translate it to the DOM objects for your paged data
eval()
the JSON and redraw the DOM for the paged data
- Rewrite the hrefs in your paging links to point to new pages.
The Code
Let's look at some of the code. I'll use the yui utilities for connection and event management since I've been playing with that.For simplicity, I'll assume that we're representing our data as a <LI>st. A table is similar, except that you need to redraw the entire table since it's read-only in IE.
Step 1: Build HTML (PHP with MySQL)
<div id="page"> <ul> <?php // Fetch all results and print them while($o = mysql_fetch_array($result, MYSQL_ASSOC)) { ?> <li><?php print $o['name'] ?></li> <?php } ?> </ul> <?php // Verify next/last page links $prev_page = ($pg<=0?0:$pg-1); $next_page = ($pg>=$num_pages?$num_pages:$pg+1); // Display navigation links, disable (via css) links that cannot be selected ?> <p class="navbar"> <a id="first-link" href="foo.php?pg=0" class="<?php if($pg == 0) echo 'disabled' ?>">First</a> <a id="prev-link" href="foo.php?pg=<?php print $prev_page ?>" class="<?php if($pg == 0) echo 'disabled' ?>">Prev</a> <a id="last-link" href="foo.php?pg=<?php print $num_pages ?>" class="<?php if($pg == $num_pages) echo 'disabled' ?>">Last</a> <a id="next-link" href="foo.php?pg=<?php print $next_page ?>" class="<?php if($pg == $num_pages) echo 'disabled' ?>">Next</a> </p> </div>
Step 2: Attach onclick handlers
var nav_links = ['first-link', 'prev-link', 'next-link', 'last-link']; YAHOO.util.Event.addListener(nav_links, 'click', navigationHandler);
Step 3: Make async call:
var callback = { success: gotResponse, failure: failedResponse } var navigationHandler = function(e) { var url = this.href + '&js=1'; YAHOO.util.Connect.asyncRequest('GET', url, callback, null); YAHOO.util.Event.preventDefault(e); return false; }
Step 4: Modify back end to check for
js=1
:<?php$js = $_GET['js']; if($js) { header('Content-type: text/json'); } else {?> <div id="page"> <ul> <?php} $json = array('n'=>$num_pages, 'p'=>$pg, 'l' => array());// Fetch all results and print them while($o = mysql_fetch_array($result, MYSQL_ASSOC)) {if($js) { $json['l'][] = $o['name']; } else {?> <li><?php print $o['name'] ?></li> <?php}}if($js) { print json_encode($json); // nothing more to output, so quit exit(); }?> </ul>
I've hilighted the code that changed, it's just a bunch of if conditions. Yeah, it's ugly, but cleaning it up is not the purpose of this article.
Step 5: Make your
asyncRequest
handler:var gotResponse = function(o) { var json = eval("(" + o.responseText + ")") ; var list = json['l']; var num_pages = json['n']; var page = json['p']; var prev_page = (page<=0?0:page-1); var next_page = (page>=num_pages?num_pages:page+1); var lis=""; for(var i=0; i<list.length; i++) { lis += "<li>" + list[i] + "</li>\n"; } var ul = document.getElementById('page').getElementsByTagName('ul')[0]; ul.innerHTML = lis;
Step 6: Rewrite paging urls:
var fl = document.getElementById('first-link'); var pl = document.getElementById('prev-link'); var nl = document.getElementById('next-link'); var ll = document.getElementById('last-link'); var url = fl.href.substr(0, fl.href.indexOf('pg=')+3); pl.href = url + prev_page; nl.href = url + next_page; ll.href = url + num_pages; fl.className = pl.className = (page<=0?'disabled':''); nl.className = ll.className = (page>=num_pages?'disabled':''); }
Steps 5 and 6 are the same function of course, so don't split them up.
A brief explanation
Well, there you have it. If javascript is disabled, the default<A>
behaviour is to make a GET
request to foo.php
with default values for pg
. On every page call, the back end changes the value of pg
in the Next and Previous links, and possibly in the Last link if records in the database have changed.If javascript is enabled, we prevent the default href from being called with our
return false;
, and instead make the same call using asyncRequest
, but with an additional query parameter saying that we want a javascript (json) object back.The back end php script still hits the database as usual, and gets back a result set, which it now builds into a PHP hash, and then converts to a JSON object. The JSON object is sent back to the client where it is converted into HTML to push into the
<UL>
.The
page
and num_pages
variables allow us to rewrite the hrefs so that they point to up to date paging links, that you can, in fact, bookmark.Improvements
To make the code cleaner, you may want to build your PHP hash at the start, and then based on the value of$js
, either convert it to HTML or to JSON. This of course has the disadvantage of having to iterate through the array twice. If you're just looking at 20 records, I'd say it was worth it, and a better approach if you start off that way.This is quite a simple implementation. You could get really creative with javascript, showing funky page transitions that keep the user busy while your
asyncRequest
returns.Update: json_encode is available from the PHP-JSON extension available under the LGPL.
Update 2: The cleaner way to write the PHP code that I mentioned in Improvements above is something like this:
<?php $js = $_GET['js']; if($js) header('Content-type: text/json'); $list = array(); while($o = mysql_fetch_array($result, MYSQL_ASSOC)) $list[] = $o['name']; if($js) { $json = array('n'=>$num_pages, 'p'=>$pg, 'l' => $list); print json_encode($json); // nothing more to output, so quit exit(); } else { ?> <div id="page"> <ul> <?php foreach($list as $name) { ?> <li><?php print $name ?></li> <?php } ?> </ul> <?php // Verify next/last page links $prev_page = ($pg<=0?0:$pg-1); $next_page = ($pg>=$num_pages?$num_pages:$pg+1); // Display navigation links, disable (via css) links that cannot be selected ?> <p class="navbar"> <a id="first-link" href="foo.php?pg=0" class="<?php if($pg == 0) echo 'disabled' ?>">First</a> <a id="prev-link" href="foo.php?pg=<?php print $prev_page ?>" class="<?php if($pg == 0) echo 'disabled' ?>">Prev</a> <a id="last-link" href="foo.php?pg=<?php print $num_pages ?>" class="<?php if($pg == $num_pages) echo 'disabled' ?>">Last</a> <a id="next-link" href="foo.php?pg=<?php print $next_page ?>" class="<?php if($pg == $num_pages) echo 'disabled' ?>">Next</a> </p> </div> <?php } ?>
Update: I've put up a working example on sourceforge.
Labels:
accessibility
,
dhtml
,
javascript
,
json
,
lsm
,
php
,
progressive enhancement
,
yui
Labels
2fa
4.01-strict
404
accessibility
acer
airport wifi
algorithm
android
apache
API
att
audio
australia
authentication
ayttm
badges
bandwidth
bbc
bcp
berlin
bigsur
blog
blogger
blogger template
bof
book
boomerang
broken
bug
byte order
c
caching
chrome
closure
cmc
cms
codepo8
colours
comic strip
comments
communication
compile
conference
confoo
congestion-control
cookies
correlation
couchdb
cracker
crash
creative
credit card
crockford
cron
crux
csrf
css
curl
cwv
data tags
database
date
db
delicious
design
developer
dhtml
dns
docker
dom
dopplr
dragdrop
DST
dynamic script node
education
email
emotion
endianness
entities
ephemeral ports
epicondylitis
error checking
esmtp
everybuddy
extensions
facebook
favicon
fc9
fedora
fidelity
filesystem
firefox
firesheep
flickr
flot
form
forms
fosdem
foss
foss.in
freebsd
freedom
freestyle
ftp
function currying
gdb
geek
geo
george
gmail
gnome
google
gradient
groupon
hack
hacker
hardy
hash
howtos
htc
html
html5
http
i18n
icici
ie
iit
im
innerHTML
instant messaging
interfaces
internet
internet explorer
ios
ip
ip address
ipc
iphone
ipv6
iso8601
ISP
jabber
javascript
jslint
json
julia
keynote
latency
latex
LC_TIME
lcp
linux
load
localisation
login
lsm
luhn
MAC
macosx
mail
mathematics
mathjax
measurement
media queries
meetup
memory
messaging
microformats
missing kids
mobile
montreal
movable type
mp3
mvc
mysql
name generator
navtiming
network
nexus
nodejs
notes
opensource
opera
partition
passwords
pdf
perception
performance
perl
phone
php
planet
png
ports
prerender
printing
privacy
programming
programming style
progressive enhancement
psychology
puzzle
recovery
redhat
regex
regular expressions
resource timing
review
rfc2822
rfc3339
rhel
roundtrip
rss
rum
rwd
safari
sampling
scalability
scripting
search
secnet
security
sed
segfault
self extracting tarball
sendmail
seo
server
shell
shell script
sigdashes
site
slideshare
smtp
soap
sockets
spoofing
SQL
ssl
starttls
startup
statistics
stoyan
strftime
stubbornella
sydney
sysadmin
tablespace
talks
tcp
tcp-slow-start
testing
theme
thisisbroken
thisisfixed
thoughts
throughput
tim berners-lee
timezone
tips
toc
toy
transactions
twitter
two factor auth
typing
ubuntu
ui
unicode
unix
url
usability
ux
velocity
vint cerf
w3c
wav
web
web services
webcam
webdev
webdu
webkit
webperf
webtiming
whois
widgets
wifi
windows
workaround
write performance
X
xss
yahoo
ydn
YQL
yslow
yui
Translate this page
- PHOTO FEED
- Blog feed bluesmoon@Mastodon
- © PHILIP TELLIS
-
The other side of the moon by Philip Tellis is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.