Introduction
Web applications have made huge leaps and bounds in improving user experience thanks to a lot of recently developed fancy-pants technology. In an effort to help you take it up a notch, we’d like to share a method for helping your site anticipate a user’s next move.
On most sites, there are usually only a handful of options that can be done at any given moment and often some of these options are more likely than others. By determining what is important on each page, we can preload the data of the user’s next action and store it on the client with JSON. When the user decides to perform their next action, they’ll see their results almost instantly because the info was loaded in the background.
See It In Action
As always, we’ve created a little demonstration for you to observe, illustrate, and dissect. It’s a paging demo that shows how we can preload in the background the next and previous queries from a database. When the user pages through, they can do so quickly without having to wait for an Ajax call to finish.
Also note that this demo and the tutorial that follows use the Prototype.js framework along with our own custom database class. You’ll find both in the zip file above.
JSON
Before we get started on the paging, let’s take a quick look at JavaScript Object Notation, or JSON. JSON is a data interchange format, similar to XML, which is built using JavaScript. The easiest way to get your head around it is to imagine the format as a variable containing multiple arrays that can be nested endlessly. Implementing JSON in JavaScript is simple and the official documentation actually does a great job illustrating lots of examples. For this tutorial, we will be returning a string of JSON from the server, and that string will contain our preloaded data. The only tricky part is that once the data is fetched, we’ll have to convert it into a JavaScript object after the client receives it. To do that, we’ll use eval().
currentPage = eval('(' + response.responseText + ')');
The JavaScript line above makes use of the eval function, which converts the properly formatted string from the server into a JavaScript object. Once that is ready to go, the rest is easy. Now, if you’re not comfortable using JSON, you can always use XML instead. To compare the two, check out PPK’s review. That man never stops doing his homework.
Determining What to Preload
Once we know how to store our data, we’ll want to determine what kind of data we’d like to store. As far as I am aware, any amount of data can be stored on the client side. It is all dependent on the client computer (however, do correct me if I am wrong). Chances are, we don’t wish to crash the client, so the choice of and amount of data becomes important. For example, an online store with hundreds of thousands of products may not want to load every single product into client memory. Likewise, loading millions of records of data that need to be paged is also not ideal, so for this example we just set up a simple next and previous page to always be preloaded.
Getting Started
To set up a paging system, we need to know a few things: the total amount of records, the current page, the next page, the previous page, and the paging size. Starting from the initial page load, here is what needs to happen:
- Grab the total amount of records so we know how many total pages there might be.
- Grab the first page of data and display it to the user.
- Grab the next page of data and store it as JSON on the client.
No previous page of data is needed, since record 0 is always loaded first. Let’s take a look at each step individually.
Total Records
To get the total amount of records possible, we need to create a basic query that returns the value to the client.
function getRecordCount() {
$db = new database();
$sql = "SELECT COUNT(*) AS recordCount FROM Accounts";
$rs = $db->query($sql);
$row = mysql_fetch_array($rs, MYSQL_ASSOC);
echo $row['recordCount'];
}
This function simply gets the record count and prints the value. The following Ajax call retrieves the value:
function getRecordCount() {
var myAjax = new Ajax.Request(
'flow.php?action=count',
{
method: 'get',
parameters: '',
onComplete: function(response) {
recordCount = response.responseText;
}
});
}
The global variable, recordCount, is now initialized. Also, note that the parameter, action. in the querystring, is used to tell the server what action to perform. In this case, it told the server to return the record count.
Get Initial Data
Once the user first hits the page, we need to show them the first page of data. To do that, we have to do a few things. First, let’s look at the function that will draw out the HTML that the user sees.
function drawTable(page, contain) {
table = '<table>';
alt = '';
for(i = 0; i < page['players'].length; i++) {
table += '<tr class="'+alt+'">' +
'<td>' + page['players'][i].lastName + ',</td>' +
'<td>' + page['players'][i].firstName + '</td>' +
'<td>' + page['players'][i].position + '</td>' +
'</tr>';
(alt == '')
? alt = 'alt'
: alt = '';
} table += '</table>';
contain.innerHTML = table;
}
This function creates a table, and inserts it as the innerHTML of an element with the ID of container. You’ll notice when the table is drawing, it is looping through an object named page. The next step is to take a look at the page object. The page object is a string of data that the server returned to the client, and the client then converted into a JSON object. On the server side, the data is retrieved and displayed like this:
function getTableData() {
$ret = '{"players" :[';
$db = new database();
$sql = "SELECT * FROM Accounts LIMIT ".$_GET['current'].", ".$_GET['size'];
$rs = $db->query($sql); while($row = mysql_fetch_array($rs, MYSQL_ASSOC)) {
$ret .= '{ "firstName" : "'.$row['FirstName'].'", "lastName" : "'.$row['LastName'].'", "position" : "'.$row['Position'].'" }, ';
} $ret = rtrim($ret, ', ').']}';
echo $ret;
}
Then, using the eval() function mentioned earlier:
currentPage = eval('(' + response.responseText + ')');
We convert the server response into a JavaScript object that looks like this:
{ "players" : [
{ "firstName" : "Ryan", "lastName" : "Campbell", "position" : "S" },
{ "firstName" : "Chris", "lastName" : "Campbell", "position" : "QB" },
{ "firstName" : "Kevin", "lastName" : "Hale", "position" : "DT" }
]}
Once the object is created, the drawTable() function just loops through all players and displays their information. So now the user is viewing the initial data, and we wish to preload the next set of data. To do this, we only have to recreate what we just did, and not call the drawTable() function. Also, we need to store the response in a variable called nextPage instead of currentPage.
Paging
At this point, we have the user viewing the current page of data, and that data is stored in a JSON object named currentPage. We also have the next page of data stored in a JSON object named nextPage. So, when the user clicks the “next page” button, we want to do a few things:
function getNextPage() {
currentRecord += pagingSize;
Increase the current record by the paging size. If they were viewing records 0 - 20, and they hit “next page,” the current record will be increased by 20, making the next set 20 - 40.
showNavigation();
Explained below, this function hides and shows the previous and next buttons as necessary.
previousPage = currentPage;
We know the previous page will become the current page, since the current page is advancing one. No need to hit the server to get previous data—just change the variables.
currentPage = nextPage;
nextPage was preloaded into a JSON object, so we can now set the current page to the next page.
drawTable(currentPage, $('view'));
Draw the new currentPage to the screen.
getNextData();
}
Preload the new nextPage. This will work until the last page is hit. When a user clicks “previous page,” the same thing happens in reverse order.
Controlling the Navigation
The last thing to do is hide the “next page” button if the user is on the last page, and hide the “previous page” button if they are on the first. The following function should do the trick:
function showNavigation() {
(currentRecord == 0)
? $('previousLink').style.visibility = 'hidden'
: $('previousLink').style.visibility = 'visible';
((currentRecord + pagingSize) >= recordCount)
? $('nextLink').style.visibility = 'hidden'
: $('nextLink').style.visibility = 'visible';
}
Basically, this function just changes the style based on the currentRecord. If we’re at 0, then we’re on the first page. If the currentRecord equals the pagingSize (or total number of pages), then we’re on the last. Hide and show as deemed fit.
Improvements and Possibilities
Lately, I have found JavaScript to be fairly stable except when a user spams an action. The same problem applies here. If a user spams the “next page” button, the code may get thrown off. In order to prevent this problem, I sometimes find it necessary to create global variables, such as isActive, that prevents anything from happening until the variable is false. Other than that, everything should work out fine when preloading data.
I also mentioned earlier on that a store could use this technique. Right now, Ajax is fast, but it’s not instant since it still has to hit the server. By storing the current page of cart items on the client, you could add to the cart instantly and the user would have no wait while browsing through products. The store example and the example I used here both emphasize heavy sets of data, but it’s important to also realize that this can be used for trivial things like preparing the next set of HTML the user will see, or allowing for mass saving at the end of multiple actions rather than saving each individual action as it happens.
As always, we’d love to hear about some more possible uses of this technique, and suggestions for improving it are always welcome.
Very nice application Ryan. It ist pretty fast and ideal to compare datalists. Greetings from Berlin, Germany!
Perhaps rather than isActive the variable, each request could return the ‘page’ it was sent for. That way the buttons can keep functioning, users can jump back pages, but the JavaScript discards returned objects for a page that the site isn’t currently on? Otherwise it’s more like SJAX than AJAX. Well, SJAJO in this instance.
Great stuff, it could easily be adapted so that it could be unobtrusive and only added to the page if the user’s browser is Ajax Comptable.
Only thing I can see missing is a buffer - why not save some of the data gained from the database into an object? Then if the user browses back and forth there will be fewer database queries.
Great tutorial!
We tested JSON when we implemented new AJAX parts in our web application but went back to pure XML due to portability reasons.
Also, it can be dangerous to second guess the user. If there are more than 1 possible actions a user can execute, you could spend time preloading data that will never be viewed.
With this would come obvious scalability problems :-)
David: Good advice.
In this case, however, the cost is not very high, incurring one extra database hit for each time the user picks a direction by which to navigate the database (up or down).
Relapse: SJAJO? You’ve just pushed me over the edge to being a proponent of naming the concept rather than the technology! ;)
Sweet but i prefer XML :)
Ross - right now the buffer only holds 1 additional page of data. Depending on how much data each buffer contains, there could be more. The only thing to wathc out for is holding too much in buffers and potential crashing the browser.
David - I think each case will be different, and the programmer just has to realize the bandwidth problems. A site that is heavy on Ajax should have significant savings as is, and a technique like this may not cause too much of a problem compared to the bandwidth used by a traditional, non ajax site.
Rather than using “active” to cause the getNext/Previous to do nothing, you could implement fallback to non-AJAX behaviours. Have the next/previous controls make a boring old HTTP request to the server to get a page of data, starting with record X, along with Javascript overrides. If the next/previous page is in local cache, replace the data as usual and cancel the event. If not, let the event continue, and the server round-trip will slow the user’s spamming.
Also it’s probably worth noting that asking the server for twice as much data as you’re going to need is probably still a bandwidth saving over asking the server for exactly the data you need, but also all the HTML to present it, as you would get in a non-AJAX environment, depending on just how much data you’re displaying on a single page. And in my view, AJAX is about enhancing the user experience, not about saving on server costs. :)
Web applications have made huge leaps and bounds in improving user experience thanks to a lot of recently developed fancy-pants technology.
application is cool but I doubt about user experience and abaut that what you call “improving”. Everything is cool with those ajax apps untill user starts to use next and back buttons on its default UI (like brauser, mouse buttons aso.)
so the next huge leap could be the ajax integration with “next” and “previous” buttons of users default UI not within the UI in default UI
Ryan, very interesting article. JSON could really make things faster.
But you don’t want the count query at the beginning. This may be quite slow, for large tables. Or not?
You can use other techniqes to know wether to show/hide next/prev links.
For example: You can ask for one extra record in the queries. If you are quering next x records, on the server you will query for (x + 1) records. If you really receive x+ 1 records, then the next link should be visable. If not, you now you reached the end. You can then pass it as a parameter along with the other data.
Stoicho, it should be a relatively fast query, but if you were to run into problems with it then the method you suggested would work out better. The one thing I like more about your suggestion is that the user doesn’t have to wait for an additional query on page load. On the other hand, the server will be hit more looking for that last query. Good suggestion that is definitely worth looking into.
What about predicting where a user will click and preloading that page or link/data? do you know if there is something like that out there? maybe we can detect where the mouse is on the screen and preload any links in that area of the screen?
Thanks Ryan, this is great information, you saved me hours of work trying to solve this very issue! Although I love JSON and it’s certainly here to stay with the wide adoption by Yahoo and other companies, does anyone else feel it’s odd that XML was created specifically for this kind of purpose and yet data structures such as JSON are being used? Don’t get me wrong, I use JSON myself and understand it’s necessity regarding speed and CDN JavaScript, it just seems like a shame that XML isn’t always the best fit under certain conditions.
Great article, and idea, and nicely executed. Exactly what the database engines do (oracle,sql server) to optimize paging, pre-fetching. not to mention its been built into hard disk drivers as read ahead buffers for decades.
How come all this stuff seems so simple once someone executes it.
Great article, great information and very well written.
Thank you for the information.
A happy subscriber of Treehouse.
Um, how does a user “spam” a button? :-)
Also, this code can be simplified:
function showNavigation() { (currentRecord == 0) ? $(‘previousLink’).style.visibility = ‘hidden’ : $(‘previousLink’).style.visibility = ‘visible’; ((currentRecord + pagingSize) >= recordCount) ? $(‘nextLink’).style.visibility = ‘hidden’ : $(‘nextLink’).style.visibility = ‘visible’; }
to:
function showNavigation() { $(‘previousLink’).style.visibility = currentRecord == 0 ? ‘hidden’ : ‘visible’; $(‘nextLink’).style.visibility = currentRecord + pagingSize) >= recordCount ? ‘hidden’ : ‘visible’; }
Which in turn suggests a further improvement:
function showNavigation() { showIf( ‘previousLink’, currentRecord == 0 ); showIf( ‘nextLink’, currentRecord + pagingSize >= recordCount ); }
function showIf( id, show ) { $(id).style.visibility = ( show ? ‘visible’ : ‘hidden’ ); }
Ack, where is the Edit button?! :-)
That last version of the code should read:
function showNavigation() { showIf( ‘previousLink’, currentRecord > 0 ); showIf( ‘nextLink’, currentRecord + pagingSize
Try that again… No preview… No edit… Embarrassed commenter…
function showNavigation() { showIf( ‘previousLink’, currentRecord == 0 ); showIf( ‘nextLink’, currentRecord + pagingSize >= recordCount ); }
function showIf( id, show ) { $(id).style.visibility = ( show ? ‘visible’ : ‘hidden’ ); }
I give up. Did I mention I was looking for a Preview or Edit button? ;-)
This is the correct code:
function showNavigation() { showIf( ‘previousLink’, currentRecord > 0 ); showIf( ‘nextLink’, currentRecord + pagingSize
Right. No less-than sign. Let’s try the good old ampersand-lt-semicolon and see if that helps:
function showNavigation() { showIf( ‘previousLink’, currentRecord > 0 ); showIf( ‘nextLink’, currentRecord + pagingSize < recordCount ); }
function showIf( id, show ) { $(id).style.visibility = ( show ? ‘visible’ : ‘hidden’ ); }
Thomas Edison was right: It’s 10% inspiration and 90% perspiration!
That was entertaining for me to watch. The dialogue really added to it :) On a positive note, I like your improvement to the code — thanks for taking the time to get the post working.
The demo will only page once, after that clicking the links doesn’t do anything. I am not hitting the button again until the next page information has been loaded, so I don’t think that this is the ‘spam action’ problem.
I am OS: XP SP2 Browser: FF 1.5
Nice tutorial. Been working w/ Prototype, AJAX and JSON for a whole day today.
Don’t know what version of prototype you are using, but there’s built-in JSON support in mine (1.4).
First: If your server-side script responds with X-JSON: header, JSON object is created automatically and is passed as a second argument to onComplete().
I used the following code:
require('Services/JSON.php'); $json = new Services_JSON(); $jsonString = $json->encode($someVarOrArray); // doesn't work if you don't specify brackets header('X-JSON:('.$jsonString.')');(Services_JSON is a PEAR package, available here - http://pear.php.net/pepr/pepr-proposal-show.php?id=198, simplifies JSON object creation)
Same thing happens if your server returns Content-type: text/javascript, but in this case response body will be eval()-ed.
First method is much more interesting, as it allows you to return content AND JSON object.
Having the ability to return content along with an object is interesting. I’ll have to play around with it. Thanks for pointing that out.
Oh, that’s good, Ryan, I’m glad it was entertaining even if I did feel like an idiot. :-)
It was funny timing, because I’d just been working on some JSONP preloading code for my own scrollable list this morning when I ran across your article. So it was interesting and enlightening to compare approaches. Thanks!
Thanks, if it is possible, could someone share a link about paging in Ajax basicly without the libraries.
This would also be great as a calendar switching application.
It would be easy to “hide” the next and previous panes to also provide the perception of speed. Very nice application.
Good work!
Preloaded next and previous page is really great idea. Nice url, using PHP heh :)
Syntatic sugar - html code nicer when use http://bennolan.com/behaviour/
Ryan,
Nice tutorial and really good work. I helps me so much understanding ajax. I refined the web page and the paging script of your demo to allow adding record. Here is what I did: After the record is added, the callback function will refresh the view page to reflect the latest snap shot (another ajax request to complish this). However, the later ajax request for the table data returns old recordset rather than the new recordset. It is the same even though I page next and previous in an attempt to retrieve the new data from the database. The new record is not shown until I close and open a new browser session. It seems there is some caching going on but I don’t exactly sure about that.
Hi, Very good example. Here’s one thing i would like to do with Ajax, it is the preloading of one yes one jpeg (not multiple jpeg) with a progress bar, a true progress bar that count the data bytes per bytes. Please let me know.
There are a lot a progress bar for multiple preloading jpeg or gif files mostly small but a lot of them, it is not the same thing here.
Thank you very much!
A sneaky bug waiting to pounce.
Here’s perhaps a reason you’ve found some of your script unpredictable when a user “spams” it.
I just learned this recently myself, so no high-horsey antics here. Just good old-fashioned info:
You know that cute little “var” keyword you’ve seen sometimes in JavaScript? If you’re like me you might’ve thought it was some nasty alias for “variant”. Which would be bad right? We don’t need no stinkin’ variants!
Except that’s not what it means. “var” is actually the variable declaration keyword in JavaScript. So up there in those functions where you’re declaring variables? I bet you think those are scoped to the functions huh? Think again! They’re actually global variables!
In JavaScript you must declare a variable with the “var” keyword and within the scope of a function or block (like if, for, etc) to have it scoped to that block.
An example is worth a thousand words:
for(i = 0; i ' + message + ''); } }You’d probably expect this to repeat the numbers 0 through 9 ten times each on a page. You might be surprised what actually happens if you paste it into a test page however…
This version has the output you probably expected of the first however:
for(var i = 0; i ' + message + ''); } }So… yeah.
And Wufoo looks very nice. :-)
Hi Ryan,
Great tutorial… I’m slowing learning this type of development and tutorials like this really help.
A few questions… 1. I’m trying to get this working using php4 and a slightly older mysql db. (I can’t change the version used, hosted solution.) I’ve added some logging and I can see the flow.php->getTableData function printing results. However, the tables are always empty!
Is there a way to debug the AJAX calls ? i.e. I can not seem to figure out if the drawTable function ever gets called.
Thanks…
Ben,
I suggest downloading Firebug. It is a free Firefox plugin that can track Ajax requests and responses among other things. That should help with the debugging problems you’re having.
And yes, the $() function comes from prototype, and replaces document.getElementById.
You’re not supposed to eval json data, you’re supposed to parse json data and pass the json data object to a method of a class made to receive it. This increases security considerably.
i download paging code. but i am not able configre it properly so that i am able to see the initial data also. so plz tell me how to configure the code and how to use it?
IMHO JSON is a bad idea. Is uses JS memory space, whereas native XML DOM uses it’s own. It’s possible to blow the JS memory stack, and using XSLT is a more efficient way to use data that converting it to JSON and back again.
Dear sir ,
I have a problem with Json.
my perl code is;;
use JSON;
@list={ name => “vipin”, id => “01221” };
$js=objToJson(@list); print “$js”;
and my js is
…
data = eval(xmlhttp.responseText)
…..
how could i get the data from this expression?
Plz give me solution.
vipin
Hi Ryan,
Great tutorial!
One problem though… do you have a sample SQL to create the database schema you use in your example and some sample data to populate it with.
Cheers,
Mark
Something like:
Cheers.
Mark
/* MySQL Backup Source Host: localhost Source Server Version: 5.0.22-community-nt Source Database: test Date: 2006/08/18 14:32:17 */
SET FOREIGN_KEY_CHECKS=0; use test;
—-
Table structure for Accounts
—-
CREATE TABLE
accounts(AccountIDint(10) unsigned NOT NULL auto_increment,FirstNamevarchar(20) default NULL,LastNamevarchar(40) default NULL,Positionvarchar(10) default NULL, PRIMARY KEY (AccountID) ) ENGINE=InnoDB DEFAULT CHARSET=latin1;—-
Records for table Accounts
—-
insert into Accounts values (1, ‘Mark’, ‘Arrowsmith’, ‘DEV’), (2, ‘Ben’, ‘Gustavson’, ‘ITMAN’), (3, ‘Christine’, ‘Nitz’, ‘MAN’), (4, ‘Richard’, ‘Something’, ‘MD’), (5, ‘Lizzi’, ‘Lovinger’, ‘MULTI’), (6, ‘Claire’, ‘Jones’, ‘NA’), (7, ‘Dane’, ‘Thomson’, ‘PH’), (8, ‘Jack’, ‘Finn’, ‘TEMP’), (9, ‘Jessie’, ‘James’, ‘GUN’), (10, ‘Derrick’, ‘Deno’, ‘XX’), (11, ‘Rigel’, ‘Jones’, ‘CC’), (12, ‘Michael’, ‘Moron’, ‘VV’), (13, ‘Viginia’, ‘Madson’, ‘BL’), (14, ‘Abbey’, ‘Zuecks’, ‘A1’), (15, ‘Bernard’, ‘Wee’, ‘A2’), (16, ‘Chris’, ‘Shiflett’, ‘A3’), (17, ‘Dennis’, ‘Leary’, ‘B2’), (18, ‘Eddie’, ‘Murphy’, ‘C1’), (19, ‘Frank’, ‘Munson’, ‘D1’), (20, ‘Henry’, ‘Winkler’, ‘E3’), (21, ‘Irina’, ‘Solokov’, ‘C2’), (22, ‘Jim’, ‘Jones’, ‘T4’), (23, ‘Lisa’, ‘Smith’, ‘Y2’), (24, ‘Marie’, ‘Curie’, ‘TR’), (25, ‘Nolene’, ‘Holmes’, ‘T3’), (26, ‘Opec’, ‘Kemp’, ‘Y6’), (27, ‘Peter’, ‘Walters’, ‘Y7’), (28, ‘Ricky’, ‘Jones’, ‘J4’), (29, ‘Terry’, ‘Ng’, ‘S3’), (30, ‘Dean’, ‘Sparks’, ‘Q2’);
Everyone needs a hug.
Everyone needs a hug. So here is your hug. I have been comparing JSON and XML. JSON sounds simple but I see some security issues. I know the article is basically teaching me how to use JSON data and its not a platform to fume over the use/misuse/abuse of any technology but can you throw some light on advantages of JSON over XML
This is cool, been trying to set it up on a real application, the only thing missing here would be a function to go to certain page, lets say i wanted to go to page 4 out of 5 something like getPage(5), anyone knows how to do this ?
nevermind, i think i got it, not as fast as i would have liked it, but it works :)
Everyone needs a hug.
Everyone needs a hug.
Thank’s for your article. i hope it can help me
Everyone needs a hug. thank’s for all u give, more inovation…
Everyone needs a hug.
Can json be accesed dinamicaly? when using xml, you can access to element values (nodevalue) and element tags (nodename). ¿Can it be done whith json? In this example I see these values hardcoded: