Data pooling w/ paged results

[rcyeager]

Can anyone provide some guidance as to how to achieve data pooling for my app?

I currently receive XML transactions from a servlet that represent pages of results. I convert the XML string into dataset data, and store these as dataset arrays in my app, to facilitate client-side caching when paging through previously retrieved results.

There aren't many good samples of data pooling. All of them use a single datapath, like this:

<view name="all">
<datapath xpath="phonebook:/contacts/contact" pooling="true"/>
<view>
<simplelayout axis="x"/>
<checkbox width="30" datapath="@checked">
<method event="onvalue">
datapath.updateData()
</method>
<method name="updateData">
return this.value
</method>
<method name="applyData" args="d">
setValue(d)
</method>
</checkbox>
<text datapath="@firstName"/>
<text datapath="@lastName"/>
<text datapath="@phone"/>
<text datapath="@email"/>
</view>
</view>


See how a single phonebook dataset is used in the sample? For my app, I need to be able to point to various datasets as result pages are selected.

I tried modifying the datapath directly via script using setXPath() on the datapath to point at my various datasets, but that didn't seem to work. The first page of results rendered OK, but nothing appeared when I tried to change the datapath to the next dataset. My concern with this approach is that it would in some way make pooling nonfunctional. Anyone know?

My Plan B is to create a singleton dataset that the datapath will refer to exclusively, then perform the data swapping into and out of the singleton dataset. My theory being that a singular dataset may be required for pooling. The documentation doesn't spell this out one way or another...

Has anyone tried anything like this? I've looked at the base classes to see how they do pooling, but sample code seems to be lacking in this area.

Robert

[rcyeager]

And the answer is:

It seems to be necessary to have a singleton dataset attached to the pooled views. I swap data in and out of the singleton dataset and the views pool nicely.

This is sweet. Now in my development environment I can iterate through cached page results in a few hundred milliseconds using data pooling, vs. 1,500-2,000 milliseconds without pooling (the cost of destroying and instantiating the same views over and over). Basically, as fast as you click the prev/next buttons the next data set is rendered immediately. Ah, the joy of re-using instantiated views. :-)

OpenLaszlo continues to impress!!

Robert

[dsalama]

Could you post some functional code for educational purposes?

BTW, in the past, the way I've done it is to set the datapath with something like:

<view name="test">
<datapath />
</view>

then somewhere in your script

test.setDatapath( "phonebook:/contacts/contact_p1" );

or then later with

test.setDatapath( "phonebook:/contacts/contact_p2" );

Thanks,
Daniel

[rcyeager]

My website Cooqy at http://www.cooqy.com is going to be upgraded with the data pooling change in about 15 minutes from now, 1:45am EDT, if you want to see the performance for yourself.

I'm going to have some champagne...I've implemented data pooling in time for eBay Live! next week. My website absolutely roars with performance now that the data pooling is functioning! Paging back and forth through the search results really hums...about as fast as you can click the buttons the screens render, especially the List View.

I didn't initially build Cooqy to be faster than eBay's HTML website, but once I got into the technology I discovered what was possible. Now I think Cooqy smokes eBay in every way possible.

Will post some sample code soon, probably tomorrow (i.e. later today 6/7 after I get some sleep).

Robert

[dsalama]

Well, congrats on cooqy! Love to see it in action and would love to see some sample code as well.

Get some rest and good luck with the launch.

Thanks,
Daniel

[rcyeager]

Just completed the upgrade...give it a whirl if you like.

Robert

[dsalama]

BTW, I don't know if it's a bug or what.

I searched for an item which gave me 63 pages of results. I changed to list view. I clicked Next until I got to page 3. As soon as I click Next again and get to page 4, the result says I now have 62 pages. I can keep clicking Next to pages 5, 6, 7, 8 and shows 62 pages. However, if I click Prev back to page 3, 2, or 1, it shows again 63 pages.

I think somewhere in your caching system, it's showing stale data since items may no longer be there and page numbers may have changed while you scoll Next and Prev.

Sorry if this is going to keep you up later than you planned or if it's being fixed in the release you are about to upload.

- Daniel

[rcyeager]

Um, actually that piece of data comes from eBay. Ebay has suffered innumerable problems with their web services, especially for the last couple of weeks. Sometimes the performance on their end is just unacceptable, with even outright failures that require us to resend transactions until they complete.

We do cache previous pages of data into Cooqy. The caching introduces a margin of innaccuracy that we deem to be acceptable for performance sake.

As a true test of how this particular problem is really on eBay's end, try just going forward through the search results and witness how the number of pages keeps changing. This may not be a problem per se...could be representative of auctions that have ended.

Thanks for the heads up! Bedtime for Bonzo...

Robert

[rcyeager]

Here a rundown of the pattern I used for paging results in and out of a singleton dataset in support of pooling:


<class name="container" ...>
<attribute name="ds_searchResults" value="$once{new Array()}"/>

<dataset name="ds_masterSearchResults"/>

<handler name="oninit">
contents.theView.contentData.setXPath(this.ds_mast erSearchResults.getDSName()+':/.../*');
</handler>

<method name="updateView">
var dsPntr = this.ds_masterSearchResults.getPointer();
if (dsPntr.p.hasChildNodes())
{
do
{
dsPntr.p.removeChild(dsPntr.p.getFirstChild());
} while (dsPntr.p.hasChildNodes());
}

var dsPntr2 = this.ds_searchResults[ndx].getPointer();
if (dsPntr2.p.hasChildNodes())
{
dsPntr.p.appendChild(dsPntr2.p.getFirstChild());
}

dsPntr.destroy();
dsPntr2.destroy();
</method>
</class>


<view name="contents" ...>
<wrappinglayout/>
<view name="theView" ...>
<datapath name="contentData" pooling="true"/>
</view>
</view>


The important part of the technique is the dataset swap done in the updateView method. Otherwise, no special tricks were involved.

Note that the wrappinglayout has a horrendous performance penalty, in that it loops N! times through views, as in 10 * 9 * 8 * ... iterations, if you had 10 views. I discourage against using the wrappinglayout...in my case I created my own layout that has an O(n) loop...one time through the views.

Robert