LPS with XML native databases

[amjad_tayea]

Hi everyone,

I was just wondering how could you make LPS work with a XML native database using either the Apache Xindice (http://xml.apache.org/xindice/) or Tamino server (http://www2.softwareag.com/Corporate/products/tamino/default.asp)?

Thank you

Amjad

[antun]

Interesting... do you know if Xindice or Tamino have a built-in HTTP interface?

-Antun

[amjad_tayea]

Hi Antun,

I am not fully sure. The reason I am asking is because I'm thinking of the architecture of a web application with a collection of xml documents making up the xml back end. I want to be abale to query using one and/or a combination of xml documents like you do with a RDBMS. I know that Xindice and Tamino are designed to do that function which is similar to what Mysql or SQl server do with traditional RDBMS. Can LPS query through a combination of xml documents or is it restricted to work with one xml file at a time?

thanks for the help

Amjad

[d~l]

Bumping this old thread ..

I started experimenting with eXist (http://exist.sourceforge.net/index.html).


downloaded the eXist-1.0b2-build-1107.war (http://prdownloads.sourceforge.net/exist/eXist-1.0b2-build-1107.war).

changed the name to exist.war - note: all lower-case (not eXist.war) - Tomcat URL's are case sensitive and the links in the eXist documentation are all lower case (exist).

deployed exist.war into %TOMCAT_HOME%\webapps\

now eXist native XML database can be accessed from Laszlo apps

also starting to explore extension of laszlo<->eXist interface by using exist_PHPAPI (http://cvs.sourceforge.net/viewcvs.py/exist/exist_phpapi/)
Update: Note that exist_PHPAPI is no longer available .. see latest list of Third Party Tools at eXist site. e.g. PheXist or XML_DB_eXist.

Early days .. but this combination looks promising.

See reference to XML-RPC and SOAP in eXist Developer's Guide (http://exist.sourceforge.net/devguide.html).

Another native XML database is Berkeley XML DB (http://www.sleepycat.com) .. but eXist seems to be a bit easier to setup.

And there is also Tamino and Xindice as listed above.

[pcawdron]

Fascinating stuff, sounds really good... But a knucklehead like me still struggles to get this promising back-end service stuff running.

I've got a working copy of eXist on my computer and realise it runs it's own version of tomcat (like OpenLaszlo) and, as there should only be one instance of tomcat, they need to get together and play nicely.

I tried your solution, renaming eXist-1.1.1-newcore-build4311.jar as exist.jar and placing it in %TOMCAT_HOME%\webapps\ which for me is C:\OpenLaszlo Server\Server\tomcat-5.0.24\webapps\ but what next?

Nothing seems to have "deployed."

Shouldn't I now be able to run OpenLaszlo with
http://127.0.0.1:8080/lps-4.0.x/laszlo-explorer/index.jsp

and eXist with
http://127.0.0.1:8080/exist/index.xml

OpenLaszlo works fine, but shouldn't eXist run as it does when I run it normally (ie, go to the local home page, sandbox, etc)

I think it's really great OpenLaszlo is so easy for everyone else to work with. It seems it's a shallow learning curve for most. For me, it's like Everest. And it's not like I don't have experience with webapps, but I'm an old ASP/SQL Server guy struggling with all this java/tomcat stuff. Most of the "help" on the net makes it all look really easy, but when it doesn't work... it could be one little thing out of place, it could be a mountain of missed details, I'm just lost.

I really hope exist can work in tandem with OpenLaszlo as it'll make that learning curve just a little easier - like climbing Everest with oxygen :)

Thanks,
Peter

[sagipolley]

Hi Peter,

You are probably less of a knucklehead than me (me being an amateur programmer at best)...

You mentioned that you placed exist.jar in your webapps directory. What you actually need is exist.war (which you can also download from the eXist site).

HTH,

Martin

[pcawdron]

http://www.tvsquad.com/media/2006/03/homer-doh.jpg

[sagipolley]

Yeah, I have plenty of those moments too :).

[d~l]

Nice to see some other climbers on the mountain face!

To experiment with other tomcat webapps (like eXist.war and phpservlet.war) I moved away from the OpenLaszlo bundled tomcat server and deployed a fresh tomcat (current version installed is 5.5.20). I installed this in c:\tomcat-5.5.20.

I also installed Tomcat Manager to make deployment of new webapps a bit easier.

Now if *.war files are downloaded (such as openlaszlo dev kit - 57 MB), Tomcat manager can be used to deploy the webapp. Or the *.war can simply be dragged into webapps folder and tomcat restarted.

There is a risk to be aware of. If you deploy any *.war with the latest version of tomcat (from 5.5.17 upwards), then develop apps in openlaszlo-3.x\my-apps\ ..

if the *.war file is later deleted (undeployed) all the sub apps are deleted too.

I was caught out by this recently. This recent thread refers .. read the tomcat manual (see link). (http://forum.openlaszlo.org/showthread.php?t=8094)

So keep your developed openlaszlo, phpservlet, exist apps in separate doc_bases outside tomcat and create context files to point to the external doc_bases. And frequently backup all webapp developed files in tomcat (such as in openlaszlo-3.3.3\my-apps\*.*).

If you have PHP installed .. say c:\PHP-5.0.2 .. you can install OpenLaszlo + eXist + phpservlet as three separate tomcat webapps. So now you can access eXist via the PHP API.

OpenLaszlo <--> PHP <--> eXist

You can also try installing montag (http://montag.sourceforge.net) as a fourth webapp in tomcat to give another level of xml db abstraction.

then you have ..

OpenLaszlo <--> PHP <--> montag <--> eXist

but you might prefer taking a more direct route to the eXist summit. E2.

[sagipolley]

So keep your apps in a doc_base outside tomcat and create context file to point to this external doc_base. and backup files frequently.

Or keep all the webapp subdirectories in a version control system :).

[d~l]

version control system

That was the advice given in that thread discussing my data loss .. but I'm still figuring out how to use subversion plus Ant etc. Perhaps that should be included in docs?

The real point is it should not be necessary to keep openlaszlo-3.3.3.war (57MB) plus the unpacked war (another 57 MB) .. both .. in tomcat-5.5.20\webapps\ .. it takes up disk space.

Add in several other wars and used disk space in webapps folder mounts up.

In earlier versions of tomcat (4) I'm sure that I could delete the *.war after unpacking.

[pcawdron]

Opps... spoke too soon. I'm getting the following error, but as you're not eXist support, I don't expect you to be able to solve it. You may, however, be able to tell me if my suspicions are correct or not.

I suspect that the problem lies with OpenLaszlo using Tomcat Servlet rather than the full-blown Tomcat (or Apache). Is it that I need to install the full version and then add both eXist and OpenLaszlo WAR files to the webapps directories and run them from there instead of from the neat little package OpenLaszlo comes in?

Cheers,
Peter


Internal Server Error

Message: Could not find component (key [org.apache.excalibur.xml.xslt.XSLTProcessor/xalan])

Description: org.apache.cocoon.ProcessingException: Lookup of transformer 'xslt' failed at file:/C:/OpenLaszlo/Server/tomcat-5.0.24/webapps/exist/sitemap.xmap:863:81: org.apache.avalon.framework.component.ComponentExc eption: transformers: ComponentSelector could not access the Component for hint [xslt] (key [xslt])

Sender: org.apache.cocoon.servlet.CocoonServlet

Source: Cocoon Servlet

Request URI

index.xml

cause

org.apache.avalon.framework.component.ComponentExc eption: Could not find component (key [org.apache.excalibur.xml.xslt.XSLTProcessor/xalan])

request-uri

/exist/index.xml

[pcawdron]

You guys are too quick....

I'd only just finished my post and found the answer already above.

It's late Sunday evening over here in Australia, so this will be a battle for another evening :) I'll give a your suggestions a go tomorrow night.

Thanks for the help, very much appreciated.

Cheers,
Peter

[d~l]

Is it that I need to install the full version and then add both eXist and OpenLaszlo WAR files to the webapps directories and run them from there instead of from the neat little package OpenLaszlo comes in?

I would recommend a fresh installation of tomcat-5.5.20 (or latest version) into say c:\tomcat-5.5.20 (as in my earler reply).

It is likely that the bundled version is a subset of the full tomcat.

And it is easier to compare installations (yours and mine).

You can keep the earlier bundled OpenLaszlo installation intact .. just do not run it at the same time as the new installation since both will try to use port 8080.

When you are happy with the new installation you can migrate the old files.

[pcawdron]

OK... that was refreshingly smooth... I'm almost disappointed it worked so well and so easily *grin*

Seriously, if anyone in a similar position reads this post, I highly recommend they follow the suggestions made above. In summary...

* Download the latest version of tomcat (I installed to C:\tomcat )
* If you're running windoze set tomcat up as an automatic service

* Download the OpenLaszlo Dev Kit version (contains the war file)
http://www.openlaszlo.org/download#stable

* Download the latest version of the eXist war file from http://exist.sourceforge.net/index.html#download

* Rename OpenLaszlo-3.4.x.whatever to just plain old OpenLaszlo.war
* Rename exist-1-0-1.whatever to just plain old exist.war

*Dump the OpenLaszlo and eXist war files into C:\tomcat\webapps\

...and these links below should run nice and smooth...

http://localhost:8080/exist/index.xml
http://localhost:8080/openlaszlo

OK, this time I really am going to bed :)

Thanks again for all the help...

[d~l]

Rename exist-1-0-1.whatever to just plain old exist.war

I recommend deploying the latest version of eXist .. exist-1.1.1-newcore.war .. which is further down the eXist downloads page. Read the release notes for details of major changes.

Also go to the link Third Party Tools at eXist home page to download the PHP5 API (SOAP / RPC client).

There are two to choose from. But you will also need to install PHP-5.2.0 locally. And then phpservlet.war as another Tomcat webapp.

I'll post some integration notes (OpenLaszlo+PHP+PHP_API+eXist) later.

[pcawdron]

Thanks d~l...

Do you have any OpenLaszlo <-> PHP <->eXist samples you could post with the integration notes? After all, that's what this is all about, reading/inserting/deleting/updating server-side data stores.

Oh, if anyone's following all this and going to try the setup out themselves, don't forget, first of all, you need the latest JDK or nothing's going to work.

Thank's again, d~l, for lending a hand.

Cheers,
Peter

[d~l]

After installing exist-1.1.1-newcore.war as exist webapp and testing that it launches, you need to populate the XML database with examples for testing (e.g. query from OpenLaszlo via PHP client).

Launch the http://localhost:8080/exist/index.xml page and in left navigation menu go down to Administration sub-menu.

Click on "Admin" and logon using

username = admin
password = <blank>

this should now logon and display System Status.

...

Click on "Browse Collections".

Tick some examples to create a collection.

e.g.
shakespeare
examples

In "New Collection" field give your collection a name .. "test"

Click on "Create Collection" .. which creates /db/test/

...

Click on "Import Example Data" to import from the newly created collection /db/test/

...

Finally, after populating the database as above you can logout and now go to

http://localhost:8080/exist/examples.xml

to try out some examples.

...

There are examples of php code in the Third Party Tools which you can start using (without OpenLaszlo initially). Later you can call these PHP proxy clients from OpenLaszlo and send/receive XML.

I'll send examples of lzx code later.

[pcawdron]

Thanks.... playing with it now :)

[Squids]

I hope I'm not digressing to much from the subject of Laszlo, but I have been reasonably successful setting up the eXist and the phpservlet with tomcat which d~l described in this and previous threads (thank you d~l for the tips). However I have hit a brickwall when I try to use the third party tools to query eXist from php. I have sent a private message to d~l regarding this issue which I will reprint below.



Hello d~l. I have created the Tomcat-phpservlet-eXist set-up you have described in some of your previous threads. Up to this point, everything works fine. I am able to run xq scripts for eXist in the browser succesfully and also run PHP scripts from the phpservlet with no problems. The problem has come when I attempt to use some of eXists third party tools to run xqueries from PHP. I am using this http://query-exist.sourceforge.net/ tool to attempt to query the database via SOAP. I have been working on it for several days and I can not seem to make any headway.

I am trying to run the test.php (after putting it and the 'include' directory in my phpservlet directory) that came with PHExist, but it only returns me a blank white screen in the browser, no error messages or anything. The only changes I made to include/exist.php was to change the $wsdl variable (defined in include/exist.php) to point to my own location which is http://localhost:7222/exist-1.1.1-newcore/services/Query?wsdl

I was wondering if you have any experience with this class tool (http://query-exist.sourceforge.net/) and if you could offer me some advice. I realize this is a bit of a digression from the subject of Laszlo, but I don't know where else to turn as I can't seem to find a forum for eXist or for the third party tools

Any help would be very appreciated.

[d~l]

Squids,

I had to recover some lost scripts under development after deleting wars in my tomcat server (tomcat 4.x did not delete apps after deleting the *.war, but apparently tomcat 5.5.20 does delete all web app files on deleting *.war). So take note and keep backups, or locate your development files in a docbase outside tomcat. This risk applies to any deployed webapp (openlaszlo, phpservlet, exist).

But from my recovered (lost) clusters, I'll dig out these openlaszlo <--> PHP <--> exist codes and report back on use of PheXist API (PHP5 class only). What version of PHP do you have installed?

Meanwhile I know that pcawdron is producing a beginner's tutorial, based on simple REST API (not using PHP). Drop him a PM for a preview of this tutorial. This will be a good start for adding contributions.

I must use PHP5 since I am planning to parse some of the queried exist XML documents, for some of my experiments.

And I don't think this is a digression from OpenLaszlo topics since native XML databases such as eXist are a natural fit.

[pcawdron]

This is a bit of a work in progress, so feel free to add comments, but here's a tutorial that (hopefully) walks someone through how to set up OpenLaszlo with eXist XML using REST.

As with anything that's been packaged, there's always the risk that something is going to get unpackaged at the other end and be not-quite-right. If this is the case, please let me know so I can correct this asap.

Hope it works well.

Cheers,
Peter

[pcawdron]

d~l pointed out that the zip files don't bundle well into the HTML so here they are separated out...

If you want to jump right in and play with the code, download (and rename .war files)

* install tomcat to C:\tomcat.................................http://tomcat.apache.org/download-55.cgi
* drop exist.war into C:\tomcat\webapps.................http://exist.sourceforge.net/index.html#download
* drop openlaszlo.war into c:\tomcat\webapps..........http://www.openlaszlo.org/download#stable

then...
* unzip xquery directly into C:\Tomcat\webapps\exist\xquery (and not any other subfolder)
* unzip openlaszlo-tutorial into C:\Tomcat\webapps\openlaszlo\dev
* rename mybooks.txt to mybooks.xml and log into exist admin and make a new collection called tutorial and upload mybooks.xml

http://localhost:8080/openlaszlo/dev/t/mygrid.lzx

That should be it... I've skimmed over a few details you can find in the tutorial but that should be enough for most to jump start into things...

Cheers,
Peter

[pcawdron]

Also, attached is the HTML doc if that's of use to you... if you're using firefox switch to the ie rendering engine to get the comments to come up with a mouseover.

The very first time you run mygrid.lzx and you click on "What's in the lounge?" or "What's in the Study?", you may wait up to 5-10 seconds. Once the grid's initialized you can switch datasets quite quickly (less than a second) but that first load seems to take some time... I think its related to OL not exist...

Cheers,
Peter

[d~l]

Squids wrote .. I can't seem to find a forum for eXist or for the third party tools

Here is the link to eXist forum (http://www.nabble.com/eXist-f4072.html)

Search for "openlaszlo" or "laszlo" or "php" and there are a few threads.

[Squids]

I am using PHP version 5.21. Thank you both for the help and tutorials, I really appreciate it. I have not even attempted to use REST yet, I just assumed I would be using SOAP calls from PHP. Is there an advantage to using REST over SOAP in some instances? The only difference I can tell is that perhaps REST is slightly less server- intensive because the user doesn't need to run a php script or log in and out of the database everytime he or she needs to pull data from it like when using SOAP.

d~l, isn't that crazy about tomcat deleting the entire directory when undeploying? I found that out the hard way too. Tomcat doesn't warn you or anything. I wonder what the Tomcat people were thinking with that little 'feature'.

Anyway, getting this PHExist class to work is driving me a tiny bit bazongas. I think the test script that comes with PHExist stops working for me at the line

try
{

$db = new eXist();

I tried printing out a test string before that line, and it prints out fine. I try to print out a string immediately after that line and it doesn't print. So I am thinking there is something thats causing a crash or something caused by that line, because I don't think anything after it is being parsed. I have gone through include/exist.php again and again but I don't see anything that should cause a problem.

Does the fact that I need to reference http://localhost:7222/exist-1.1.1-newcore
instead of http://localhost:7222/exist cause a problem? I think this has something to do with the environmental variable EXIST_HOME but I don't know how to set it because the instructions in the eXist documentation are for Jetty, not Tomcat.

[d~l]

Does the fact that I need to reference http://localhost:7222/exist-1.1.1-newcore
instead of http://localhost:7222/exist cause a problem?

I think not, since I am seeing the same blank page in Apache+PHP server. I'll have to do some more testing (might be some setting needed in php.ini).

Incidentally, you refer to includes/exist.php.

I hope you mean includes/eXist.php .. since remember that tomcat is case sensitive.

More later after testing PheXist. I might try xdebug and Fiddler to inspect the session.

[d~l]

Squids

In php.ini have you set [soap] extension as shown here (http://devzone.zend.com/node/view/id/689)?


[soap]

soap.wsdl_cache_enabled = "1"
; enables or disables WSDL caching feature

soap.wsdl_cache_dir = "/tmp"
; sets the directory name where SOAP extension will put cache files

soap.wsdl_cache_ttl = "86400"
; (time to live) sets the number of second while cached file will be used
; instead of original one


Try some of these SOAP client/server examples at this Zend site.

[d~l]

I'm starting to use Xdebug (http://www.xdebug.org/install.php) and WinCacheGrind client (http://sourceforge.net/projects/wincachegrind/) to inspect PHP apps.

Inspecting cachegrind.out (files dumped into c:\tmp) in WinCacheGrind GUI

I see
include ..
followed by eXist->_destruct

After installing Xdebug dll into

c:\php5.2\ext\php_xdebug-2.0.0rc1-5.2.0.dll

and creating tmp folder in c:\tmp

place these settings at the bottom of php.ini .. then restart server.

If phpinfo() is then run, Xdebug (if installed correctly) should be registered next to Zend logo in phpinfo() report.


; --------------------------------------------------------------------------------------
; xdebug
; http://www.xdebug.org/install.php

zend_extension_ts="c:\php5.2\ext\php_xdebug-2.0.0rc1-5.2.0.dll"
; --------------------------------------------------------------------------------------

; general settings
xdebug.auto_trace=On
xdebug.collect_includes=On
xdebug.collect_params=On
xdebug.collect_return=Off
xdebug.default_enable=On
xdebug.extended_info=On
xdebug.show_local_vars=On
xdebug.show_mem_delta=On
xdebug.max_nesting_level=100
xdebug.trace_format=Off
xdebug.trace_output_dir="c:/tmp"
xdebug.trace_options=Off
xdebug.trace_output_name=timestamp

; debugger settings
; xdebug.remote_autostart=Off
; xdebug.remote_enable=On
; xdebug.remote_handler=dbgp
; xdebug.remote_host=127.0.0.1
; xdebug.remote_mode=req
; xdebug.remote_port=9000

; xdebug.idekey=tim

; profiler settings
xdebug.profiler_append = Off
xdebug.profiler_enable = On
xdebug.profiler_enable_trigger= "Off"
xdebug.profiler_output_dir = "c:/tmp"
xdebug.profiler_output_name = timestamp

; --------------------------------------------------------------------------------------

[Squids]

Copy and pasted straight from my php.ini...


[soap]
; Enables or disables WSDL caching feature.
soap.wsdl_cache_enabled=1
; Sets the directory name where SOAP extension will put cache files.
soap.wsdl_cache_dir="/tmp"
; (time to live) Sets the number of second while cached file will be used
; instead of original one.
soap.wsdl_cache_ttl=86400

[pcawdron]

Squids,

Give the XML REST tutorial a try, I think you'll be pleasantly surprised...

SOAP, the Simple Object Access Protocol isn't all that simple to get working. REST is incredibly simple, in essence you're passing a URL with a bunch of parameters, that's it. It is no more complex than posting from an HTML form. If security is a concern, use HTTPS.

The challenging part was getting the XQL to manage the REST input and perform a request, insert, delete and update. But, for me at least, it means I've cut out a layer of complexity (PHP) and interact directly with eXist.

Let me know how you go....

Cheers,
Peter

[Squids]

Checking out the tutorials right now...

[pcawdron]

If you're viewing the HTML version of the tutorial with Firefox, switch your rendering engine to ie and the comments will appear with a mouseover.

[Squids]

Right now I am on the part in the tutorial concerning deleting and inserting. This is a very lucid and helpful tutorial, Peter. Thank you for creating and posting it. It would be cool if there was some place you could post this so more people would see it, like a wiki.

[pcawdron]

Squids,

If you can confirm that the code all works OK, I'll post to the OL wiki

Cheers,
Peter

[d~l]

An update on testing PheXist PHP5 class to access eXist via SOAP.


There was one serious oversight on my part ..

Each time I re-install PHP5.x I forget that the list of extensions in php.ini does reflect the full list of *.dll's in c:\php5.2\ext\

In particular, I had not included this in php.ini

extension=php_soap.dll;

I enabled this extension, restarted and checked that SOAP was listed in phpinfo() report

Then I tried again with PheXist.

.
But I hit another error .. this time in eXist

XQuery:

for $speech in //SPEECH[SPEAKER &= "witch" and near(., "fenny snake")] return $speech

java.rmi.RemoteException: query execution failed: XPST0003 : Ampersands (&) must be escaped (following character was not a name start character).


.
Searching the eXist forum I found .. this relevant thread (http://www.nabble.com/please-help%3A-XQST0003%3A-Ampersands-%28-%29-must-be-escaped-%28...%29-tf3187070.html#a8845635)


the bug fix is easy: just edit
src/orb/exist/soap/QuerySoapBindingImpl.java and remove or comment out
line 270:

xpath = StringValue.expand(xpath);

Afterwards, recompile the eXist jars by calling build.bat or build.sh.

So there is a bug in eXist SOAP .. this error when ampersand is in XQuery string does not show up in the eXist sandbox test.

.
I shortened the XQuery .

$query = 'for $speech in //SPEECH[SPEAKER and near(., "fenny snake")] return $speech';

and this time I received the correct response.



_________________________________

Why bother with PHP SOAP client when you can access eXist directly via REST (as pcawdron shows in his tutorial)?

PHP SOAP client allows the front end OpenLaszlo app to run in SOLO mode and access the full web service features of eXist on a local or remote tomcat server, including authentication.

But there are several ways of integrating OpenLaszlo + eXist; different mixes.

REST is a good start. SOAP is not easy.

....

So to conclude .. you can have

OpenLaszlo(proxied, REST) <--> eXist(REST)

OpenLaszlo(proxied) <--> PHP SOAP client <--> eXist(SOAP)

OpenLaszlo(SOLO) <--> PHP SOAP client <--> eXist(SOAP)

not tried REST or XML-RPC yet.

....

The more we test these variants the better.

[pcawdron]

I haven't tried REST to eXist in SOLO mode yet, might have a play with that tonight. I would suspect it would work as, from the OL side of things, it's just a dataset, nothing fancy...

I'll post the results...

Cheers,
Peter

[d~l]

To clarify, I'm testing OpenLaszlo (SOLO) on Apache+PHP5 server, with eXist as remote server (running as SOAP web service).

I expect that OpenLaszlo (REST) <--> eXist (REST) should work on the same tomcat server.

And Apache to Tomcat should work too, via REST protocol.

[pcawdron]

OK.... Solo REST XML works but....

Some of the timing is a little out in that if you insert/delete a record, the refresh of the dataset will need to be delayed a little longer but the insert/delete works. As it stands you don't see the inserted record added or the deleted record removed until you click on the fetch buttons.

At the moment the timeouts are set quite low, something like 50-100ms, if this was increased to 500-800 (still less than a second) I think it will resolve the issue.

Updates work fine.

No problem with remote exist server so long as the crossdomain.xml is in place.

Cheers,
Peter

[d~l]

I've now had an opportunity to walk through the REST tutorial ...

Starting with exist.lzx .. I encountered an LzLoader warning .. some timeout issue


WARNING: «LzLoader#0| idle».returnData: «XML#1| constructor="[type Function]" __proto__="undefined"<resultset constructor="[type Function]" __proto__="undefined"><body constructor="[type Function]" __proto__="undefined"><bookcase constructor="[type Function]" __proto__="undefined" location="Study"><book constructor="[type Function]" __proto__="undefined" pk="4" publisher="American Christian Press" type="Hardback" author="Wierwille">The Word&apos;s Way</book><book constructor="[type Function]" __proto__="undefined" pk="5" publisher="Colins" type="Paperback" author="Compliation">One Brisbane, Many Stories</book><book constructor="[type Function]" __proto__="undefined" pk="6" publisher="Colins" type="Paperback" author="Crichton">Congo</book></bookcase></body><headers constructor="[type Function]" __proto__="undefined" /></resultset>» already loaded


Moving on to exist2.lzx this worked better since the XQuery is invoked onclick.

Then on to mygrid.lzx which worked fine other than a pause when making the XQuery (but see a few points below)


In Debug window
undefined font "Courier"
.
Click on "What's in the study?"
Click on "Insert Record"
New-Record added (takes 6 seconds)
Select New-Record
Click on "Delete Selected Record"
New-Record is not seen to be deleted from grid view.
.
Click again on "What's in the study?"
The New-Record is now seen to be deleted
.
Now click on "What's in the lounge?"
New-Record (with no key specifed) is still viewed in list
Cannot be deleted?
.

So this is an excellent test and tutorial of using REST protocol. Well done.

Issues seem to be to refresh grid view after a record is deleted.

And to investigate further the timeouts.

I'm wondering if a REST proxy (such as PHP script) might still be useful between OpenLaszlo and eXist to provide a buffer against such timeouts.

I'm still pursuing testing of SOAP.

[pcawdron]

d~l

Thanks for giving it a go...

Try changing the method waiting to....

<method name="waiting">
<![CDATA[
this.setAttribute("_timeoutCounter",this["_timeoutCounter"]+1)
if(this.xset.getLoadTime()==null)
{
if(this["_timeoutCounter"]>=100) // 100 * 200 is 20 seconds
{
var myMsg = new xqlMsg
myMsg.xqlWin.setAttribute('title','XQL Time Out Error')
myMsg.xqlWin.xqlMsgText.setAttribute('text','Pleas e inform your system administrator about this error')
myMsg.xqlWin.bContainer.bOK.setAttribute('visible' ,'true')
return
}
this.waitDelegate = new LzDelegate( this, "waiting" );
LzTimer.addTimer( this.waitDelegate, 200 );
}else{
this.setAttribute("_timeoutCounter",0)
this.waitDelegate = new LzDelegate( this, "fetchX" );
LzTimer.addTimer( this.waitDelegate, 700 );
}
]]>
</method>

That will cause OpenLaszlo to check every 0.2 of a second to see if eXist has responded to the XQL and then allow an extra 0.7 of a second before reloading the dataset. You could also try 0.5...

That should solve the problem.

The challenge is in knowing how to fine-tune the routine so the user isn't left waiting but to still allow time for the HTTP REST request to be processed and handled properly before invoking the refresh. I got turn around times of 0.3 seconds with my set up (OL & Exist running on the same instance of tomcat) which made everything pretty much instantaneous.

The update is nice in that the grid doesn't actually refresh from the underlying server-side dataset, it's just a client side update. The actual XQL update request is sent and forgotten, so that comes across as instantaneous to the user, it's the inserts and deletes that are troublesome.

I toyed with the idea of "faking" the inserts and deletes, meaning simply add a node to the client dataset for an insert at the same time as sending the insert XQL to the server. The problem is, the new record needs a primary key which must come from eXist or it can't be updated.

Deletes, though, could be handled in a virtual manner. Perform a two step process, remove the node from the client dataset (which would appear instantly) at the same time as the XQL request is sent to eXist (which might actually take a second or two) and then don't worry about refreshing the grid as they're both in sync anyway.

I'll have a play with the deletes, but the insert is going to take some savvy...

Perhaps insert the client side node but lock the node, not allowing any edits until the XQL returns the key. The time delay would be the same, but at least the user would see the new record and know the system hadn't hung.

I thought about running two XQL, one to get the next key to add to the client side dataset and the other to actually add the server side record. That would work in 99.99% of cases but if two users simultaneous inserted records there could be some inconsistency, with record keys getting mixed up, which is why I went with the simpler approach. Simple is generally better (and faster)

Your thoughts are definitely appreciated.

Cheers,
Peter

[pcawdron]

Oh.... that works well...

<method name="deleteX" >
<![CDATA[

var dp = this['_thisGrid'].getSelection();
var xPrimaryKeyValue = dp.xpathQuery(this.primaryKey)
var p=new LzParam();
if(this.XMLfile != "") {p.addValue("XMLfile",this.XMLfile, false)};
if(this.parentNode != "") {p.addValue('XMLnode',this.parentNode + this.childNode, false)};
if(this.primaryKey != "") {p.addValue('XMLkey',this.primaryKey, false)};
p.addValue('XMLkeyValue',xPrimaryKeyValue, false);
this.xset.setAttribute('src',this.XMLdelete)
this.xset.setQueryString(p)
this.xset.doRequest();

dp.deleteNode()
]]>
</method>

Drop the last two lines in the delete method and replace with dp.deleteNode() and the effect is instantaneous. Very smooth.

The insert is OK for me, nice and quick, but I can understand why it wouldn't always be that way because out of the three actions (update, delete & insert) it's the only one that's actually dependent on a return dataset.

Looking forward to hearing back from you.

Cheers,
Peter

[pcawdron]

Guys...

Thanks for your help fine-tuning the the tutorial. I've updated all the code (on the previous screens) and added some more diagrams. I'll post this under tips and tricks and start a new post for it...

Also, I'm off to London on business next week so you wont hear from me from Saturday (Australian time)

Cheers,
Peter

http://forum.openlaszlo.org/showthread.php?p=28238#post28238

[Squids]

I have gotten to Chapter 7 (Interaction with Laszlo) and I am trying to teach myself how the xql class that you've written works. To be perfectly honest, this part is currently way over my head...I didn't even know what a 'primary key' was. Sorry for being a little slow on the feedback.

[pcawdron]

Squids,

You could use anything as a primary key, essentially its just a unique identifier. There's no primary key structure as such in xquery so I've made one up by simply grabbing the highest number in that attribute and adding one (so it works like an auto-number automatically assigning the next number in the sequence). It will just keep counting and counting and counting.

A word of warning, if you ever add a node without the primary key attribute or with a non-numeric attribute in the primary key, the eXist max function will fail and the you won't have any unique way of identifying which fields to update or delete, etc. So long as you enter new nodes using the XQL class in Open Laszlo you'll be fine. If you do add one by accident, you've just got to manually delete it and everything will return to normal.

I highly recommend you grab the latest files and latest version of the tutorial as I've corrected a couple of bugs and added some diagrams explaining what's happening between Open Laszlo and eXist during the updates & deletes, etc.

Hope this helps,

Cheers,
Peter

[d~l]

Squids wrote ..

I didn't even know what a 'primary key' was ..

These links might give more insights into native XML database usage ..

http://www.rpbourret.com/xml/

http://www.rpbourret.com/xml/XMLDBLinks.htm

http://www.rpbourret.com/xml/XMLAndDatabases.htm

See 6.3 - "What is a native XML database" (http://www.rpbourret.com/xml/XMLAndDatabases.htm#nativedefinition)

See 6.3.3.8 - "Indexes" (http://www.rpbourret.com/xml/XMLAndDatabases.htm#nativeindex)

See 6.3.4.2 - "Referential Integrity" (http://www.rpbourret.com/xml/XMLAndDatabases.htm#nativeintegrity)

[Squids]

Again I apologize for the lapse of time between my comments...I had to put working on database on hold for a bit while I ported my application over to the dhtml runtime. Right now I feel a bit overwhelmed, partly because data requests seem to be broken when a dhtml application is embedded in an html page. I need to find out what is wrong with this before I can continue. I am making a post in the 4.0 forum about this. Thanks for your patience.

[pcawdron]

Squids... I've expanded the tutorial examples to include the ability to add, delete, update, insert and restore previously deleted records, allowing an Open Laszlo client to manager server-side XML

Check out...

http://forum.openlaszlo.org/showthread.php?p=28909#post28909

[Squids]

That looks very nice. I am going over the comments in xql.lzx right now.

The tutorial and classes you've written are really great. I think they should be included with the Laszlo distribution as options for a back end.

Some additional points: I can't find any info on how to setup a standard user management system (user account creation, login and password) with exist. I would love to know how best to set this kind of thing up and I have questions such as 'Should their be a separate XML document for every user?" Everything I find online pertains to MySql. Do you know of any tutorials for this?

Also, a tutorial on how to handle file uploads with Exist would be great.

Anyway, thank you very much for posting your work on this stuff, it's filled a huge and important void as far as Laszlo development goes IMO.