XML £, € and Less Than (<) Symbols

[Salicar]

Hi All,

Not posted for a while but hoping someone can help me with this issue. We have the following setup.

Java Servlet which using Hibernate returns string via an intemediary Dom4J Document. The Dom4J Document is set to have UTF-8 encoding.

Openlaszlo 4.07 rendering the xml data into a LzText field.

We have the following issues.

1. Pound (£), Euro (€) and various other "unusual" characters render as the square symbol.
2. The < symbol in the database will cause all the text after itself to not render.

I have tried the following solutions for these problems.

a. Changed encoding on XML output to ISO-8859-1 - no success on 1 or 2
b. Changed xml output to put the # entries in for 1. - success however see I. below.
c. Changed xml output to put a double encoding on < symbol for 2. - success however see I.
d. SOLOed the the solution rather than keeping as proxy. - partial success the £ symbol renders but nothing else works still.

Notes

I. our solution has to potentially support a large amount of data over multiple users, i cannot afford for it to be changing "string/xml" data after the grab operation to encode it for display purposes.
II. putting the data into the database in its encoded form is also not valid as the same database is used by several disconnected systems and has to represent the true data.
III. banning certain characters is not possible as we have several sections which have to allow "free text"

Also does an LzText which simply shows text rather than be a simplified HTML renderer exist?

I have read several posts on these issues but nothing is working,

Thanks in advance

Richard

[hqm]

Can you post a small test example of your UTF-8 encoded XML file with the character data which is causing the problem?

The LzText field tries to display its text using a rudimentary form of HTML, so '<' chars need to be escaped at some point before setText is ultimately called on a text field.

If you could post a zip file attachment that would be best, otherwise the forum software will probably corrupt it.

You can email me directly at hminsky@laszlosystems.com, or file a JIRA bug with the example data attached, and send me a note.

If it is not possible to HTML-escape the text string, it may be possible to force the underlying flash TextField object to display data verbatim rather than interpreting it as HTML, but there would be some issues involved in making sure it then had the correct font style; I think the underlying TextField .setNewTextFormat( textFormat ) method would need to be set up before adding text.



Thanks

[hqm]

I made an xml data file like this

text.xml:

<foo><![CDATA[£, € and Less Than (<) Symbols£, € and Less Than (<) Symbols]]></foo>


And wrote code which escapes the pointy braces, like this


<canvas width="100%" height="80%" debug="true">
<debug fontsize="12"/>
<dataset name="ds1"><foo><![CDATA[£, € and Less Than (<) Symbols£, € and Less Than (<) Symbols]]></foo></dataset>
<dataset name="ds2" src="http:text.xml" request="true"/>

<class name="etext" extends="text">
<method name="applyData" args="v">
this.setText(LzText.prototype.escapeText(v));
</method>
</class>

<simplelayout/>
<text>cdata literal:</text>
<text id="dc"><![CDATA[£, € and Less Than (<) Symbols£, € and Less Than (<) Symbols]]></text>
<text>dataset literal:</text>
<etext id="d1" datapath="ds1:/foo/text()"/>
<text>dataset http:</text>
<etext id="d2" datapath="ds2:/foo/text()"/>

</canvas>


For back compatibility reasons, I just remembered, input text fields
are plain text only, they don't display HTML, so you could make a class
of text which subclasses input text but is disabled for user input,
and display your text in verbatim form that way:

<class name="plaintext" extends="inputtext" >
<attribute name="enabled" value="false"/>
</class>
<plaintext id="foo" width="400" datapath="ds2:/foo/text()"/>

[Salicar]

thanks for the quick reply. unfortunately i am stuck in meetings all day but i will put together a quick sample system early tomorrow and post it up. as well as trying your suggestions.

regards,

Richard

[Salicar]

Hi,

Thanks for the test application. I enhanced it to work with our back office system and within 2 minutes spotted our issue. Essentially I had failed to put character encoding on the HTTPResponse i.e. response.setCharacterEncoding("UTF-8").

Hopefully this will give assistance to other people who have similar issues however.

Again thank for the quick reply and help.

Regards,

Richard