Installed Erlang on OS X 10.6.3 Snow Leopard

I bought 'Seven Languages in Seven Weeks' and one of those languages is Erlang. The version on Fink is v.12, but the latest is version 13, so I decided to install it myself from source. I discovered the directory path for installation is not allowed to have a space in it, so I made a path with no spaces and downloaded there. First the download and unpack; OS X doesnt have wget by default, but curl is there:

curl 'http://www.erlang.org/download/otp_src_R13B04.tar.gz' -o 'otp_src_R13B04.tar.gz'

tar -xzvf otp_src_R13B04.tar.gz

curl 'http://www.erlang.org/download/otp_doc_man_R13B04.tar.gz' -o 'otp_doc_man_R13B04.tar.gz'

Create the directory for installation:

sudo mkdir -p /sw/erlang

Now configure, make and install:

cd otp_src_R13B04

./configure --prefix=/sw/erlang --enable-threads -enable-smp-support --enable-kernel-poll

make

sudo make install

Copy the documentation into place and extract it there:

cp ../otp_doc_man_R13B04.tar.gz /sw/erlang/

cd /sw/erlang

sudo tar -xzvf otp_doc_man_R13B04.tar.gz

sudo rm otp_doc_man_R13B04.tar.gz

Edit your ~/.bashrc to include the documentation and the new erlang binary in the appropriate paths; I use the 'pico' editor, but the lines you add are:

export PATH=$PATH:/sw/erlang/bin

export MANPATH=$MANPATH:/sw/erlang/man

...and source that script to get the new paths with . ~/.bashrc.

Now type 'erl' to get in the Erlang repl, and 'ctrl-C a' to exit.

W3C schema for the RFC 1123 date format

Here's a quickie I didn't find anywhere: XSD for the RFC 1123 date format, as used in http headers; note its only for GMT and years 2000 -2099 (my particular case):

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:annotation><xs:documentation>Schema for RFC 1123 date format type</xs:documentation></xs:annotation>

<xs:simpleType name="RFC1123_date">
<xs:annotation><xs:documentation>RFC 1123 date format.</xs:documentation></xs:annotation>

<xs:restriction base="xs:string">
<xs:pattern value="(Mon|Tue|Wed|Thu|Fri|Sat|Sun), [0-3][0-9] (Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) 20[0-9][0-9] [0-2][0-9]:[0-5][0-9]:[0-5][0-9] GMT">
</xs:restriction>
</xs:simpleType>
</xs:schema>

Python http PUT request

I have a CherryPy service which accepts 'PUT' requests with a named payload parameter (it's name is , surprisingly, 'payload'). From cURL, this is easy to specify with the -F option, but I couldn't find anywhere how to do this with Python. I discovered something a bit odd, which seems to be that the payload needs to be specified twice:

import urllib, httplib

import time, calendar

import base64

if __name__=='__main__':

timenow=time.gmtime()

t=time.asctime(timenow)

iov1=str(calendar.timegm(timenow))+'000000000'

iov2=str(calendar.timegm(timenow) + 1) +'000000000'

p=''+t+''

px=urllib.quote_plus(p)

pe=urllib.urlencode({"payload":p})

urlbase='putserver.cern.ch:8081'

testPut='DEVDB10/ATLAS_SCT_COMMCOND_DEV/PUTS200/TEST/PUT/Multi/timespan/'+iov1+''+iov2+"?payload="+px

test='/cooldb'

testMerge='/cooldb/merge'

print px

print urlbase

authHeader={"Authorization": "Basic "+base64.encodestring('name:pwd'), "Content-Length":str(len(px))}

#payload=px

h=httplib.HTTPConnection(urlbase)

h.request('PUT',testPut,px,authHeader)

r=h.getresponse()

print r.status, r.reason

print r.read()

h.close()

I wanted an iomanipulator to escape XML

...and so began a steep learning curve. I had some strings to insert into XML being output on the cout stream, and ideally I would simply like to do something like this:

string myXml("xml&<>");

cout<<escapexml<<myXml;

..and the output should be 'escaped'.

Of course I could have made a function and simply transformed one string into another, but I was also keen to learn about the iostream framework. Here are a few ways which iterate towards the final goal

1. 'Adapt' the string

Custom inserters can be made for a given class, so by wrapping the string in a class, I cold simply make an inserter for that class:

Here is the header:

class Wrapper{

std::string &m_data;

public:

typedef std::string::value_type value_type;

Wrapper(std::string & data):m_data(data){ }

std::string &str(){

return m_data;

}

};

std::ostream &

operator << (std::ostream & os, Wrapper & s);

...and the definition for that inserter:

std::ostream &

operator<< (std::ostream & os, Wrapper & s){

XmlEscape x(os);

for_each(s.str().begin(),s.str().end(),x);

return os;

}

where XmlEscape is a functor:

class XmlEscape{

static const char * m_chars;

static const char * m_replacements[];

static const unsigned int m_nchars=5;

std::ostream & m_os;

public:

XmlEscape(std::ostream & os);

void operator()(char c);

std::ostream & ostream();

};

with implementation:

const char * XmlEscape::m_chars ="&\'\"><";

const char * XmlEscape::m_replacements[]={"&amp;", "&apos;", "&quot;", "&gt;", "&lt;"};

XmlEscape::XmlEscape(std::ostream & os):m_os(os){

//nop

}

void

XmlEscape::operator()(char c){

unsigned int i(m_nchars);

while(i--){

if (c==m_chars[i]) {

m_os<<m_replacements[i];

return;

}

}

m_os<<c;

}

2. Adapt the stream

WSDL 2.0 and all that

I'm trying to make a directory of my web services to make them more visible (findable) and started with this:
Keith Chapman's Blog: RESTfull Mashup with WSDL 2.0 - WSO2 Mashup Server
and the referenced WSDL file he uses. I think he in turn uses the useful IBM page on this subject, and the WSDL primer, core definition and adjunct pages. Apart from that, there's not much out there on using WSDL for RESTful services.

In theory this could all also be done in RDF, or even a mixture of the two... this might provide both the functional definition in WSDL and the ontological framework ('find me all the atlas web services which give run information') in RDF. I'll start with WSDL, progress to UDDI and then try to backtrack and insert some RDF.

Sweetxml: My first WSDL 2.0 - generating Axis2 client code for a REST API

Sweetxml: My first WSDL 2.0 - generating Axis2 client code for a REST API

Safari 4: Does Acid pass the Acid test?

OK, so I updated to Safari 4 and yes, sure enough it passes the Acid3 test with flying colours. Must be good, then. So I tested it against my two outstanding bugs which I stumbled across:
Corrupt screen memory: Go to that link after visting another page and you'll see the new page just billboarded on top of the one you were looking at.



Try resizing the window to see ugly stuttering visual history as the window moves...



...and then there's simple (but big number) arithmetic inside XSLT. Safari just gets it wrong. Come on guys, where are those 64 bits you promised us? Heres an example of hex conversion, where the large integer here simply gets converted to zero inside safari, but the correct answer is given in Firefox.

So Acid 3 is all very well at testing the fancy new features, but I would have thought that CSS styling of Plain Old Xml would be even more fundamental than testing the new selectors.. so Acid 3 is failing my acid test, at least.

Parametrized Xslt from javascript, using prototype

Here's the final product; the original page has 'a' links in a tag with id="displayableModules". The links have the form href="BadChannelsFile.xml#12345.0" where the module id is indicated after the file. This is a kind of xpointer syntax, but I remould it so the id gives the module part and then listen for 'click' events on those ids; this causes an XSLT to be run with the id as a parameter and a new window contains the SVG which results. Along the way, I experimented with various file loading mechanisms (so I made a separate loading class) but stuck with a synchronous Ajax request, although discovered that Safari requires the method to explicitly stated as 'get' in lower case. Enjoy...

var Loader = Class.create({

  initialize: function(){

    this.result="";

  },

  

  loadXml: function(docPath){

    this.ajaxLoad(docPath);

  },

  

  ajaxLoad: function(docPath){

  //bind the passed function to the original class

  //the method (get) needs to be explicitly written in lowercase for the Safari browser!

    new Ajax.Request(docPath, {asynchronous: false, onSuccess: this._callback.bind(this), method: "get",

    onFailure: function(r){alert("load failed");}})

  },

  

  _callback: function(r){

    this.result=r.responseXML;

  },

  

  doc: function(){

    if (this.result==""){

      alert("no doc");

    }

    return this.result;

  }

});

var Transformer = Class.create({

initialize: function(){

  this.processor = new XSLTProcessor();

  this.paramNames=[];

  this.paramValues=[];

  this.xslPath="";

  this.xmlPath="";

  this.fileLoader = new Loader();

},

setXslt: function(xslPath){

  this.xslPath = xslPath;

  this.fileLoader.loadXml(xslPath);

  this.stylesheet = this.fileLoader.doc();

},

setXml: function(docPath){

  this.xmlPath = docPath;

  this.fileLoader.loadXml(docPath);

  this.xmlDoc = this.fileLoader.doc();

},

setParam: function(paramName, paramValue){

  this.paramNames.push(paramName);

  this.paramValues.push(paramValue);

},

exec: function(thisdoc){

   this.processor.importStylesheet(this.stylesheet);

  for (var i=0;i<this.paramValues.length;i++){

    this.processor.setParameter("",this.paramNames[i], this.paramValues[i]);

  }

  var fragment = this.processor.transformToFragment(this.xmlDoc, thisdoc);

  return fragment;

}

});

//global functions

function changeLinks(){

  //take existing links in the div 'displayableModules' and add an id, and suppress the href link

  var allLinks=$("displayableModules").getElementsByTagName("a");

  for (var i=0;i

    var thisElement=$(allLinks[i]);

    var thisElementUrl=String(thisElement.href);

    var moduleId="module_".concat(thisElementUrl.substring(thisElementUrl.lastIndexOf('#') + 1));

    thisElement.setAttribute("id", moduleId);

    thisElement.setAttribute("name", thisElementUrl.substring(0,thisElementUrl.lastIndexOf('#')));

    thisElement.setAttribute("href","");

  }

  listenToLinks();

 }

function listenToLinks(){

//take existing links in the div 'displayableModules' and listen to them...

var allLinks=$("displayableModules").getElementsByTagName("a");

  for (var i=0;i

    var thisElement=$(allLinks[i]);

    var thisModule=String(thisElement.id);

    thisElement.observe('click',transform);

  }

}

function transform(e) {

  var moduleElement=e.element();

  var thisModule=moduleElement.id;

  var sourceFile=moduleElement.name;

  var transformer = new Transformer();

  transformer.setXslt("./xsl/graph.xsl");

  transformer.setXml(sourceFile);

  transformer.setParam("id",thisModule.substring(7));

  w=window.open("",thisModule,"width=670,height=500");

  w.document.open();

  w.document.write('

Bad channels for module ' + thisModule.substring(7) + '

');

  f=transformer.exec(w.document);

  w.document.getElementById('insert').appendChild(f);

  w.document.close();

}

document.observe('dom:loaded', changeLinks);

Prototype and running XSLT from JavaScript

I had some problem marrying the usual method of running an XSLT with prototype's library. Here's my project: Given an xml file of 'bad channel identifiers' for a module (where channels run 0-767), display the bad channels as bars on a graph in SVG in response to a user clicking on another web page link somewhere. I chose to generate the SVG using a parametrised XSLT, but I need to pass the parameters...so I'll be running the XSLT 'manually' from JavaScript instead of using the stylesheet association.

Step 1. : Perform a transform manually using a loaded stylesheet and xml. I'll load both via an Ajax request... (this is not production code, but might help someone... and note that I already loaded the prototype library)

var Transformer = Class.create({

initialize: function(){

  this.processor = new XSLTProcessor();

},

setXslt: function(xslPath){

  //need to bind, otherwise 'this' refers to the event context, not the class

  new Ajax.Request(xslPath, {asynchronous: false, onSuccess: this._setXsl.bind(this),

  onFailure: function(r){alert("failure to load xsl");}});

  

},

_setXsl: function(r){

  this.stylesheet=r.responseXML;

  if (this.stylesheet==''){

    alert("Empty stylesheet");

  }

  this.processor.importStylesheet(this.stylesheet);

},

setXml: function(docPath){

  //need to bind, otherwise 'this' refers to the event context, not the class

   new Ajax.Request(docPath, {asynchronous: false, onSuccess: this._setXml.bind(this),

  onFailure: function(r){alert("failure to load xml");}});

},

_setXml: function(r){

  this.xmlDoc = r.responseXML;

  if (this.xmlDoc==''){

    alert("Empty xml doc");

  }

 },

exec: function(){

  var fragment = this.processor.transformToFragment(this.xmlDoc, document);

  return fragment;

}

});

function transform() {

var transformer = new Transformer();

transformer.setXslt("xsl/test.xsl");

transformer.setXml("xml/hello.xml");

f = transformer.exec();

$('insert').appendChild(f);

}

Java 'PUT' for RESTful service

I was looking around for a Java example of 'PUT' for  the CherryPy/COOL server, and at least came across a 'GET' at Jose Sandovals blog... but not much on the other methods. Andrea Formica (Atlas Muons) is trying to use it to upload calibrations. Here's the solution he came up with, using Apache's library:

public String coolPutCherryPyHttpClient() throws IOException, FileNotFoundException{

String res = "none";

HttpClient client = new HttpClient();

PutMethod put = new PutMethod(this.writenodeUrl);

    try {

String payloaddata = " 999999. "

+ "";

client.getParams().setAuthenticationPreemptive(true);

Credentials defaultcreds = new UsernamePasswordCredentials(this.name, this.psswd);

client.getState().setCredentials(new AuthScope(serverUrl, portNumber, AuthScope.ANY_REALM), defaultcreds);

NameValuePair par = new NameValuePair();

par.setName("payload");

par.setValue(payloaddata);

NameValuePair[] params = new NameValuePair[1];

params[0] = par;

put.setQueryString(params);

// Execute the method.

        int statusCode = client.executeMethod(put);

        if (statusCode != HttpStatus.SC_OK) {

          System.err.println("Method failed: " + put.getStatusLine());

        }

        // Read the response body.

        byte[] responseBody = put.getResponseBody();

        // Deal with the response.

        // Use caution: ensure correct character encoding and is not binary data

        res = new String(responseBody);

        System.out.println(new String(responseBody));

      } catch (HttpException e) {

        System.err.println("Fatal protocol violation: " + e.getMessage());

        e.printStackTrace();

      } catch (IOException e) {

        System.err.println("Fatal transport error: " + e.getMessage());

        e.printStackTrace();

      } finally {

        // Release the connection.

        put.releaseConnection();

      }  

return res;

}

Saxon XSLT 2.0 format-number problem (update!)

Later: following the post below, I managed to ask on the saxon-help list about this. The reply from Michael Kay (He Should Know!)

is that this is the expected behaviour. Looking in the W3C recommendation, it refers to the JDK 1.1 DecimalFormat class, which indeed states:

Symbol Meaning  

0      a digit  

#      a digit, zero shows as absent  

.      placeholder for decimal separator  

,      placeholder for grouping separator.  

;      separates formats.  

-      default negative prefix.  

%      multiply by 100 and show as percentage  

? multiply by 1000 and show as per mille  

¤ currency sign; replaced by currency symbol; if

         doubled, replaced by international currency symbol.

         If present in a pattern, the monetary decimal separator

         is used instead of the decimal separator.  

X      any other characters can be used in the prefix or suffix  

'      used to quote special characters in a prefix or suffix.  

...so , perverse as it seems, the behaviour is as advertised.

Original post:

One of the (many) advantages of using XSLT 2.0 is its supposed ability to understand scientific notation, e.g. 1E-2 is correctly interpreted as 0.01. However the only implementation I know of for XSLT 2.0 is Saxon, so the XSLT 2.0 standard basically defines this one application. Here's a nasty I discovered while trying to format exponential numbers using Saxon 9.1.0.6: Formatting zero with

format-number(xs:double($myNumber),'#.#######')

causes an empty string to be output.

Heres a sample xml:

<numbers>
<test> id="1">1.3</test>
<test> id="2">8E-4</test>
<test> id="3">8E-8</test>
<test> id="4">0.0</test>
<test> id="5">0</test>
</numbers>

and an xslt:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
exclude-result-prefixes="xs"
version="2.0">
<xsl:output method="text"/>
<xsl:template match="numbers">
<xsl:apply-templates/>
</xsl:template>
<xsl:template match="test">
<xsl:value-of select="concat('Original: ',text())"/>
Format with #.#####
'<xsl:value-of select="format-number(xs:double(.), '#.#####')"/>'
Format with #.#########
'<xsl:value-of select="format-number(xs:double(.), '#.#########')"/>'
</xsl:template>
</xsl:stylesheet>

But the output is not what one would expect:

Original: 1.3
Format with #.#####
'1.3'
Format with #.#########
'1.3'

Original: 8E-4
Format with #.#####
'.0008'
Format with #.#########
'.0008'

Original: 8E-8
Format with #.#####
''
Format with #.#########
'.00000008'

Original: 0.0
Format with #.#####
''
Format with #.#########
''

Original: 0
Format with #.#####
''
Format with #.#########
''

CherryPy Caching: reprise

Finally I had to turn off caching, even with my fix in place. Users reported a malformed xml being returned for certain queries, and it turned out that anything longer than the maxobjectsize was being cached and returned truncated. I'm sure there's an easy way around this, but not sure I want to spend more time on it just now.

CherryPy Caching

I just tried turning on CherryPy web caching, this is one of the few things which receives only very cursory treatment in Sylvain Hellegouarch's  excellent book, 'CherryPy Essentials'. I managed by incorporating the 

conf = {'/':{'tools.caching.on':True}}

line in my code, however I was dismayed to discover that once on, I couldn't form a request would not get the cached result. Neither 'Cache-Control:max-age=0' nor 'Pragma: no-cache' would do it. Looking into the code (I'm using v. 3.1.1), I found the caching.py tool and indeed saw it pays pretty much no attention to the http request headers except (bizarrely, I think) the 'Vary' header. I've inserted a few lines into caching.py to fix this and submitted a trac bug, so we'll see whether this changes. The code lines are reproduced here:

#some people still use Pragma : no-cache to demand a fresh resource

        pragma_no_cache=False

        pragma_header=request.headers.get('Pragma')

        if pragma_header:

            pragma_values=[header.strip() for header in pragma_header.split(',')]

            for this_value in pragma_values:

                if 'no-cache' in this_value:

                    pragma_no_cache=True

                    break

        #added by shaun, look at cache-control

        max_acceptable_age=MemoryCache.delay

        age_seconds=int(response.time - create_time)

        cache_control_header=request.headers.get('Cache-Control')

        if cache_control_header:

            #split string on commas to get the multiple words

            cache_control_values=[header.strip() for header in cache_control_header.split(',')]

            #look for max-age

            for this_value in cache_control_values:

                if ('max-age' in this_value) and ('=' in this_value):

                    age_pair=[age.strip() for age in this_value.split('=')]

                    if age_pair[1].isdigit():

                      max_acceptable_age=int(age_pair[1])

                    break

                  

        #return if the cache is older than the acceptable age

        if (age_seconds > max_acceptable_age) or pragma_no_cache:

            request.cached = False

            request.cacheable = True

            return False

            

        # Add the required Age header

        response.headers["Age"] = str(age_seconds)