CherryPy Caching: reprise

Finally I had to turn off caching, even with my fix in place. Users reported a malformed xml being returned for certain queries, and it turned out that anything longer than the maxobjectsize was being cached and returned truncated. I'm sure there's an easy way around this, but not sure I want to spend more time on it just now.

CherryPy Caching

I just tried turning on CherryPy web caching, this is one of the few things which receives only very cursory treatment in Sylvain Hellegouarch's  excellent book, 'CherryPy Essentials'. I managed by incorporating the 

conf = {'/':{'tools.caching.on':True}}

line in my code, however I was dismayed to discover that once on, I couldn't form a request would not get the cached result. Neither 'Cache-Control:max-age=0' nor 'Pragma: no-cache' would do it. Looking into the code (I'm using v. 3.1.1), I found the caching.py tool and indeed saw it pays pretty much no attention to the http request headers except (bizarrely, I think) the 'Vary' header. I've inserted a few lines into caching.py to fix this and submitted a trac bug, so we'll see whether this changes. The code lines are reproduced here:

#some people still use Pragma : no-cache to demand a fresh resource

        pragma_no_cache=False

        pragma_header=request.headers.get('Pragma')

        if pragma_header:

            pragma_values=[header.strip() for header in pragma_header.split(',')]

            for this_value in pragma_values:

                if 'no-cache' in this_value:

                    pragma_no_cache=True

                    break

        #added by shaun, look at cache-control

        max_acceptable_age=MemoryCache.delay

        age_seconds=int(response.time - create_time)

        cache_control_header=request.headers.get('Cache-Control')

        if cache_control_header:

            #split string on commas to get the multiple words

            cache_control_values=[header.strip() for header in cache_control_header.split(',')]

            #look for max-age

            for this_value in cache_control_values:

                if ('max-age' in this_value) and ('=' in this_value):

                    age_pair=[age.strip() for age in this_value.split('=')]

                    if age_pair[1].isdigit():

                      max_acceptable_age=int(age_pair[1])

                    break

                  

        #return if the cache is older than the acceptable age

        if (age_seconds > max_acceptable_age) or pragma_no_cache:

            request.cached = False

            request.cacheable = True

            return False

            

        # Add the required Age header

        response.headers["Age"] = str(age_seconds)

Python, libxml2 and run lists

So this last week I've been working on the run list as a web application. The front end is not so difficult, although I was requested to make the server side in Python; not my first choice... particularly as CERN's web server is not set up to use python as flexibly as PHP. So I run it as a python script in cgi-bin. This in turn uses my CherryPy server to retrieve xml. Then the fun begins: I am using libxml2 which is really not well documented. I'm compiling some notes on my wiki which might be helpful... difficult to believe that in the whole world there might be only a handful of programmers who have tried this.