MishMash

OS data to be made freely available?

Potentially big news, even if limited: some Ordnance Survey data (you know, the stuff the tax payer paid for, directly or indirectly) is to be made freely available. The free our data blog has some comments. Importantly:

Key points: it involves “mid-scale” maps from 1:10,000 upwards; and it kills off the “derived data” rows that government departments and everyone else has been having for so long. Derived data will have a stake through its heart.

Oh, and – this “free” will extend to being free for commercial use. That’s right, you’ll be able to build a business with it. Though it’s not clear yet whether you’d be able to take the maps and create *printed* ones. Must ask about that.

I hope that means it will become possible to obtain e.g. landranger mapping for the Thames tidal floodplain without making a dozen requests through Digimap. That was a painful process.

I suppose it also means that all of the wierd and arcane restrictions on use will be lifted (including different restrictions for printed and online papers, as far as I could interpret the Ts and Cs for digimap).

It's only a step, but it sounds like a significant step. It's significance goes beyond Ordnance Survey, too. It seems to me to undermine the argument that government requires that data be charged for. Other goverment agencies take note ...

Cameron Neylon, Capturing process

Just been at a seminar organised by the Comp Sci people here in Newcastle. Cameron Neylon, currently of the Science and Technology Facilities Council on:

Capturing process: The promise and challenges of connecting experimental records to the semantic web

The promise of the semantic web in its broadest terms to integrate structured data, and in particular scientific data, is breathtaking. Yet our ability to deliver on that promise remains severely limited. On one side, battles are fought over vocabularies and structured description languages, while on the other computer scientists argue over design and architecture. In the middle the experimental scientist is usually left bemused as to what all the fuss is about and how it can have any relevance to their research, which is naturally far too specialised and ground-breaking to fit into any framework that an outsider could ever provide. The challenge lies in finding a pathway that moves towards satisfying the needs and interests of these disparate groups while providing both value and a usable interface to the end user.

We have adopted a blog based approach to capturing the record of research in an experimental laboratory. The use of a free form framework along with templates encourages the user to structure the description of their work in a way that matches their needs. Cycles of template design and optimisation lead to structures with strong similarities, and occasional important differences, with designed ontologies and vocabularies. The challenge for the future lies in determining how best to guide the actions of users towards the use of appropriate controlled vocabularies and structured descriptions, while using their choices to inform the design of these systems. The potential for Google Wave to provide a framework to enable local structuring as well as integration with global structured descriptions will be explored.

Some interesting ideas, as abused by my memory:

• People tell stories, they don't organise stuff the way it needs to be organised for machines to process. A big part of the disconnect between scientists and the folks building "e-science" tools is that these tools don't help people tell stories.
• So we need to start capturing structure through storytelling. That is, allow people to work in a natural way and pick up the structure from what they do.
• You can create a virtuous circle between metadata, which a user would normally not bother filling in or fill in incorrectly, and user interface. Arrange the UI so that the metadata populates the UI. The user wants that UI to make their life easier, and they start to notice that getting the metadata right is necessary.
• (Lab) instruments as bloggers. Have them push results straight into the blog. Take the human out of the loop (I might qualify, as I think was intended: where they add no value).

Cameron has had early access to Google Wave, it seems, and is impressed with the potential. He had some cool examples of Wave "robots" going off and filling in extra information, drawing graphs, creating links, tidying up references.

He was less impressed by the Google client: too much stuff going on in a single UI, too non-standard. He also expressed concern about users understanding the model. It's the eternal problem with geeks developing stuff for "normal people" again. We're geeks because we do abstract with ease.

There was a lot that I recognise, and some new thoughts as well. I'm all about computational analysis in engineering decision making rather than lab-based science, so there are some differences, but there's a lot of overlap too. In theory we should have an easier time here since so much of our "stuff" comes in the form of information artefacts, and our "experiments" take place in silicon. There are instruments, and the process of getting data from them into our analysis is currently extremely painful, but most of that is "upstream" of our analysis.

I liked the picture of an lab process generating artefacts and narrative, and then a separate process of arranging these things into a different narrative for publication. The story we tell when we want to explain our results to others isn't the story of the trials and tribulations of getting to those results, its a story that gets the reader as smoothly as possible from their current knowledge to where we want them to be. That's true of engineering analysis as of science. And in both cases, we'd be in a much better place if, given that latter story, we could easily investigate the artefacts referred to: look at them, do our own visualisations or further processing, and follow the trail back to see exactly how they were generated.

Fix for keyboard weirdness in VMWare Workstation linux

VMWare Workstation 6.5 (and Server, but I'm not running Server any more, so I haven't tried to port this fix) as installed by default on a Ubuntu Jaunty 9.04 linux host, has some keyboard oddities with a Windows guest. Arrows, home, end, insert, and delete keys don't work.

The fix as given on the VMWare forums involves adding the following lines to the file

~/.vmware/config

and restarting VMWare:

xkeymap.keycode.108 = 0x138 # Alt_R
xkeymap.keycode.106 = 0x135 # KP_Divide
xkeymap.keycode.104 = 0x11c # KP_Enter
xkeymap.keycode.111 = 0x148 # Up
xkeymap.keycode.116 = 0x150 # Down
xkeymap.keycode.113 = 0x14b # Left
xkeymap.keycode.114 = 0x14d # Right
xkeymap.keycode.105 = 0x11d # Control_R
xkeymap.keycode.118 = 0x152 # Insert
xkeymap.keycode.119 = 0x153 # Delete
xkeymap.keycode.110 = 0x147 # Home
xkeymap.keycode.115 = 0x14f # End
xkeymap.keycode.112 = 0x149 # Prior
xkeymap.keycode.117 = 0x151 # Next
xkeymap.keycode.78 = 0x46 # Scroll_Lock
xkeymap.keycode.127 = 0x100 # Pause
xkeymap.keycode.133 = 0x15b # Meta_L
xkeymap.keycode.134 = 0x15c # Meta_R
xkeymap.keycode.135 = 0x15d # Menu

Beware the OSX case insensitive filesystem (Zope/Plone, Ubuntu, OS X, and virtualisation)

I've just lost a couple of hours trying to trace a problem I was experiencing installing some Zope/Plone software. This post is for the record, in case someone else (my future self included) tries to solve the same problem using the Google method.

I was installing on Ubuntu 9.04, running as a guest in VMWare Fusion 2 on Mac OS X 10.5. I was working on the Mac OS filesystem, mounted in the virtual machine using the VMWare HGFS driver.

The problem, I think, was that my OS X installation uses a case insensitive filesystem, and the tools running on Ubuntu expect it to be case sensitive.

The fix is to keep everything on the virtual machine's virtual hard drive. Unsatisfying, because that way I can't synch as easily with my work desktop and my working files are split up.

Case insensitive filesystems *grumble* *grumble*.

The error messages I encountered follow for Google's indexing pleasure. First, on trying to run ArchGenXML:

ndph@ndph-ubuntu:/mnt/hgfs/ndph/projects/hinf-comm/archgenxml$ bin/archgenxml
Traceback (most recent call last):
  File "bin/archgenxml", line 16, in ?
    import archgenxml.ArchGenXML
  File "/mnt/hgfs/ndph/projects/hinf-comm/archgenxml/eggs/archgenxml-2.4.1-py2.4.egg/archgenxml/ArchGenXML.py", line 22, in ?
    import archgenxml
  File "/mnt/hgfs/ndph/projects/hinf-comm/archgenxml/eggs/archgenxml-2.4.1-py2.4.egg/archgenxml/ArchGenXML.py", line 25, in ?
    import utils
  File "/mnt/hgfs/ndph/projects/hinf-comm/archgenxml/eggs/archgenxml-2.4.1-py2.4.egg/archgenxml/utils.py", line 15, in ?
    from archgenxml.interfaces import IOptions
ImportError: No module named interfaces

Next, on trying to start an instance of Plone configured using the listen buildout:

ndph@ndph-ubuntu:/mnt/hgfs/ndph/projects/hinf-comm/listen-p3.1-remember$ bin/instance fg
Traceback (most recent call last):
 File "bin/instance", line 61, in ?
  import plone.recipe.zope2instance.ctl
 File "/mnt/hgfs/ndph/projects/hinf-comm/listen-p3.1-remember/eggs/plone.recipe.zope2instance-3.3-py2.4.egg/plone/recipe/zope2instance/__init__.py", line 17, in ?
  import zc.buildout.easy_install
 File "/tmp/tmpjcLFGO/zc.buildout-1.3.0-py2.4.egg/zc/buildout/easy_install.py", line 32, in ?
 File "/mnt/hgfs/ndph/projects/hinf-comm/listen-p3.1-remember/eggs/setuptools-0.6c9-py2.4.egg/setuptools/package_index.py", line 2, in ?
 File "/usr/lib/python2.4/urllib2.py", line 108, in ?
  import cookielib
 File "/usr/lib/python2.4/cookielib.py", line 35, in ?
  from calendar import timegm
 File "/usr/lib/python2.4/calendar.py", line 8, in ?
  import datetime
 File "/mnt/hgfs/ndph/projects/hinf-comm/listen-p3.1-remember/parts/zope2/lib/python/DateTime/__init__.py", line 13, in ?
  from DateTime import DateTime
 File "/mnt/hgfs/ndph/projects/hinf-comm/listen-p3.1-remember/parts/zope2/lib/python/DateTime/DateTime.py", line 21, in ?
  from datetime import datetime
 File "/mnt/hgfs/ndph/projects/hinf-comm/listen-p3.1-remember/parts/zope2/lib/python/DateTime/DateTime.py", line 21, in ?
  from datetime import datetime
ImportError: cannot import name datetime

Malevolent voices that despise our freedoms

Philip Pullman on the loss of liberty:

Expressing an opinion is a dangerous activity

Whatever your opinions are, we don't want to hear them

So if you threaten us or our friends with your opinions we shall treat you like the rabble you are

And we do not want to hear you arguing about it

So hold your tongue and forget about protesting

What we want from you is acquiescence

Do read the whole thing.

Spectacular photographs of electrical storm in the dust cloud from Chaitén

Fotos Tormenta eléctrica en erupción del volcán Chaitén. linked to by Earl Mardle.

2000 Terrorists. Don't trust anyone!

Craig Murray in fine fettle:

The truth is that since September 11 Islamic militants have killed about 70 people in the UK. That's 12 people a year in a country of 60 million. Every death is terrible, but a threat to our existence it is not. You have a much better chance of drowning in your own bath, of being struck by lightning or of winning the national lottery than of being killed by a terrorist. But that wouldn't persuade you to give up your civil liberties, or that we have to invade more oil rich countries for our security.

And from a commenter to Murray's post:

Perhaps the government could give us a fear list, from 1 to 10, of the things we should fear most. Then we'd know where we are, and could start some serious and structured fearing without having to worry about whether we're fearing about entirely the wrong thing.

Half price!

That's a long queue for half price chips. Are they really worth the wait?

School or factory?

Who needs personal contact when we can monitor pupils using RFID tags (via Schneier on Security). *Shudder*.

San Francisco in jell(y|o)

Thanks Heather for a pointer to Liz Hickok's jelly model of San Francisco. Sadly the earthquake doesn't test it to distruction ...