Heck of a job, Phippsy

It’s been a busy summer, but not on the Registry front.

We’re currently working on integrating the ARC library so we can handle RDF a bit more intelligently. This will give us import capability, a SPARQL endpoint, and the ability to express vocabularies in more RDF serializations. We’ve also made some improvements to our URI-building feature, adding support for ‘hash’ namespaces and tokenized identifiers (rather than simply numeric). This means that a URI like http://www.w3.org/2008/05/skos#Concept will be built for you properly instead of having to edit the current default http://www.w3.org/2008/05/skos/12345 to get what you want. None of this even on the beta site, primarily because we haven’t had time to test it at all, and there are some things we know are still broken.

There’s also now a fairly simple PHP script that accesses the new Registry API to retrieve data remotely. You can see this in action at http://rdvocab.info/roles.rdf — there’s no data actually maintained on rdvocab.info, the data is retrieved from the Registry. We’re not publishing the script yet or documenting the API because, like so many things, they’re not quite finished — the script needs to be even simpler, tested with PHP4, and less dependent on .htaccess. The API needs a few more methods and also needs to require a key for some operations.

Expect to see some of this stuff appear in early September.

The grant to work on the Registry runs out in September, but I’ll keep working on it and hope to have some collaborators. I’ve been pretty poor at creating a welcoming collaborative environment, networking, and promotion so that may be a vain hope.

There’s a fairly long list of things yet to do and some of them are major. Application profile management is the biggest, but there are also things like the ability to follow, twitter-like, activity on a vocabulary, and more extensive control over notifications, and integrated discussions are needed to help support the vocabulary development features. The ability to import, export, edit, re-import, and have changes tracked throughout the process is also pretty critical. We want very much to integrate the sandbox into the main Registry, at least integrating user registration and making it possible to easily move a vocabulary from the sandbox to the registry. And there needs to be much more extensive help, better explanations of what’s going on, a place to report bugs and make suggestions that integrates with trac.

I’m off messing about in Canada on holiday for the next 2 weeks, so some of the things that I finished up this week will have to wait until I get back before they’re integrated into the site — I hate to potentially break things and then disappear.

Registry Installation Instructions

Jeepers, no posts for 3+ months and then two in one day! The truth is that I hadn’t realized the last post was still sitting in my drafts folder more than a month after I wrote it.

Moving on…

A number of folks have been interested in installing the Registry, especially since we’ve talked before about ‘easy installation’ being one of our design goals.

We’re pleased to announce that we have finally tweaked things to make a reasonably simple install from our subversion repository possible and provided some hopefully simple instructions detailing how to get the Registry up and running. We don’t provide enough tweaking or instructions (yet) to fully customize the interface, so once it’s installed it’ll still look exactly like the Registry, just running on your server instead of ours.

Whenever we update the production server, we’ll tag that code in subversion and update the link in the instructions (tying a string around my finger to help me remember as we speak), but there won’t be any other ‘release’ announcement unless we do something major.

Whenever we modify the database structure, we’ll provide a sql script to alter the database with each release. These scripts will always modify the database as it was after the previous release, so if you skip releases you’ll need to run the scripts sequentially. But this will all be on the instructions page.

We expect to update the production code quite often over the next few months.

Metadata Schema

If you’ve been watching the Registry closely (and we know you have), you’ll have noticed that a few weeks ago we started supporting the registration of metadata schemas. It’s not finished and far from perfect, but the perfect can often be the enemy of the good and at the moment it’s, well, good enough for now.

What makes it tough to get schema registration right is that our approach to what we’re calling registration attempts to be cross-cultural — trying to create a bridge from the technologies supporting the Semantic Web to the somewhat more ‘traditional’ data transfer technologies like XML.

We’re also trying to ‘eat our own dog food’ and are using an internally registered Application Profile to define the properties we’re using to describe metadata schemas and ultimately Application Profiles. This AP helps drive the schema registration user interface and we hope at some point we’ll be able to use a registered AP to generate many different interfaces, both human and application. It’s arguably too ambitious, but baby steps…

Vocabulary Management
The Registry is really more Vocabulary Management Application than Registry at this point, since we’ve layered so many management services on top of the basic registry functions. It manages two types of vocabularies:

• Value vocabularies — unordered lists of values (terms) that we express as skos concept schemes in RDF and a simple enumeration in XML Schema
• Class/Property vocabularies — lists of classes, properties (or attributes depending on your mental model) that we currently express as rdf:properties and rdfs:classes

Much of our terminology (value vocabularies, metadata schema, application profile) stems from our work with the Dublin Core Community more than the Semantic Web Community and maybe we’ll refactor some of those names as we move forward. But we hope the semweb folks can translate and we hope that the DC folks won’t hold our ultimate departure from some of their terms against us.

In the meantime, feel free to play in the sandbox.

Makes my head hurt

I was talking with Diane this morning about building the schema portion of the Registry and I feel the need to write down some of what we discussed.

For purposes of discussion, we have a draft schema property interface that defines some basic metadata schema property properties. We started the conversation because I was trying to get away from the “property property” nomenclature and because I couldn’t quite figure out the best way to extend the too-simple model to incorporate repeatable, typed notes/annotations.

Over the course of the discussion we came to a few conclusions:

• What we’re really discussing is an Application Profile in the old DC sense of that term (it has since been changed to “Description Set Profile” to reflect the more DCAM-centric viewpoint of the current DC Community) in which we’re defining schema property restrictions, namespaces, and usage requirements: There can be only one token, definition, label, type and they’re required; ‘Type’ utilizes a controlled vocabulary containing the concepts ‘property’ and ’subproperty’; etc.
• We have a Schema Properties Vocabulary registered that identifies these schema property description ‘terms’ as ‘concepts’, but this isn’t really correct because they’re actually properties of a metadata schema ‘property’ (and so we’re back to property properties <sigh>) and as such they should be registered as an Application Profile rather than a Vocabulary.
• The properties of each schema we register should be based on its own Application Profile, since there will be many different requirements and we’d like to provide some flexibility. For instance the RDA schema may need to have an additional property property that declares a relationship between the property and a FRBR entity.
• We can’t register a Metadata Schema Properties Application Profile until we can register a Schema
• In order to register a metadata schema we need a generic Metadata Schema Properties Application Profile
• We’re stuck with “property properties”
• This stuff makes my head hurt

In the interest of moving forward, stopping the spinning, and headache relief we’re going to pretend that a generic Metadata Schema Property Application Profile (MSPAP — pronounced ‘ems-pap’) exists and slap something together and make the interface fairly inflexibly tied to it. At some point in the future we’ll (hopefully) make it flexible enough to be based on any registered MSPAP.

TimeSlices and Versions

You can now retrieve a snapshot in time of the RDF or XSD serialization of a Concept/Scheme/Vocabulary by appending a ‘TimeSlice’ to the URI. For example:

http://metadataregistry.org/uri/NSDLEdLvl/ts/20060422200002.rdf
or
http://metadataregistry.org/uri/NSDLEdLvl/ts/20060422200002.xsd

will always and forever retrieve the SKOS/RDF or XML Schema representation of the NSDL Ed Level Vocabulary as it appeared at 2 seconds after 8PM on April 22, 2006 (2006-04-22 20:00:02). If you follow the above .rdf link you’ll notice that the concept URIs that the TimeSliced Vocabulary references have also had a TimeSlice appended:

http://metadataregistry.org/uri/NSDLEdLvl/1001/ts/20060422200002.rdf

in order to lock them into that precise point in time on an individual basis as well. We hope the utility of being able to reference a vocabulary at a particular point in time regardless of subsequent changes will be, well, useful.

In order to make retrieving TimeSlices for specific events in the history of a vocabulary a bit handier, we added a TimeSlice link to every history event. You can specify a TimeSlice for any point in time regardless of its relationship to a history event, but the link just makes it simpler (it’s over on the right side of each line):

Uploaded with plasq’s Skitch!

Named Versions

You’ll maybe also have noticed that there’s a ‘Name’ link nestled to the right of the RDF and XSD links. If you’re a Vocabulary Administrator, then you now have the ability to label a TimeSlice with a distinct version name. That link again is there to make it easy to reference a point in historical time and clicking on it pre-enters that TimeSlice into the Create new version form:

Uploaded with plasq’s Skitch!

There’s no limit to the number of versions you can create, and versions (unlike TimeSlices) can be deleted or edited by any Vocabulary Admin:

Uploaded with plasq’s Skitch!

…although we think that either editing or deleting a version is likely to be a less-than-ideal practice. Still, we allow it — it’s your vocabulary after all.

Once a named version has been associated with a TimeSlice, it will appear in the event history list just above the point in time it references:

Uploaded with plasq’s Skitch!

The RDF and XSD links on the right side of the version line now reference the version name:

http://metadataregistry.org/uri/NSDLEdLvl/version/release+1.0.rdf

But this is where it gets a little incorrect… Since the named version URL is just a TimeSlice reference, it does a silent redirect to the referenced TimeSlice. It should probably do a 303 redirect instead. We’ll fix this later, unless it’s a show-stopper for one of our many users.

Improved user management

We’ve been promising for a while now that we’d make it easier, actually a better word is ‘possible’, for Vocabulary Owners to add ‘Members’ to Owner/Agents and ‘Maintainers’ to Vocabularies. We finally implemented it today! It’s unfortunate that it has taken us so long, since one of the primary goals of the Registry is to support multi-user vocabulary development, but it turned out to require more infrastructure twiddling than we thought it would.

If you’re a vocabulary owner and are logged in, you can add other registered folks as ‘members’ of your Owner/Agent and you can even make them administrators if you want:

Uploaded with plasq’s Skitch!

We hope the process is pretty self-explanatory.

Once you’ve added a user as a member of your Owner/Agent group, you can add them to your vocabularies as Vocabulary Maintainers or Administrators.

Uploaded with plasq’s Skitch!

We realize that this is still somewhat limited and of course the documentation is ummm, poor, but we’ll be doing more with user management shortly.

Registry News

We’ve updated the front page of the Registry to include a Registry-specific news feed from the Registry blog. You can subscribe to it right from (t)here and stay up-to-date with the Registry as we move it forward. Notes like this will be typical and probably pretty frequent over the next few months.

Major update

The registry database was updated last week, in both the sandbox and the registry, to support history tracking. This is in preparation for finally enabling timeslice retrieval and versioning.We also made some significant changes to the site layout and css, so if things still look a little funky, try refreshing your browser — most browsers seem to be caching our css and not detecting the changed files.In the process, we broke search (you may not have even noticed), but it’s fixed now.

New Host

Since the beginning of the project, we’ve been running our own web server. After a significant number of local service outages on the part of Time Warner this year, particularly in July, we finally decided that had to change.

As of today, we’re running on Dreamhost. Dreamhost isn’t perfect, but they offer a nice package of services, reasonably good support and, with the right promo code, the price was right. So we’re going to give it a try for a while.

Moving from a host where we had complete control of the Apache and MySql servers to a shared host where we don’t has required some minor configuration changes to the blog and the wiki, and some significant changes to trac and subversion, but no changes to the Registry. As a side effect of the wiki tweak, we can have prettier URLs — http://metadataregistry.org/wiki/index.php?title=Main_Page becomes http://metadataregistry.org/wiki/Main_Page. Links will default to the new style, although the older form will still work in order to keep older bookmarks working too.

We updated trac and subversion to the latest stable releases, moved trac from a lighttpd server to Apache and moved subversion from a subversion server to Apache as well. This has changed the user authentication process for both services, hopefully for the better.

Registry update

We made some major/minor changes to the Registry software last week. If you’ve used it lately you may have noticed the fairly ugly tabs that replaced the links at the top of the browse pages. They still need some work, but it’s generally a lower priority than some other things.

The major change was reworking a significant amount of the Registry code to bring it up to the Symfony 1.0 release. We’d been working with a fairly ancient version of Symfony while waiting for the final and as a result there were significant structural changes required. We also wrote a bunch of automated functional tests so that we can be a bit more confident that future releases will have fewer bugs. You shouldn’t notice any of this when using the Registry, but please let us know if we broke anything.

We cleared up a few small existing bugs, one of which was that the Registry was a little too aggressive in checking to see if a prefLabel was unique, so the check is now local to the Vocabulary instead of the entire Registry. Another was that under some circumstances the URIs generated by the Registry Sandbox had a hard-to-see double slash after the base domain, causing the URIs to fail to resolve properly. In addition to fixing the bug we also fixed all of the bad existing URIs.

More updates are imminent.