Maintaining and Open Relationship with Google

Google's mission is to "Organize the World's Information" and they do a rather smashing job of it as long as they alone are doing it.   Even though they do a better job than most as staying open, there is still a significant risk when putting all your eggs in the Google basket and few options for backing out.   Particularly with the rise of Google+, Google Music beta and other such services going all in on Google could prove a big liability for individuals and companies in terms of being able to shift to new or better services as they emerge or just re-establish ownership over your own content.  Every business would do well to act with caution in opting for the convenience of any service as that convenience would too easily transform into abducting the ownership of your content.   I would suggest that losing control of your content in a world where ideas and content are a commodity is the same as losing control of your life or business.

With a little forethought however some convenient ways exist to both leverage the convenient services offered by Google and remain managers of your information.  The Digital Liberation Foundation launched it's "Google Takeout" service, that allows the harvesting and export of your information from various Google services into open formats.  Open formats are the key to keeping your content flexible and mobile and in a world where 5 years is an entire era of information management practices, this is critical to surviving and thriving in the modern world.  The group is starting with export features related to Google services but plan to expand their ability to other services as well.  I assume (hope) this means Facebook and Yahoo! based services but only time will tell.

Even with groups like this helping to keep information open and portable a healthy dose of caution is advised when flirting with services providers like Google.  Grass-roots efforts are fragile at best and ultimately the pressure needs to be on the big companies to keep their standards open.  A world that encourages the generation of ideas and helps communities evolve is what will bring the kind of innovation we're all seeking and like it or not our practices and the good faith of major corporations are going to heavily influence our success at this. 

Game of Thrones Redux

Even knowing what was coming in the series, the end left me pleased and stunned as I was when I first started reading the books.  The characters were visualized and brought to life in amazing ways and I was blown away at how lean and mean the storytelling was.  The series highlights just what a powerful contribution a great set of writers and directors can make to a series.  I'm not one of those fans who feels unhappy because their favorite character wasn't featured enough.  The breath of the story itself meant that there was just no way every character could be touched on and they obviously had some hard decisions about who to feature and how.  Given the realities of what they had to film I think the obviously painful choices were good ones.

Even given the lean nature of the storytelling there is a long list of standout characters for me in the series already.  Danny and Drogo, Tyrion, Jon and Jamie were just brilliant and it's only in Game of Thrones that that would seem like a short list of stand-out characters.  Pulling that off in any series would be amazing, here it's just a miracle.

Some thing else the series really highlighted for me was the just how telling the story on screen drove the need for different decisions than telling it in the book.  Most obvious to me is the fact that in print there is no background, everything is foreground in writing.  You don't accidentally see a sentence in a novel so every detail, character and event is right there in the front of your mind.  The series instead was unafraid to let many things play out in the background, which gave readers of the book a special treat while watching the series instead of making the whole thing feel reduced.  I could look around a scene and squee "Look that's Hodor!" or speculate that the guy Ned gave Stannis' letter to was Davos.    Having a background on television just made everything much more tasty to me.

I really have to say I'm looking forward to season 2 and a year seems far too long.  In the meantime Dance of Dragons is hitting shelves around my birthday and that's exactly the kind of present I can look forward to.  I definitely hope the series is going to kick Mr. Martin into high gear with his writing, because if he waits as long for the next book as he did for Dance the entire series will have had it's run and be well over before Winds of Winter comes out.

Going to conclude by going "bullet time" on some other random observations about the show:

Renly and Ser Loras were perhaps the only big mistake in terms of character portrayal in the series.  I thought they came off as too effeminate which in turn I felt insulted the characters and did an injustice to the portrayal of gay characters.  The characters in the book are portrayed as every bit as dangerous as everyone else in the world and Renly himself likened to a young Robert at his most fierce.  This I thought was lost on the character and I think it was a missed opportunity to have a storyline with characters who just happened to be gay without having to have "gay" characters.  They could have shown a relationship and instead I felt like they showed a caricature.

The choice to flesh out Jamie as a character right from the start was overall a good one, though I felt his exchange with Ned about killing Arys "feeling like justice" was too contrived.  It's not that facts about that get revealed later in the book for him that bothers me but more than Jamie didn't need anyone's approval, he was remorseless about his choices and very accepting of himself.  I think setting him up as more self doubting early will reduce the power of his character arc in the long term.  Though I do have to say his capture by Rob was exactly the Ser Jamie I wanted to see so at least they returned to it in some fashion.

The Lisa and Robin (Robert) thing was… well over the top for me.  It creeped me out though I suppose it needed to.  Still it seemed a bit forced or contrived to me and can anyone imagine them being able to continue with that kind of story for Robin 3 seasons (years) from now when the actor is 15?

All the kids I thought were great but the year between seasons thing makes it hard to see how they're going to pull of the younger characters.  As with Robin can anyone work out a 13 year old Bran?  15 or 16 year old Arya?  I'm curious to see what they'll do with those actors.  I hate the thought of them being replace because, DAMN are they doing a great job.  Not my problem though so fingers crossed it comes off well.

The Hound is a character I really enjoy from the books.  I thought choosing to move the exchange about what happened to him at the hands of his brother to Little Finger was a missed opportunity.  It didn't build anything between Little Finger and Sansa and I think might reduce the development of the Hound as a character later on.

Drogo for me was even better than he was in the book and I hope Jason Moma gets appropriate credit for his work there.  One thing I didn't like however was the shift in how Drogo got the wound that eventually killed him.  Choosing to move it to him letting someone cut him seemed to make the character less intelligent to me, not dramatically but it was something I found annoying.

I could go on and on but I'm running out of energy here.  Needless to say I'll probably give the series another watch before next year and I'm looking forward to Spring 2012. Winter can't come soon enough!

New Rise of the Apes European Trailer

There's a new trailer out for the Planet of the Apes reboot called "Rise of the Apes" and it really looks fantastic.    It's amazing how much story they can convey in a simple trailer and I have hopes the movie has a lot of depth in it.  There seems to be a lot of potential for commentary about the nature of intelligence and self, the respect for life and compassion or what a lack of it brings.  Seeing the "acting" of Caesar with the John Lithgow character, who seems to have Alzheimer, brought a tear to my eye as you watch the chimp express compassion for the human who is obviously struggling.    This could be the start of a great movie franchise, at least I hope it is.

Django Libraries for XML and eXist DB

We often use XML at Academic Libraries and decided to create a set of libraries to ease our work connecting our XML and repository based work to the Django framework by building a central set of libraries.  We'll be continuing to build these libraries out and recently released the code as open source projects on GitHub.

EULxml provides XPath parsing features in python and mappings for xml documents to pythonic objects as well as features to provide Django Form to simple XML objects.  The code is available on GitHub and some documentation and examples up on read the docs. 

EULexistdb provides connections and XQuery capability to eXist DB and Django Queryset like objects for rich interaction between Django and XML data stored in eXist DB.  Combined with the XML Django Forms from EULxml (on which it depends) it has enabled us to do a lot with our Library collection.  This library is also available on GitHub and has some documentation and examples up on read the docs.

We're excited at the possibilities of leveraging the power of Django with our XML databases and repositories.   We're open sourcing it in hopes others may find it useful and may want to contribute to the libraries as well.

They Shoot URLs Don’t They?

I've had a rather lengthy and interesting blogging life these last nine years and stopped over the last few for a number of reasons.  A backlog of 2400+ posts however have given me a rather interesting dataset to test when it comes to URL persistence and as I'm going over old posts I find and example of URL persistence that seems very backwards to me.

I use to run a funny little site for Gamespy called Paragon City Hall, just a community based site for an as-then-un-release game called City of Heroes and posted about that site back in 2002

In another post around the same time I reference my depression over a news story from Reuters that made me want to kill myself.  Over dramatic yes, but hey, it was 9 years ago so give me a break.

The amazing thing to me is that the Reuters story results in a dead link, nothing, no forward, no search suggestion, nothing.  The Paragon City Hall link however STILL WORKs, even though I shut the site down 8 years ago and it's unlinked by the network.

What kind of world do we live in when RPGPlanet has better URL persistence than Reuters?

Although I came to this realization later than I should have in my career, Content in the web is a Social Contract.  Tim Berners-Lee made this case quiet eloquently in an article I cite quiet frequently and while I might not expect Gamespy to understand it, of all agencies Reuters should get it.  They should have gotten it perhaps before even TBL posted anything about it in 1998.

A particular fear creeps over me when an agency like Reuters is letting links expire like that and offends me as an adopted digital librarian.   (They found me floating down a bitstream in a whicker basket and took me in as their own)

Let's hope these agencies realize are better about it 10 years from now than they were 10 years ago but honestly I'm not that confident about it since even today.  Reuters seems better from their URL structure though that hashed ID in the slug makes me a bit concerned.

I think I might sit down this weekend and write a quick script to see how many of the links reference in my blog over the last 9 years are still resolvable.  Could be interesting.

Django AuthenticationForm For User Login

Django already makes it insanely easy to log a user in and out via their generic views.  Engineers will often want to create their own login view to provide some flexibility, say an Ajax login or other spin on standard login.  A number of examples are given in Django for that as well, and as with most of the framework this is a snap too.  A convenient feature of Django that doesn't make it into many of the examples I've seen is the AuthenticationForm that provides a convenience Django form with associated logic render a login form, validate input and throw errors if they do things like forget to supply a password and do the basic authentication check.

The form provides that all for you and all you really need to do in your view is read the user submitted data, validate the form and take the final step of logging the user in.

This is just one form in a group of 7 or so that provide all kinds of convenience features like Password Changes and User Registration.  Not only do they provide a developer with very easy access to common functions but they can extended or subclassed like any other Python Class to add or override functionality.

Here's an Example of a simple view method using the AuthenticationForm.  Something of a 'gotcha' for developers who normally use Django form is the POST values are passed as the second argument to the form.  The request object can be passed but that is normally only done to check for authentication cookies.  See the Source for more info on the form..  

from django.contrib.auth import login
from django.contrib.auth.forms import AuthenticationForm
from django.shortcuts import render
from django.shortcuts import HttpResponseRedirect
from django.core.urlresolvers import reverse

def authenticate_user(request):
    """Logs a user into the application."""

    if request.user.is_authenticated():
        return HttpResponseRedirect(reverse('account:index'))

    # Initialize the form either fresh or with the appropriate POST data as the instance
    auth_form = AuthenticationForm(None, request.POST or None)

    # Ye Olde next param so common in login.
    # I send them to their default profile view.
    nextpage = request.GET.get('next', reverse('account:index'))

    # The form itself handles authentication and checking to make sure passowrd and such are supplied.
    if auth_form.is_valid():
        login(request, auth_form.get_user())
        return HttpResponseRedirect(nextpage)

    return render(request, 'account/login.xhtml', {
        'auth_form': auth_form,
        'title': 'User Login',
        'next': nextpage,

The associated template code for this renders the form and any errors if needed.  Note that the form may have individual field errors in the case of a blank username or password and returns a ValidationError if the credentials provided were invalid and this is dispalyed to the user via the 'non_field_errors' attribute if present.{% if auth_form.non_field_errors%}

    {% block message %}

        <div id="error_msg">

        {{ auth_form.non_field_errors }}


    {% endblock %}

{% endif %}

{% block content-body %}

    <form action="{% url account:login-form %}" method="POST">

    {% csrf_token %}

        {% for field in auth_form %}

            <div class="fieldWrapper">

                {{ field.errors }}

                {{ field.label_tag }} {{ field }}


        {%  endfor %}

    <input type="submit" value="Login" />

    <input type="hidden" name="next" value="{{ next }}" />


    {{ ct }}

{%  endblock %}

Django Tempate Tag for Gravatar Images

Gravatar images seem to be growing in popularity across a number of sites and the services already makes it incredibly simple to grab a profile picture there via URL.  The Gravatar site itself has a number of examples on how to grab an image off of the service, as well as more detailed examples of grabbing more information.

They do provide a examples for grabbing an image via Python and even a Django example which renders the image as a template note.  For displays like this I generally prefer an inclusion tag since I can render the image in a template rather than having to build it each time on my own.

The template tag itself is just:

from django import template
import urllib, hashlib

from yourapp import settings

# Provide Default settings so users only need to provide them in if they want to override.
GRAVATAR_BASEURL = getattr(settings, "GRAVATAR_BASEURL", "")
GRAVATAR_SIZE = getattr(settings, "GRAVATAR_SIZE", 40)

register = template.Library()

def gravatar_url(email, size):
    Builds a Gravatar Image URL based on the provided email.

    :param email: Email address to query for a gravatar image.
    :param size:  Size to request and render the image in pixels.

    attrs = {
        's': size

    gravatar_url = "%s%s/?" % (GRAVATAR_BASEURL, hashlib.md5(email.lower()).hexdigest())
    gravatar_url += urllib.urlencode(attrs)

    return {'gravatar': {'url': gravatar_url, 'size': size}}

def gravatar_for_email(email, size=GRAVATAR_SIZE):
    Renders a gravatar image for user with the specified email via a template.

    {% gravatar_for_email "" 40 %}

    :param email:  String representing the users email.
    :param size: Size of gravatar to use in pixels.  OPTIONAL
    email = "%s" % email
    size = int(size)
    return gravatar_url(email, size)

This approach also has the advantage of being extendable and it's easy enough to build additional template tags such as 'gravatar_for_user' providing it a user instance instead that queries for their email and calls the `gravatar_url` method.

Lastly you only need to provide a template to render the return in, in this example it's calling the template 'account/snippets/gravatar.xhtml' which is simply:

{% if gravatar.url %}
    <img src="{{ gravatar.url }}" height="{{ gravatar.size }}" width="{{ gravatar.size }}" alt="Gravitar image" class="gravitar" />
{% endif %}

You can store the template with your other templates but I suggest storing in a templates directory within the app you create the template tag in. If your settings.TEMPLATE_LOADERS contains 'django.template.loaders.app_directories.load_template_source' it will add any templates directory in an installed app by default.

Gaming Wiki Back Online

I brought my Gaming Wiki back online on the site here after several months of being down. I apologize for that and I don't have any better excuse than not really taking the time to do it.  I had some difficulties with my previous web host and all I was ever able to get was a *.tar.gz download of the wiki database and the service would timeout every time I tried to download a gzipped directory of the mediawiki itself.  I kept hoping I'd find a backup up copy on a CD somewhere but no joy.

So it languished in 404 hell for a bit while I came to terms with the fact that I'd have to take the DB dump from an unknown older version of MediaWiki and try to upgrade it to work with a modern download.  

All in all I have to give it to the MediaWiki folks, I was essentially able to just run the update scripts and only had to make a few settings changes that took a bit of looking up.  More or less though the whole thing came back up.

Because I couldn't get a backup of the files stored in my wiki though some of the file links and thumbnails wont work until I come up with some plan to rebuild them.

Thanks to everyone for your patience.

SemTech 2011 Redux

The SemTech 2011 Conference delivered a lot to the attendees and I thought I'd jot down a few of my thoughts and note some highlights as the conference draws to a close.

By and large I have to say that the technology has definitely arrived and we're capable of some exciting advances in linking data and having the web to begin to fulfill some of the promises of being a real knowledge base.  I just hope Skynet appreciates all the work we're all doing on it's behalf when it finally becomes self aware. 

What had the structured data crowd buzzing the most was last weeks announcement of MicroData format support by Google, Microsoft and Yahoo at  Annoyance aside at Microsoft trying to put up as if it was some small independent standards board, I think the Microdata format seems just fine to me.  Essentially a competitor to RDFa it targets easy markup of information in a web page and is a bit leaner and easier than the current RDFa 1.0 standard.  The crowd here being a bit biased toward RDFa, there wasn't a lot of positive talk about but I find I can't really care too much one way or another.  What we need to develop is a community of practice, and the technology should be secondary to that as long as it's not a barrier.  To me Microdata or RDFa are both fine standards and the only logical argument I would make to prefer one over another is that's aim to to mark up information for better searching while RDFa is aimed at marking up knowledge.  It may seem a subtle difference but misaligned motivations like this can be the cause of big directional shifts and I think it's important for us as a community to not lose site of our aims here.  We're not trying to create a more searchable web, we're trying to make a more sensible web.  

This makes a nice segue into what I think was one of the better presentations at the conference,  Ivan Herman's presentation of on RDFa 1.1 .  The specification should be released fairly soon and 1.1 greatly simplifies the already simple standard.  In particular, when stacked up against the stuff at I think RDFa 1.1 is just as simple and easy to use and I feel more comfortable with it's extensibility over microdata formats. I am particularly interested in the applications of RDFa and TEI with some of our local collections, if they were like I hope I think it could be a fantastic step forward in presenting our TEI tagged collection.

On a side note I have to say Mr. Herman gets my vote for presenter I most want to be my instructor at Hogwarts.  The guy is just an awesome presenter and tremendously knowledgeable.  We need more advocates like him out there.

Much ado was given across several presentations about OWL, SKOS as well as automatic tagging and named entity recognition.  While services like Open Calais seems great for contemporary content and news, these services fall short in dealing with academic information or literature.    Fuzzy as it is I think this kind of automatic marking and tagging does have quiet a bit of value in speeding the cataloging process up and for standardizing tagging and ontologies.  Coupling auto-tagging and entity recognition with full text indexing can be very powerful for enhancing the  overall user experience and I just want to put that into the "lets do more of that" category.  I didn't see a particular presentation that stood out about this topic but it came up enough to be worth mentioning.

As I mentioned in a previous post the folks over at O'Reilly did a great presentation on the evolution of their own RDF and Linked Data strategies. In particularly they highlighted that this is an evolution for everyone and just plain doing something trumps thinking about it anytime.  They also highlighted how powerful working with triple-stores (and N-stores) can be in joining distributed data and systems and also made a great case for why RESTful services make like better for everyone.

Bernadette Hyland had a great Linked Data Cookbook presentation covering what goes into the mix of producing a great Linked Data Strategy.  Throughout the conference she was also a great advocate for the need for a community driven effort for semantic practices and the benefits we can all derive from it.  

Zepheira and MIT gave an interesting presentation on Exhibit3 using pure JavaScript an update of Exhibit a framework for the display of rich data.  Zepheria also demonstrated Recollection, a Django based open source offering on github for easily creating custom interactive displays for data collections on the web.   I think it's going down the wrong path to thing of any software as a solution but when looking at these as a tool in a bigger toolbox I think these have a lot of promise for curators frustrated by not being able to get their data out there.  The use of Django/Python in particular creates a good lower bar to deployment and positions it well to be enhanced, extended and to grow.

There were other highlights and I will try to link or at least tweet links to the slides as the come online.  

I appreciated meeting and talking with everyone at the conference and look forward to exploring this more.

San Fran Pic-So

Conference ended by mid-day here so I decided to take an open top  Bus Tour around San Fransico.  Great experience and you could get on and off all day so I got to see  more of the city in 4 hours than I have on most of my previous Trips.  Posted Pics to Picasa Web and linking blow.