Hmm, I know VCALENDAR and can hack-parse html …

So 3 hours later and I have a very basic program that reads the Schedule wiki page, sucks out all of the tables that represent the daily schedules and generates a .ics ready for importing into your favourite calendar program.

Requires Python 2.6 or 2.5 with ElementTree installed: generate_summit_calendar.py

enjoy!

Note: If you run into any configure or make issues you may need to install some of the from-source tools I had already in place from my work to get Mozilla Firefox to build from source. I’ve covered that in detail on my Mozilla wiki page.

I used the current source tarball from the MCabber site – make sure you get the MCabber source tarball and also the Loudmouth library tarball.

Extract both into your favourite working/source tree setup

tar xf loudmouth-1.4.3+gitb5a9de5b.20100413.tar.bz2

tar xf mcabber-0.10.0.tar.bz2

Build the Loudmouth library. Note: I put all of my source installed libs into /opt that way I can set LDFLAGS and CPPFLAGS to point to anything non-OSX that i’ve installed.

cd loudmouth-1.4.3+gitb5a9de5b

./configure –prefix=/opt LDFLAGS=”-L/opt/lib” CPPFLAGS=”-I/opt/include” –with-ssl=openssl

make

sudo make install

Switch over to the MCabber directory and build it

cd mcabber-0.10.0

./configure –prefix=/Users/bear

make

make install

and that’s it – you now have a console XMPP Client!

Second, the endpoints used to access an API are, at most, 1/3 of what makes up an API – the most important part of an API is the data passed to and also from those endpoints to do the tasks that the endpoints enable.

So sure WordPress and Tumblr have a Twitter-compatible API but what happens when Twitter changes how one of the endpoints work, like they did when they recently changed how since_id works or even how re-tweets work. At best the other sites will be able to make similar changes but the worse case scenario is that they won’t or cannot make the changes and then things start to get ugly. Anyone remember RSS 0.93? or even the blog related API’s to make posts from clients – all suffered from fragmentation.

That is not to say I am not excited to see the API pattern used by Twitter is being more widely adopted, it’s a well thought out and functional API – I just think that crowning it the King of micro-blogging APIs is a bit premature until the *whole* API, data formats and all, is documented.

After following the normal Prosody install methods and this time making sure my SSL environment is clean, I then switched over to the wiki pages to get the component configured and installed. Except for the small bit about Java SSL code (FedOne is written in Java) not liking good old normal PEM formats, everything went according to the documentation – go figure! ;)

Anthony Baxter clearly documented installing the component into the Prosody environment on the FedOne wiki so I don’t need to go into messy details here – thanks Anthony!

After restarting Prosody, running the FedOne component, it’s time for a test and thankfully the Google gang included with FedOne a basic command line client to test with:

./run-client-console.sh username

Following this wiki guide on how to use the test client, I was able to test my local server and now I’m set for testing later – I think. :)

Total time to do the install with the following, about an hour:

• Ubuntu Jaunty
• Google Wave FedOne (w/SSL)
• Prosody XMPP server (w/SSL)

What I mean is that since a wave document is an ever changing entity, any bot attached to a wave has the amazing chance to both deliver updates to the wave and also to receive updates (commands?) from the wave and I think most early bots are just doing the former.

This bug was preventing parsing of 25 August 2009 – which seems like a silly bug to have except when you realize that it came about from adding more robust locale support for short month/week names.

Anywho, patch applied, some other minor fixes worked in and a new release!

This is great because it allows you to have a single node service both bots and people. Without this you would have to define proxy bots that are subscribed to nodes and then they would send out chat messages for each incoming PubSub event for people in their roster.

Not exactly a friendly model IMO.

So to prepare for this I started working on a XSLT file to transform an Atom payload to HTML – but first I needed to either learn XSLT real fast or “borrow” from someone :)…

After about an hour of googling I finally found one that a) worked and b) targeted the 1.0 version of Atom at the OpenSearch site – wow was that a gold-plated, kitchen-sink-included version! It handles Atom, RSS (all of them), RDF and some others – and after a rather great question from @metajack (aka Jack Moffitt):

Why bother with RSS at all?

I started pulling out all of the non-Atom bits. He also helped me grok a template scope problem that was leading me to curse XPath royally – thanks Jack!

So after many, many rounds of edit/test/curse I boiled it down to a basic template (tho I did keep some of their snazzy person and url templates – *very* nice!) and then proceeded to tweak the HTML it generates.

So if you pass in this Atom payload:

<entry xmlns=“http://www.w3.org/2005/Atom”>
    <title type=“html”>Test Atom</title>
    <link href=“http://seesmic.com/videos/FOO”/>
    <id>http://seesmic.com/videos/FOO</id>
    <author>
        <uri>http://seesmic.com/foobarbaz</uri>
    </author>
    <published>2009-01-03T20:46:19+00:00</published>
    <updated>2009-01-03T20:46:19+00:00</updated>
    <link href=“http://t.seesmic.com/thumbnail/FOO_th1.jpg” type=“image/jpeg” rel=“enclosure” title=“Test Atom”/>
    <link href=“http://v.seesmic.com/flv/FOO.flv” type=“video/x-flv” rel=“enclosure” title=“Test Atom”/>
    <source>
        <generator>SeesmicBot</generator>
        <id>http://feeds.seesmic.com/user.foobarbaz.atom</id>
        <updated>2009-01-03T20:46:19+00:00</updated>
        <title type=“html”>Test Atom</title>
        <rights>Creative Commons Attribution</rights>
    </source>
    <in-reply-to xmlns=“http://purl.org/syndication/thread/1.0” type=“application/xhtml+xml” href=“http://seesmic.com/video/FOO” ref=“FOO”/>
    <category term=“en”/>
    <rights>Creative Commons Attribution</rights>
    <link href=“http://creativecommons.org/licenses/by/3.0//us/rdf” type=“application/rdf+xml” rel=“license”/>
</entry>

You will get this:

<html xmlns=“http://www.w3.org/1999/xhtml” xmlns:xhtml=“http://www.w3.org/1999/xhtml”>
<body>
    <div>
        <a href=“http://seesmic.com/videos/FOO”><div class=“x-escape”>Test Atom</div></a> by
        <a href=“http://seesmic.com/foobarbaz”>http://seesmic.com/foobarbaz</a>;
        <p>categories: en; </p>
        <p><a href=“http://t.seesmic.com/thumbnail/FOO_th1.jpg”>Test Atom</a> (image/jpeg)</p>
        <p><a href=“http://v.seesmic.com/flv/FOO.flv”>Test Atom</a> (video/x-flv)</p>
        <p><a href=“http://creativecommons.org/licenses/by/3.0//us/rdf”>Creative Commons Attribution</a>
        </p>
    </div>
</body>
</html>

Right now we think there is a bug in how Tigase handles this (well, actually since it’s not generating the body there is a bug *somewhere*), so I can’t show you a live example :( but as soon as the dev guys fix it, or tell me what I did wrong, I’ll make another post.

You can find the XSL template I’m using here

Update: sure enough, Bart demostrated it working on his test server so I dug deeper and discovered that the Java XSLT processor is *much* more pickier than xsltproc about errors. One of the helper templates was missing but it was never called so xsltproc just gleefully went on it’s merry way where as Java’s tossed an error and just refused to do anything more. So now Tigase is generating <body></body> *and* <event><event> payload data!

Lately Nathan and I have been battling subtle bugs in Tigase (which Artur and Bart have been great at fixing), subtle bugs in our code (oh yea, even there! ;) ) and the usual other 30 distractions as the team delivered the beta version of the new site.

With this latest round of tweaks we are very close to calling it “shipped”:

• We turned on thenode that delivers the public timeline data (or what others call the “firehose”) and that is now sending Atom payload events to it’s subscribers,
• We are seeing a more stable auto-refresh of the timelines (well, more stable after this last tweak)
• We can now dial the BOSH timeout up or down to find that sweet spot for load management
• The Tigase team delivered a great new feature: Server Monitoring which I’m hoping we can wire into our infrastructure for much more accurate stats

All in all it’s been a really crazy couple of weeks – but man is this stuff fun!

So I’m hoping that the XMPP community allows us a bit of boasting as we have a XMPP PubSub system running “in the wild”. I’m sure there are others out there, but so far they have been rather quiet on how things are done or they have quietly gone away. We are working on an architecture article for the Seesmic Developer’s site that goes into more details on how and why.

Special thanks to Jack for taking the time to review how we structured our Atom payload — hopefully it’s less messy now :)

The journey to Pubsub hasn’t been smooth for sure, along the way we have learned how many different interpretations of the word implemented can be used when applied to a very big and complicated spec like XEP-0060 just on the server side alone. Then you get into the realm of client libraries and I can see why there was hardly any well known implementation of Pubsub.

Sure there are plenty of behind the scenes implementations, but those are not public and they are not using open source tools – that’s the key difference in what we are doing.

In the last couple months a lot of activity has been happening in this space, so much so that I’m hoping we can delivery publicly so we can brag about being being one of the first (heck, if not the first) open source based implementations.

Back to work, just because we have a end-to-end test finally working doesn’t mean it’s all done :)

Technorati Tags: pubsub, seesmic, xmpp

An email reminding me that I had checked in 0.8.5 with the damn debug flag still enabled gave me the poke I needed to go thru the Issues listed (wow – people have been filing bugs!) and generate a new release.

See the Changes.txt entry for 0.8.6 for details.

It can be found via the Python Package Index, Google Code or my own
website.

Thanks to Bernd Zeimetz (Debian maintainer) for submitting a patch to localize parsedatetime for the German (de_DE) locale!