So, instead of running that errand I should be running and getting lunch…

As soon as I heard that the WaveSandbox was allowing federated connections I just had to try it out…

Wanting to be ready for the day when Google Wave opens the federation spigot, I decided to revisit the FedOne component reference code and get it setup with my test XMPP Server.

After following the normal Prosody install methods and this time making sure my SSL environment is clean, I then switched over to the wiki pages to get the component configured and installed. Except for the small bit about Java SSL code (FedOne is written in Java) not liking good old normal PEM formats, everything went according to the documentation – go figure! ;)

Anthony Baxter clearly documented installing the component into the Prosody environment on the FedOne wiki so I don’t need to go into messy details here – thanks Anthony!

After restarting Prosody, running the FedOne component, it’s time for a test and thankfully the Google gang included with FedOne a basic command line client to test with:

./run-client-console.sh username

Following this wiki guide on how to use the test client, I was able to test my local server and now I’m set for testing later – I think. :)

Total time to do the install with the following, about an hour:

• Ubuntu Jaunty
• Google Wave FedOne (w/SSL)
• Prosody XMPP server (w/SSL)

One of the niftier configuration items for a XMPP PubSub node (see Configuring a Node) is the ability to define an XSL template to allow the PubSub component to generate a message body to go along with the message payload – basically a human readable version of the payload.

This is great because it allows you to have a single node service both bots and people. Without this you would have to define proxy bots that are subscribed to nodes and then they would send out chat messages for each incoming PubSub event for people in their roster.

Not exactly a friendly model IMO.

So to prepare for this I started working on a XSLT file to transform an Atom payload to HTML – but first I needed to either learn XSLT real fast or “borrow” from someone :)…

After about an hour of googling I finally found one that a) worked and b) targeted the 1.0 version of Atom at the OpenSearch site – wow was that a gold-plated, kitchen-sink-included version! It handles Atom, RSS (all of them), RDF and some others – and after a rather great question from @metajack (aka Jack Moffitt):

Why bother with RSS at all?

I started pulling out all of the non-Atom bits. He also helped me grok a template scope problem that was leading me to curse XPath royally – thanks Jack!

So after many, many rounds of edit/test/curse I boiled it down to a basic template (tho I did keep some of their snazzy person and url templates – *very* nice!) and then proceeded to tweak the HTML it generates.

So if you pass in this Atom payload:

<entry xmlns=“http://www.w3.org/2005/Atom”>
    <title type=“html”>Test Atom</title>
    <link href=“http://seesmic.com/videos/FOO”/>
    <id>http://seesmic.com/videos/FOO</id>
    <author>
        <uri>http://seesmic.com/foobarbaz</uri>
    </author>
    <published>2009-01-03T20:46:19+00:00</published>
    <updated>2009-01-03T20:46:19+00:00</updated>
    <link href=“http://t.seesmic.com/thumbnail/FOO_th1.jpg” type=“image/jpeg” rel=“enclosure” title=“Test Atom”/>
    <link href=“http://v.seesmic.com/flv/FOO.flv” type=“video/x-flv” rel=“enclosure” title=“Test Atom”/>
    <source>
        <generator>SeesmicBot</generator>
        <id>http://feeds.seesmic.com/user.foobarbaz.atom</id>
        <updated>2009-01-03T20:46:19+00:00</updated>
        <title type=“html”>Test Atom</title>
        <rights>Creative Commons Attribution</rights>
    </source>
    <in-reply-to xmlns=“http://purl.org/syndication/thread/1.0” type=“application/xhtml+xml” href=“http://seesmic.com/video/FOO” ref=“FOO”/>
    <category term=“en”/>
    <rights>Creative Commons Attribution</rights>
    <link href=“http://creativecommons.org/licenses/by/3.0//us/rdf” type=“application/rdf+xml” rel=“license”/>
</entry>

You will get this:

<html xmlns=“http://www.w3.org/1999/xhtml” xmlns:xhtml=“http://www.w3.org/1999/xhtml”>
<body>
    <div>
        <a href=“http://seesmic.com/videos/FOO”><div class=“x-escape”>Test Atom</div></a> by
        <a href=“http://seesmic.com/foobarbaz”>http://seesmic.com/foobarbaz</a>;
        <p>categories: en; </p>
        <p><a href=“http://t.seesmic.com/thumbnail/FOO_th1.jpg”>Test Atom</a> (image/jpeg)</p>
        <p><a href=“http://v.seesmic.com/flv/FOO.flv”>Test Atom</a> (video/x-flv)</p>
        <p><a href=“http://creativecommons.org/licenses/by/3.0//us/rdf”>Creative Commons Attribution</a>
        </p>
    </div>
</body>
</html>

Right now we think there is a bug in how Tigase handles this (well, actually since it’s not generating the body there is a bug *somewhere*), so I can’t show you a live example :( but as soon as the dev guys fix it, or tell me what I did wrong, I’ll make another post.

You can find the XSL template I’m using here

Update: sure enough, Bart demostrated it working on his test server so I dug deeper and discovered that the Java XSLT processor is *much* more pickier than xsltproc about errors. One of the helper templates was missing but it was never called so xsltproc just gleefully went on it’s merry way where as Java’s tossed an error and just refused to do anything more. So now Tigase is generating <body></body> *and* <event><event> payload data!

Well, ok, not quite that dramatic – but for sure the final 20% is going to turn the last of my beard gray before the year ends.

Lately Nathan and I have been battling subtle bugs in Tigase (which Artur and Bart have been great at fixing), subtle bugs in our code (oh yea, even there! ;) ) and the usual other 30 distractions as the team delivered the beta version of the new site.

With this latest round of tweaks we are very close to calling it “shipped”:

• We turned on thenode that delivers the public timeline data (or what others call the “firehose”) and that is now sending Atom payload events to it’s subscribers,
• We are seeing a more stable auto-refresh of the timelines (well, more stable after this last tweak)
• We can now dial the BOSH timeout up or down to find that sweet spot for load management
• The Tigase team delivered a great new feature: Server Monitoring which I’m hoping we can wire into our infrastructure for much more accurate stats

All in all it’s been a really crazy couple of weeks – but man is this stuff fun!

So I’m hoping that the XMPP community allows us a bit of boasting as we have a XMPP PubSub system running “in the wild”. I’m sure there are others out there, but so far they have been rather quiet on how things are done or they have quietly gone away. We are working on an architecture article for the Seesmic Developer’s site that goes into more details on how and why.

Special thanks to Jack for taking the time to review how we structured our Atom payload — hopefully it’s less messy now :)

update: stpeter says it much better than I do, and in fewer words :)

Earlier today I commented on a blog posting by the Gnip team about how they are dropping XMPP support:

Without going into a full blown post about XMPP, our take is that it’s a good model / protocol, with too many scattered implementations which is leaving it in the “immature” bucket. Apache wound up setting the HTTP standard, and an XMPP server equivalent hasn’t taken hold in the marketplace.

My first reaction to this was how they were blending XMPP protocol with customer expectations with XMPP server issues so I left a comment on their blog and that lead to a discussion that went from their blog, to identi.ca across over to Twitter and other places.

While my reading of their post still generates the same questions, one of the big points that settled out of the conversation was that Gnip’s issue was with the ecosystem that surrounds any given XMPP server – not XMPP itself! Still quite a big generalization IMO but a point that I would have to agree with – that’s just the danger of using open-source projects in your business plan.

The problem in a weird way is the success of LAMP (Linux, Apache, Mysql, Python/Perl) and particularly the Apache part of the LAMP stack – it has become a ubiquitous part of any Linux install and there are tons and tons of information and tools available to scale an HTTPd server in any direction you want.

But that totally misses the point – XMPP is not a stateless protocol that was designed for the job it does. Of course it’s going to behave differently and require new skills to manage properly. I remember going thru this exact situation when Apache was the new kid on the block and people were using it because development had stopped on the NTSC HTTPd code base (yes, I am that old.)

Back then it had the same issues and documentation basically meant reading patch and revision histories. As the web grew and businesses grew and things progressed, Apache became very stable and is now almost a no-brainer. Tho I challenge folks to setup an Apache server that does HTTPs, Bosh and can handle a mix of static files and dynamic requests – it’s not really designed to do all that at the same time.

XMPP is at that point now – it’s stable and full of features that are making developers want to include them in their plans but all of the current documentation and use-cases are all XMPP server developer driven. It’s only recently that PubSub has gone from the server dev realm into the systems realm and the stress points are showing :)

That’s what we have been doing at Seesmic – trying to get the final 20% of the classic 80/20 rule implemented with Tigase. It’s also what folks like Jack over at Chesspark are trying with ejabberd and others.

more later about admin tools, documentation and other things that make life in the Ops world easier and that are lacking.

sorry for the rambling post :)