Bear’s Journal

The new dayjob has been keeping me very busy so I’ve neglected working on parsedatetime.

An email reminding me that I had checked in 0.8.5 with the damn debug flag still enabled gave me the poke I needed to go thru the Issues listed (wow - people have been filing bugs!) and generate a new release.

See the Changes.txt entry for 0.8.6 for details.

It can be found via the Python Package Index, Google Code or my own
website.

Thanks to Bernd Zeimetz (Debian maintainer) for submitting a patch to localize parsedatetime for the German (de_DE) locale!

Technorati Tags: news, parsedatetime

Over at Seesmic we have setup a server for 3rd Party devs to start testing their tools and apps.

Announced over on my Seesmic API page

Technorati Tags: api, developers, news, seesmic

Matthew and I have been working out some of the core ideas behind the new RESTful API that I am suggesting to Johann that Seesmic adopt. Basically we looked at what the clients and third party developers would need to consume the information that the servers inside of Seesmic generate and came up with a basic set of data.

Once we had that data set identified it was really straightforward then to create a REST API that delivers the information in a consistent manner. Another benefit of “going REST” with the API is that we can take full advantage of the features Merb offers - it supports out-of-the-box REST style routing and does most of the dirty work for you in a sane manner.

Once we realized with a quick prototype the advantages (and a few disadvantages for sure) that Merb was offering some of the other ideas I’ve been working on started to fall into place:

• memcached cloud support
• API proxy to control outside access to the API
• OAuth/OpenID based authentication without having to bother the server
• Scalable either up or out - very nice
• Already in use by large sites

So, now that we have settled on Merb and some of the other ideas, we get to create a working demo and show the team and convince them we are not crazy!

Technorati Tags: api, seesmic

Saw a serious flurry of JDev chat-room messages earlier today and was wondering what the fuss was all about and then I read this post by Florian and just about wet myself
That is great news that AOL, one of the bastions of closed-source’ness is adopting an open-source protocol (and XMPP to boot!)

While it’s still alpha quality, I look forward to ditching all things Oscar from my code.

Technorati Tags: news, xmpp

Ok, a little background. My new job is working for Seesmic and there I’m helping in a number of ways and one of them is to map out with the team a way to take the current infrastructure and make sure it can handle higher loads.

Currently there are about 3,500 users with about 400 or so active at various times of the day and those 400 or so users have generated approximately 850 videos per day with peaks on the weekend topping out at 1,650 videos. Soon the boss wants to open the door to the approximately 11k users on the waiting list!

Now you know what I’ve been dreaming about lately - and it hasn’t been about nice fluffy bunnies but rather the kind of bunny that was in Monty Python’s Holy Grail movie
I’ve been reading on how various other systems are handling the issue of scaling - from YouTube to Twitter. YouTube in some ways is similar as they are in our domain (videos) but their videos are all stand-alone uploads that are batch processed and don’t have any strict time constraints on when they appear. Our videos are expected to be available within a minute as the poster is taking part in a conversation so any major delay would just ruin the experience.

Twitter has a similar real-time update issue but only sends small packets of data (but admittedly to huge numbers of users) unlike our video data (currently FLV but soon also MP4).

So while YouTube solved it with some snazzy footwork in the database department and Twitter with a custom Ruby message queue server they call Starling - we will have to create a hybrid solution that takes advantage of pieces of all of them We know memcache will play a part, load-balanced httpd servers also (for REST API and file serving) but they will need to be stitched together in interesting ways. Already we are using XMPP to interconnect different parts of the system that need to know about events and are using Jabber bots to interact with users, but the final solution is still shrouded in clouds.

We were using ActiveMQ and thought about RabbitMQ to solve some of the stability issues ActiveMQ was giving us, but we need something more than just a message broker. And with already using XMPP we will probably switch to using an XMPP server to enable a single user account, user-to-user chat, group features - in other words all the things that XMPP is quite good at doing.

So, that’s the introduction to the problem, I’ll be posting more information as we chew thru the problem and start testing what works and what doesn’t.

Technorati Tags: seesmic