responding to Jesse’s call to “open twitter’s api”
Evidently Jesse Stay and I had similiar thoughts about all of the chatter about the adoption of Twitter’s API by WordPress and Tumblr (among others) and while I spent the day watching and shoveling 2 feet of snow Jesse was way more productive and put forth a group project to think about how the API could become more open and not tied to any specific vendor :)
At first I was thinking that this idea has already been tossed under the bus by many folks who thought about creating open versions of Friend Feed and I even thought that in a way it’s already being proposed by the Laconi.ca (aka Status.net) crew and the OMB protocol but I realized that I was focusing on how I would implement such an animal and not how to enable data transfer, which is after all the core part of what an API offers.
After realizing I was giving the thought some serious short shrift so I sat and chewed on it and came up with an idea and also an immediate concern:
First the idea, it should be a gateway that allows for sending/receiving of items using one of the many existing excellent examples of data APIs
Any one of the above would make an excellent gateway API interface as they allow for all of the elements that exist in a Twitter item.
The one immediate concern I have is about identity. As a gateway, the system would need to understand how to map identity from/to the incoming and target network so that a proper flow of the conversation can happen.
The good thing is that recently a lot of work has happened recently on just that item :) and I think that the newer Web Finger and the existing OpenID options could work because already quite a few of the larger data silos allow for mapping of OpenID to their internal systems. The protocols above also all, if I remember correctly, have some method of storing enough metadata to allow for the incoming item to maintain it’s integrity even if on delivery some items are lost – the gateway would be able to remember the transaction even if the target mangles some of it.
With the proper storage of metadata then other protocols like Salmon can be used to track the conversation across all of the “jumps”.
I think the important part is to not create yet another data silo that has to be maintained and polled just to allow for all parts of the conversation to maintain itself – rather I would love to see this as something that could be used to enable each unique identity to maintain the information required and become the master of their own conversations. Of course large sites could offer internal proxies to their own user base who don’t wish to implement their own, but it would also allow power users to do so and then the other gateways and exchanges points become secondary backups and the whole thing becomes a lovely distributed system.
Just some quick thoughts that I wanted to get down before I go to sleep
Tags: api, developers
20. December 2009 at 06:45
The problem with the Twitter API is that it's only “publishing” part… the hard (and most interesting) is by far the pubsub, broadcasting part.
20. December 2009 at 06:50
I think I agree :) 100% with you and for sure I was doing some “hand waving” in my post because I didn't want to get into a lot of details.
But yes, receiving of the incoming post is the easy part, figuring out the intended target and any secondary sinks that may be interested in it is another concern.
My thought is that that problem is solved by the sender picking a gateway that is aware of the sender's preferred choices and knows how to broadcast to more than just the target. This allows the sender to use more than one gateway or to chain gateways depending how much, and what type of, service they provide.
20. December 2009 at 06:56
My approach toward this : every user, whatever network they're own should be able to subscribe to any user on any service….
I haven't said anything new so far :) Any user on any service is only availbale through RSS/Atom feeds. On Twitter you're http://twitter.com/statuses/user_timeline/16133..., while on your blog, you're http://code-bear.com/bearlog/feed/atom/ etc…
So, for me the “open-twitter” (or rather open social network) is something that allows me subscribe to feeds.
Then, these feeds have to be “richer” then just feed/entries/link. I want activitues and more dettails about them => ActivityStreams come into the game.
The whole “push” and federation system is built via PubSubHubbub hubs. Your feeds should always tell which hub they “ping”. So when I subscribe to you (your feed, wherever it is), my “social network consuming tool” (a feed reader on steroids actually… ask Loic to do that!) should be able to know where to get the data from (which hubs). It then asks the hub to ping him.
What do you think?
20. December 2009 at 07:05
I think we are very close to being on the same page and the differences are, I hope, minor because I believe your approaching it from a
distribution/gateway perspective and i'm approaching it from a personal micro blog publication tool. (or at least that's how I see it right now)
Seeing it all as feeds makes perfect sense when looking at the data flow from one node to another – a feed is nothing but one form of the API that a
gateway would allow for retrieving information. But that also shows the problem that i'm worried about – identity. How do you maintain that when you only
consuming the item as a feed, even a “rich” one such as an activity stream.
20. December 2009 at 08:19
Fully agree with you, once more :) For the identify, I think OAuth isn't the answer (or maybe I'm missing something). I think eventually, we will have to accept that identity cannot be enforced accross social networks and that julien on twitter may not be julien on identica. the only way we can deal with that is just provide better 'user search' apis (twitter's suck to that regard)
20. December 2009 at 11:45
The problem with the Twitter API is that it's only “publishing” part… the hard (and most interesting) is by far the pubsub, broadcasting, federating part.
20. December 2009 at 11:50
I think I agree :) 100% with you and for sure I was doing some “hand waving” in my post because I didn't want to get into a lot of details.
But yes, receiving of the incoming post is the easy part, figuring out the intended target and any secondary sinks that may be interested in it is another concern.
My thought is that that problem is solved by the sender picking a gateway that is aware of the sender's preferred choices and knows how to broadcast to more than just the target. This allows the sender to use more than one gateway or to chain gateways depending how much, and what type of, service they provide.
20. December 2009 at 11:56
My approach toward this : every user, whatever network they're own should be able to subscribe to any user on any service….
I haven't said anything new so far :) Any user on any service is only availbale through RSS/Atom feeds. On Twitter you're http://twitter.com/statuses/user_timeline/16133..., while on your blog, you're http://code-bear.com/bearlog/feed/atom/ etc…
So, for me the “open-twitter” (or rather open social network) is something that allows me subscribe to feeds.
Then, these feeds have to be “richer” then just feed/entries/link. I want activitues and more dettails about them => ActivityStreams come into the game.
The whole “push” and federation system is built via PubSubHubbub hubs. Your feeds should always tell which hub they “ping”. So when I subscribe to you (your feed, wherever it is), my “social network consuming tool” (a feed reader on steroids actually… ask Loic to do that!) should be able to know where to get the data from (which hubs). It then asks the hub to ping him.
What do you think?
20. December 2009 at 12:05
I think we are very close to being on the same page and the differences are, I hope, minor because I believe your approaching it from a
distribution/gateway perspective and i'm approaching it from a personal micro blog publication tool. (or at least that's how I see it right now)
Seeing it all as feeds makes perfect sense when looking at the data flow from one node to another – a feed is nothing but one form of the API that a
gateway would allow for retrieving information. But that also shows the problem that i'm worried about – identity. How do you maintain that when you only
consuming the item as a feed, even a “rich” one such as an activity stream.
20. December 2009 at 13:19
Fully agree with you, once more :) For the identify, I think OAuth isn't the answer (or maybe I'm missing something). I think eventually, we will have to accept that identity cannot be enforced accross social networks and that julien on twitter may not be julien on identica. the only way we can deal with that is just provide better 'user search' apis (twitter's suck to that regard)
30. December 2009 at 18:55
http://microformats.org/wiki/rel-me should be enough to say twitter julien = identica julien, no?
30. December 2009 at 22:40
that works perfectly if the aspect of trust exists – the problem with self-referential identity is that it allows for easy spoofing that looks authentic. now if rel-me pointed to some sort of signed or trusted identity profile where other items can be discovered, that would be the killer item IMO
30. December 2009 at 23:42
Reciprocal links, then, like google's social graph?
31. December 2009 at 00:01
yes! it's just like PGP key signing, the chain of trust must include one link to a service or item you trust *and* you have to be able to out-of-band verify that expression of trust is reciprocal. sounds easy but hard to do within the web framework IMO