Pingback protocol sequence
Position Paper for Federated Social Web 2nd-5th May 2011, Berlin (Germany)
Authors:
FOAF profiles bind together the most widely deployed distributed Social Network in existence, despite the lack of widely agreed upon ways to automate the creation of new friending relations. A similar problem appeared on the world wide distributed social conversation that is the blogosphere, where users were keen to keep track of articles mentioning ones they posted. A number of solutions were developed and widely deployed. We describe these solutions here and show how they can be extended to the social web, to the benefit of all, in a light, easy to use and Web Architecture friendly manner.
The Social Web, as a decentralized system, requires a friending mechanism. Let us illustrate this with a short use case. Romeo meets Juliet at a party and they get along well. Juliet enjoys Romeo’s conversation and gives him a card with the URL of her home page or blog. Later Romeo enters one of those links into his Social Web Server (SWS), which then displays Juliet’s information and gives Romeo the opportunity to add her WebID to his friend list. Currently, (May 2011) there is still no widely deployed way for Romeo to alert Juliet of his having added her to his list of friends as Juliet does not publish her e-mail for fear of spam. And even if she did provide an e-mail, Romeo does not want his friendship with Juliet to be dependent on her ability to edit XML files.
What Romeo wants is for this process to be automated by their SWS. Romeo’s server should upon updating his foaf profile be able to ping Juliet’s server. On receiving the ping, Juliet’s SWS should verify it, and inform her (now or later) of this friendship request. If she accepts, her SWS can then give Romeo greater access to her restricted resources by allowing him to authenticate using the WebID Protocol.
Here we are doing something very similar to what is known on the blogosphere as pingback. We will describe these protocols and then show how we can extend them to cover the friending mechanism, and much more.
The visibility of a Web resource is determined by the quality of the links to it. The Blogosphere noticed this very soon and sought to help automate the weaving of the distributed conversations that make it up. This resulted in the development of a number of protocols that were designed to make it easy for web sites to link back to blogs that referred to them: Refback, Trackback and Pingback.
Refback is the simplest of all pingback protocols. It simply uses the HTTP Referer header that browsers include in HTTP requests and that web servers log to find pages that link to a given resource. The server can then check the referred-to resource to verify the link.
An HTTP Referrer header will be generated whenever a link is clicked in a browser and is also very easy to generate outside the limits of the browser. But as a result it is impossible for the server to distinguish intentional and unintentional pings.
The Trackback protocol, developed in 2002, was the first to enable intentional pings. If Juliet published an article referring to her dear friend Nora’s entry on “Nightingales”, Juliet’s blog server would find the link in the newly published blog entry, fetch Nora’s post and search it for a piece of embedded RDF/XML to discover the resource to ping. Juliet’s server could then POST the Trackback ping consisting of the title and url of Juliet’s new post and an optional excerpt of the content as well as the blog_name. As a result of this Nora’s server would publish a pointer back to Juliet’s article containing the excerpt sent in the ping. She might do this only after checking with Nora, of course.
As is clear from these field names, this protocol was designed for and limited to blog posts, and so would not be suitable for other usages, such as a the friend ping we are interested in.
Trackback also had the further disadvantages of tying the protocol to a representation embedded in a representation - RDF/XML inside a comment inside HTML - for very little benefit and with no standard support, making parsing very difficult. This is currently considered very bad practice in the semantic web community, which has developed RDFa to allow a clean expression of RDF in html.
The success or failure of the ping was described in a specially developed XML format returned in the response, with a field containing 0 for failure and 1 for success. Clearly the standard HTTP status codes should have been used instead.
Pingback removed the RDF/XML embedding for discovery of the ping service, and provided the choice of two standardized methods: a link relation of type pingback in the html header: <link rel="pingback" href="pingback server" />, or an X-Pingback attribute in the HTTP header: X-Pingback: http://charlie.example.com/pingback/xmlrpc.
This allowed any resource, including pictures and videos to define their ping service, so that their owners could be made aware of resources that referred to them.
The format of the ping used XML-RPC in order to notify the pingback service with two pieces of information: the URL of the linking resource and the URL to the linked-to resource, which is exactly the information produced by the Refback protocol. So one may wonder then what does Pingback really enabled over and above Refback? Apart from using POST and making intentionality of the ping clear as Trackback does, the only other answer is that it enables third-party ping services. This allows people with resources posted on servers they do not control to nevertheless receive pings.
The XML-RPC protocol was popular at the beginning of the millennium but now feels very dated. RESTful web services would now shy away from hexadecimal error responses encoded in XML.
Semantic pingback makes it easy for documents in any format to declare a pingback service. It does this by abstracting from formats and defining the protocol at the semantic level. A format need just specify a transform into RDF. XML formats can do this using GRDDL, and JSON formats can use json-GRDDL, similar transforms could be created for most other format.
The main advantage of Semantic Pingback is the ability to accept not only dumb untyped links from the source to the target but also typed links in the sense of an owl:ObjectProperty and any other statements which uses the target as subject or object.
This allows any kind of remote comments, polls or any other activity just over Semantic Pingback and goes far beyond the friending use-case described here.
Any resource can point to a simple xml-rpc pingback service by declaring a relation from the resource to such a service in one simple statement written in Turtle as:
@prefix ping: <http://purl.org/net/pingback/> .
<> ping:service <http://pingback.aksw.org/> .
This allows integration with existing deployments of XML-RPC pingbacks. But it also allows a more RESTful protocol which we will next describe.
A resource can describe its relationship to a RESTful ping service by relating itself via the ping:to relation to a ping:Container. In Turtle the relation would be written as:
<> ping:to <http://pingback.aksw.org/> .
An html representation could include the above relation using RDFa as shown here:
<html xmlns:ping="http://purl.org/net/pingback/">
<head><link rel="ping:to" href="http://pingback.aksw.org/"/></head>
<body>your html page</body>
</html>
The same can be done in any other format for which the semantics are clearly specified as explained earlier.
The advantage of this relation over the ping:service relation that points to an XML-RPC endpoint, is that the ping:Container linked to is just a normal web resource defined as one that returns in its html representation a form with the input fields "source", "target" and "comment" and the action="" attribute.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" xmlns:ping="http://purl.org/net/pingback/">
<head><title>Test PingBack Endpoint</title></head>
<body typeof="ping:Container">
<form method="post" action="">
<p>source: <input type="text" name="source" /></p>
<p>target: <input type="text" name="target" /></p>
<p>comment: <input type="text" name="comment" /></p>
<p><input type="submit" name="submit" value="Send" /></p>
</form>
</body>
</html>
The pingback.aksw.org container was the first one to be deployed: try it out!.
Here, a human being that follows his nose to the form in a browser should be able to fill it out by hand and get the same result a robot would. A robot will know that it can send a form with those fields to that container because it has declared itself to be a ping:Container with RDFa such as <body typeof="ping:Container"> shown above.
Notice the action="" which means that the POST will go to the same resource from which the html came. This means that the robot sending the ping can be sure that the resource has the same interpretation of the meaning of the source, target and comment fields, and not be a request of a totally different nature, such as the commitment to buy a product for example. This is important because html forms fields are not namespaced, and their meaning is therefore resource dependent.
The pingback server on receiving the POSTed form can verify that the source does indeed link to the target, and depending on a local policy, notify the owner of the target. In our initial example Romeo’s SWS would find the ping endpoint from the blog or home page given by Juliet, and send a ping with an optional message. This message would be verifiably attributed to Romeo, if his server connects over TLS and authenticates with WebID. Juliet’s server would then verify that Romeo’s foaf file does indeed point to Juliet - which may in fact be only visible to Juliet's SWS if it authenticates as Juliet to Romeo's SWS using WebID. It is then up to Juliet's SWS to use this information as it sees fit. One would expect it for example to forward the message to Juliet with information about the place of Romeo in her social network as backing evidence, and if possible verifications of as many claims as it can in the foaf profile. From there Juliet could accept the request and add Romeo to one of her friend categories, giving him more or less access to the protected resources on her server when he next connects and identifies himself with WebID or OpenID. Having done that Juliet can also send Romeo a ping...
The access to the ping Collection can also be protected using WebID, allowing pingback services to find the identity of the pinger and to restrict ping senders for certain resources to a specific group. Juliet's SWS could by protecting her endpoint with WebID discover the WebID used by Romeo's SWS, and so present this ping correctly as coming from Romeo himself.
The RESTful pingback protocol makes it very easy to grow one’s social profile. But it can be used for a lot more. Some ideas would be:
Compared to Refback, RESTful pingback can send a comment, is more extensible, and can work with servers the user has very little control over.
Compared to Trackback, RESTful Pingback (RP) comes a decade of semantic web development and research later. It works nicely with html and does not require any specific formats to work.
Compared to Pingback, RP can work with most formats, does not require XML-RPC, works with HTTP return codes for error messaging, allowing robots and humans to work together (by simultaneously accessing the same resource).
Further studies would need to compare this to the Salmon and OStatus stack. Some intial thoughts are that the ping:Container can of course return a representation in Atom format marked up with the Atom Activity Extensions (Note that the Atom Activity Extension can itself be described semantically as the Atom Activity Streams RDF mapping project is doing.) If the Container could return an Atom format, then there is no reason why one could not also POST an atom entry to such a container too. pingback.aksw.org already accepts both XML/RPC POSTS and the RESTful pingback. It certainly seems possible to allow certain types of Atom Entries to be posted too.
If there was a way of semantically marking up forms then the attribute names would no longer need to be hard coded as they are now. A better mapping from html forms to the semantic web would be very useful. So would better and more flexible html forms standards, which would allow the post a graph of relations. Perhaps SPARQL UPDATE can be thought of as laying the ground for that.
The W3C wiki Pingback page points to a current list of implementations of the protocol. The following implementations were important to the writing of this paper:
foaf:Person) has a mailto: address, the service will send a ping notification message to this address.