In another post, I discussed a comparison of various alternative architectures for the Web found in section 5.2.1 of Roy Fielding’s dissertation. The focus of this analysis was the transfer of information from server to client as the movement of data to the processor distinguishes REST from other architectural styles that move processing agents to the data.

In this post, I will re-examine these alternatives, extending the analysis to the information flowing from client to server and the protocols used to exchange information. By doing so, I hope to clarify the advantages provided by REST for input event processing.

Option 1: Client-Server

The first alternate architecture is based on a client-server model where information is rendered on the server and the resulting image is sent to the client. As discussed previously, this provides data encapsulation and decoupling at the expense of bandwidth usage and increased server-side processing. Also, the creation of secondary clients such as spiders is inhibited.

Now, let’s turn our attention to input processing: in such a system, the client has no knowledge of what input events are relevant to the application and what effect they might have. The only information the server has provided the client is an image to display on the screen – the client has no idea where the links or other input controls are located. There is nothing to tell it what areas of the screen respond to mouse-over events, clicks, etc. And so every mouse or keyboard event must be communicated to the server just in case they are relevant. This results in wasted bandwidth, and wasted server-side processing of irrelevant input events. The responsiveness of the client would likely seem quite slow as well because even the most minor change to the display would require a round trip to the server.

Option 2: Mobile-Object

The second option is a mobile-object style architecture where the server sends a combination of both the data and a rendering engine (i.e. processing logic or code) to the client. Here, the client-side events can be filtered and processed by the mobile code running on the client, allowing communication with the server to take place only when it is truly needed by the application. The processing of an event might result in an update to the rendered image based on local processing and in other cases result in a request to the server, though many events could simply be ignored.

The client also has the ability to establish a connection with the server and communicate using virtually any protocol as the protocol stack is included in the code that comprises the mobile-object and the communication protocol can be optimized for individual applications (though this flexibility could not be afforded to the protocol used to download the initial mobile object). Thus client-to-server bandwidth-usage, user-perceived latency and server-side processing are optimized. This comes at the expense of visibility as intermediaries are not able to interpret the information exchanged between the client and server and thus cannot perform any processing on the data or optimization of the communication flows (e.g. caching).

Option 3: Raw Hyper-Data

In the third architecture the server sends the raw data to the client along with a media type that indicates the data format allowing the client to choose a rendering engine for the data. Here, the downloaded data not only drives the rendering of output, but also provides instructions to the client on how to interpret input events –  ignoring irrelevant events, handling events locally where possible and generating requests to the server for more data in other cases. Thus the client can be described as possessing an interactive rendering engine.

Unlike Option 2, these rendering and input-handling instructions do not take the form of downloaded code, instead they are embedded in the data as declarative controls. Control data informs the rendering engine on how to solicit input events from the user, for example, indicating that a certain string of text should be clickable and should therefore be rendered in a special way (underlined and blue) so that it (perceivably) affords clicking. The control data also tells the rendering engine that mouse click events targeted at the area of the screen occupied by the text must be processed in a certain way, for example, by downloading new data to drive the rendering engine.

In this architecture, mobile code implementing a protocol stack cannot be downloaded and run by the client. Instead, communication must be restricted to a protocol that is already understood by the client. The control data indicates how to construct the data to be sent to the server in requests. This request data may be constructed from other control data (e.g. a logical name for the next set of data to download performing a similar function to a URI) or from data received from the user via previous input events (e.g. keyboard input event).

This architecture has some of the same benefits as the mobile code architecture in that the client is able to process events locally rather than forwarding them all to the server. However, in this case event processing is restricted to the set of control types that can be expressed by the data format. Also, just as this architecture requires the client and server to pre-agree on a data format (inhibiting data encapsulation), they must also pre-agree on a set of data transfer protocols (restricting interoperability). However, as discussed in my earlier post, this comes with the advantage that because data is downloaded instead of code, not only is less bandwidth likely used but also the client has more flexibility – it is not restricted to running the code; it can perform alternate functions such as index spidering without actually rendering the data. The fact that the control information is embedded in the data enhances this capability allowing these alternate non-rendering clients to not only understand how the data would be rendered as output but also how the interactive rendering engine would process input events.

REST

As discussed in my earlier post, Option 3 is not quite REST.  REST additionally requires that data be transferred in a representation format “matching one of an evolving set of standard data types, selected dynamically based on the capabilities or desires of the recipient and the nature of the resource”, addressing the data encapsulation issues in Option 3. REST also requires the use of an evolving set of standard transfer protocols (that can all be mapped to a uniform interface). This addresses the issue in Option 3 where the client and server are required to pre-agree on a (potentially non-standard) protocol. In REST, a resource is referenced by a URI that uses a standard URI scheme. The URI scheme tells the client which standard protocol to use; this reduces pre-agreement between client and server to an agreement on a common space of standard protocols. This use of URIs to identify resources also allows the data format standards and the data transfer protocols to evolve separately (Larry Masinter has published a great discussion of this principle and it is also discussed in the first “rule” in Roy Fielding’s guidelines on REST APIs). Thus, the use of standard formats and transfer protocols allows REST to address the client-server coupling issues of Option 3.

REST also adds the use of code-on-demand to Option 3, allowing scripts and bytecode to be used as representation formats. This allows REST to achieve many of the benefits of Option 2. While Option 3 restricts the client’s rendering and input processing capabilities to what can be expressed by the data format, REST allows scripts and embedded objects to extend the client’s capabilities. Increased use of code-on-demand comes at the cost of decreased visibility – a tradeoff that must be kept in mind when designing a system. Practically, the more script-driven a web site is, the less non-browser clients (spiders, intermediaries, analytics and authoring tools, etc.) can do with it.

It should be noted that REST does not relax the constraints that require standard formats and protocols in the context of code-on-demand. For example, HTTP is used for communication performed by browser scripts in an AJAX application. To some, this may seem like a disadvantage, providing Option 2 with more flexibility than REST. However, standard protocols have some clear advantages over custom protocols. While I will leave a detailed discussion of this point for a future post, the advantages primarily come from the ability of intermediaries to act on communications, providing services such as caching and content transformation.

In my last post, I asked if adding links to data was sufficient to turn it into hypermedia. It seems that there is more to hypermedia than links; hypermedia is an event filter. In a RESTful system, input events from the user are processed by the client according to the controls specified by the currently-loaded hypermedia document. These controls determine if an event is simply dropped, transforms the document or application state in some way, or is translated into a request on the uniform interface for a new document to be loaded. Embedding controls in the data transfered to the client offers performance benefits over a client-server model that transmits all events to the server, and visibility benefits over a mobile-code architecture where downloaded code processes events. Controls are more than links; hypermedia is more than linked data.

1. It provides a generic definition of a “link”;
2. It establishes a registry for link relations; and
3. It defines the HTTP link header.

The first point is one of those things that surprisingly hadn’t been done before – at least as far as I know anyways. Sure, links have been defined in the context of specific formats, and the semantic web has a fairly generic definition of a link, but the web linking RFC provides an application and serialization agnostic definition, which is a pretty useful thing to have.

The RFC defines a link as a typed connection between two resources: a context resource and a target resource. The resources are identified by URIs (well, technically IRIs) and the type is identified by a link relation. A link relation identifier can be one of the relation types from the registry established by the RFC, or it can be an extension (non-registered) relation identified by a URI. The type defines the semantics of the link. For example, the “stylesheet” relation means that the target resource is a stylesheet for the context resource.

The HTTP link header provides a mechanism for specifying links in an HTTP response when the representation format being used does not natively support them. For example, a response to an HTTP GET request for a PNG image could use the link header to indicate that the resource is linked to other resources in various ways.

Now some might not immediately see the benefit of this specification. There are tons of (so-called) “RESTful” APIs out there stuffing links in JSON data structures all over the web right now. For example, what about the following HTTP request:

GET /someresource.json HTTP/1.1
Host: example.com

and response:

HTTP/1.1 200 OK
Content-Type: application/json

{ “node” : {
  “id” : “http://example.com/identifiers/12345”,
  “next” : “http://example.com/nextres.json”,
  “value” : “http://other-example.org/somelink.html”
}}

Well this data structure seems to contain 3 URIs – but does it contain three links? It’s hard to tell without knowing more about the data format, but if we make some assumptions from the names, the properties demonstrate 3 distinct uses of URIs. The “id” property uses a URI as a unique identifier, similar to how URIs are used in XML namespace identifiers or in the Atom id element. The “value” property is just a string, it could be anything, but in this example that string happens to contain an http URI – one might even argue that this isn’t really a URI at all, but just a string that looks like one. The “next” property is the only link out of the three, providing the URI of another node that follows the one we are looking at (i.e. the nodes form a linked-list).

Of course, if the property names were “x”, “y” and “z” we’d have a much harder time figuring this out. This is an example of where serving data with a content-type of “application/json” (or “application/xml”) falls short. You can’t tell how to interpret the data just by looking at the response – you need some extra information or documentation to understand it. This is where self-descriptive messages are critical – the response would be self-descriptive if the data format had its own media type (e.g. “application/node”) with an associated specification that described how to interpret the individual properties. This specification would define the semantics of the link relation associated with the “next” property (i.e. this format doesn’t use a generic link construct with an extensible relation field), allowing the consumer to follow the link to fetch the following node in the list. While this format specification could also be considered “extra information or documentation” the difference is that it isn’t specific to the URIs you are interacting with (in a linked environment you can find yourself interacting with new URIs that you hadn’t expected) and relevant specification can easily be identified via the communicated media type.

This hopefully makes it clear that it is not sufficient for data to simply contain URIs for it to be “linked”. There must be a specification of the format that identifies those URIs as links, and either defines the link semantics or how they can be determined. The link might be part of a generic link construct like the Atom and HTML <link> elements, referencing a relation from the link relation registry that provides the link semantics. Alternatively, the link semantics might be defined in the data format, as was the case in the “next” property from our example.

The Web Linking RFC is really useful because you don’t have to go and redefine what a link is in every data format you define – you can reference the definition in the RFC. The specific link types in your format can reference the definitions in the link relation registry. For example the definition of the format used in our example above could simply reference the “next” relation in the registry. Alternatively, a generic link serialization could be used that leaves the relation as a field allowing any relation in the registry (or a proprietary relation) to be used.

For example, the response in the example above could be re-defined as follows:

HTTP/1.1 200 OK
Content-Type: application/node

{ “node” : {
  “id” : “http://example.com/identifiers/12345”,
  “links” : [ {
    “href” : “http://example.com/nextres.json”,
    “rel” : “next”
  } ],
  “value” : “http://other-example.org/somelink.html”
}}

(Update: I changed the content type here from “application/json” to the fictitious “application/node” media type — see Nathan’s comment below. We assume that the specification for this type properly defines the “links” field as an array of links.)

Alternatively, the link header could be used as follows:

HTTP/1.1 200 OK
Content-Type: application/json
Link: <http://example.com/nextres.json>; rel=”next”

{ “node” : {
  “id” : “http://example.com/identifiers/12345”,
  “value” : “http://other-example.org/somelink.html”
}}

However, this has the disadvantage of separating the link from the content – if you store the body somewhere (e.g. to disk) you lose the link information that was sent along with it. It also means that you can’t communicate the link information if you communicate the structure using a protocol that doesn’t support the link header (e.g. FTP). It is best to embed the link information in the response body unless the format doesn’t allow it (e.g. it’s an image).

I hope it’s clear that the Web Linking RFC makes it really easy to add links to any resource on the web. But does this mean that any resource can be turned into hypermedia? Is a linked resource the same thing as hypermedia? As discussed in my last post, we can summarize “hypermedia” as data-guided controls. Is adding links to data sufficient to provide “data-guided controls”? This is something I’ll continue to explore in future posts.

Let’s take a look at the Wikipedia definition of “hypermedia”:

Hypermedia is used as a logical extension of the term hypertext in which graphics, audio, video, plain text and hyperlinks intertwine to create a generally non-linear medium of information.This contrasts with the broader term multimedia, which may be used to describe non-interactive linear presentations as well as hypermedia.

This seems to define hypermedia as an extension of media designed for human consumption. So does it make sense to use the term hypermedia for something that isn’t consumed through some sort of user interface?

Perhaps hypermedia has a slightly different definition in the context of REST. Instead, let’s look at the definition of distributed hypermedia in Roy Fielding’s dissertation:

Hypermedia is defined by the presence of application control information embedded within, or as a layer above, the presentation of information. Distributed hypermedia allows the presentation and control information to be stored at remote locations.

In a more recent 2008 ApacheCon presentation, Fielding defines hypertext as:

The simultaneous presentation of information and controls such that the information becomes the affordance through which the user obtains choices and selects actions.

and then sums it up as:

Hypertext = data-guided controls

This is interesting — “controls” implies the ability to effect change in an application through some sort of input or action. In the field of design, the term “affordance” means the set of possible actions that a user can take, though it is more often used to mean the set of possible actions that the user is made aware of – the “perceived affordance”. An on-screen control “affords clicking” if the user believes that this is a useful and meaningful action to take.

In a browser, hypertext determines what text and graphics are presented on the computer screen as well as what on-screen controls are made available. Realizing controls not only requires communication to the user of what areas of the screen can be clicked or respond to keyboard input, but also effecting the response to this input. The input should of course be meaningful as the user is being made to perceive it as such.

In short, the hypertext informs the browser how to turn input and output resources (a screen, keyboard and mouse) into interactive information. This is actually very much in line with the Wikipedia definition we started with which, in addition to the non-linear nature of hypermedia, identifies interactivity as a characteristic which separates hypermedia from multimedia. Perhaps Wikipedia wasn’t such a bad source after all!

This leads us to the question of how to realize interactivity in a machine-to-machine context. In the presentation slides referenced above, Fielding notes:

Hypertext does not need to be HTML on a browser – machines can follow links when they understand the data format and relationship types

This evokes the notion of a spider crawling through linked documents, and it is certainly common to see attempts at building RESTful clients take an approach that is similar to spiders. However, in the context of HTML, spiders are form of “secondary client”. Unlike browsers they do not realize the controls described by the hypermedia document. HTML is a declarative format – it is a description of the interactive output of the browser. As described by the Principle of Least Power, a secondary client like a spider can analyze what the browser would do when given a specific hypertext document as input without actually realizing the presentation of information and controls itself. If hypertext documents were instead written in an imperative language like Java, this would not be possible.

Because a spider is able to determine what effect the activation of a specific control (e.g. clicking a link) will take, it is able to perform the same action itself (e.g. GET the document referenced by the link URI). This is not the same thing as using the control as realized by the browser. The browser uses graphical means (usually blue, underlined text and a special mouse cursor) to indicate that a section of text is a link that can be clicked. The browser receives an input event to tell it when the section on the screen where the link resides has been clicked which triggers the associated action. The text itself provides additional details to the user regarding the meaning of the link, whereas the spider may use this text as well as other information, such as a link relation, that is hidden from the user.

A spider does not use the controls described by the hypertext document — even if it could realize the controls, they are designed to be used by human beings. Rather, it attempts to understand the meaning of the control based on the declarative description of what is conveyed to the user about that control as well as control metadata such as link relations. Based on this information it determines whether to take the action associated with the control. Because of this, spiders are typically limited in what they can do. For example, spiders usually cannot fill in and submit a form without some out-of-band knowledge about the web site (e.g. that the form is designed to capture the details of a book purchase).

To deal with this issue, machine-specific control information is often layered onto a user-specific HTML page. Link relations, microformats and the like are constructs for this purpose. Another approach is to use an entirely separate format from HTML for machine interaction with a service. Unfortunately, the controls offered to the client programs are quite anemic, modeled after the <link> tag in HTML with nothing but a simple link relation to drive the control. Interestingly, the <link> tag is not associated with a UI control in HTML; the de facto standard for machine-to-machine hypermedia controls isn’t a control at all. It is a declaration of a typed association between two resources – not the same thing. The <a> tag is a control, but <link> is not even though they are both types of “links”.

I suggest that a new approach to hypermedia design is required to address the needs of machine-to-machine systems; one that is based on the design of data-guided controls that are appropriate for the specific machine-driven clients that are relevant to a problem space. An approach that treats a machine control as an analogue of user interface controls: a construct that provides an equivalent to perceived affordance suitable for machines and processes input events from machines. I intend to explore this further in upcoming posts.

Self-descriptive means that the type is registered and the registry points to a specification and the specification explains how to process the data according to its intent. The specification does not need to be a standard (a.k.a., a measure that everyone agrees to). It would help, but most useful standards are defined through use. Whoever starts sending the data first should define the specification according to what is being sent, not try to get everyone to agree first.

Roy later goes on to explain:

This is one of those gray areas of increasing RESTfulness that will doubtless drive some people nuts. The problem is that I can’t say “REST requires media types to be registered” because both Internet media types and the registry controlled by IANA are a specific architecture’s instance of the style — they could just as well be replaced by some other mechanism for metadata description.

The broader question is what does it take to create an *evolving* set of standard data types? Obviously, I can’t say that all data types have to be *the* standard before they are used in a REST-based architecture. At the same time, I do require enough standardization to allow the data format sent to be understood as such by the recipient. Hence, both sender and recipient agree to a common registration authority (the standard) for associating media types with data format descriptions. The degree to which the format chosen is a commonly accepted standard is less important than making sure that the sender and recipient agree to the same thing, and that’s all I meant by an evolving set of standard data types.

And so standardization isn’t necessarily the deciding factor in self-descriptiveness. At the end of the day, the sender and receiver need to have agreement on the format being used for a representation (through the media type indicated in the transfer protocol) and have a well understood way to map that format name to a specification. This doesn’t seem to say much about the nature of the formats themselves though.

In the same posting, Roy identifies section 5.2.1 of his dissertation as the definition of the self-descriptiveness constraint (at least as it pertains to the data format). I find this section particularly useful because it compares the RESTful architecture of the Web to three alternative architectures for distributed hypermedia systems. Contrasting the Web with other possible solutions makes the distinguishing features of REST much more clear and highlights the strengths and weaknesses of the style.

The three alternatives proposed can be summarized as follows:

1)      A client-server architecture where requested information is rendered on the server and the resulting image is sent to the client. A key benefit of this architecture is the encapsulation of the server-side data. This decouples the client from the server, isolating it from implementation details and allowing the service to evolve independently from the client – new features could easily be added to a web site in this architecture, the image sent to the client would just change appropriately. This comes at the cost of scalability issues as the server must take on the processing associated with rendering images, as well as the bandwidth cost of transmitting each page as an image. Additionally, the client is restricted to being a browser – or at least something that can deal with page images. For example, it would be practically impossible to implement a spider in this architecture.

2)      A mobile-object style architecture where the server sends a combination of both the data and a rendering engine (i.e. processing logic or code) to the client. This provides data encapsulation within the downloaded rendering engine running on the client. The server also offloads some processing to the client at the cost of the increased bandwidth for downloading application code. Here the client is even further restricted in what it can do – little more than simply running the downloaded application code.

3)      An architecture where the server sends the raw data to the client along with a media type that indicates the data format allowing the client to choose a rendering engine for the data. This offloads the processing associated with rendering from the server and limits the bandwidth requirements as only the raw data is downloaded. The client is fairly unrestricted – it could be a browser, or a spider, or anything that is able to process the raw data. However, this comes at the cost of data encapsulation. The client is exposed to the server’s raw data, increasing coupling and limiting the server’s ability to evolve or change its implementation.

Option 3 addresses the shortcomings of Options 1 and 2 at the cost of data encapsulation. Also, while Option 3 allows for a wider range of clients, the processing that can be performed by any single client implementation in Option 3 is fixed. Option 2, through downloaded code, affords a single client more flexibility.

One might think that Option 3 is REST, but it isn’t – there are subtle differences. REST is described as follows:

REST provides a hybrid of all three options by focusing on a shared understanding of data types with metadata, but limiting the scope of what is revealed to a standardized interface. REST components communicate by transferring a representation of a resource in a format matching one of an evolving set of standard data types, selected dynamically based on the capabilities or desires of the recipient and the nature of the resource. Whether the representation is in the same format as the raw source, or is derived from the source, remains hidden behind the interface. The benefits of the mobile object style are approximated by sending a representation that consists of instructions in the standard data format of an encapsulated rendering engine (e.g., Java).

The first key distinction is that the raw data is not sent to the client in REST; instead, the data is converted into representation format that is:

a)      “Matching one of an evolving set of standard types” – this is elaborated in detail above.

b)      “Selected dynamically based on the capabilities or desires of the recipient and the nature of the resource” – this is essentially content negotiation. The format used to represent the data when transmitting it to the client isn’t fixed; an appropriate format can be selected for each request. Obviously different sets of formats are applicable to the various classes of resource (e.g. there are distinct formats that can be used for images and hypermedia). The client is able to constrain the set of formats it is willing to accept in the request (the HTTP accept header is designed for this, though separate URIs are often used in practice).

For example, rather than sending raw database query results to the client, you convert them into the standardized format most appropriate for the client before returning it. For a web browser, this is likely HTML, for a voice browser – VoiceXML. Of course, as Fielding points out, some resources are simply static hypermedia pages. Here, the raw data format is the same as the representation format, but should they be different, the client only ever sees (i.e. depends on) the representation format.

The second key distinction is that some of the standard data formats supported by a client can be bytecode or scripts – code on demand. This provides similar benefits as Option 2 as a single client is provided with increased flexibility. Of course, the more downloadable code is used, the more you are impacted by the drawbacks of Option 2. It is up to the architect to properly balance the tradeoffs for a specific system’s needs.

There is another set of tradeoffs to consider in the nature of the hypermedia format used – this gets us back to the subject of my previous post. As we have just seen, a RESTful service does not send its raw data to the client (that’s Option 3, not REST); it must be converted to a self-descriptive format. Given the data encapsulation benefits that this is intended to provide, it should be clear that simply serializing your raw data into XML or JSON doesn’t cut it. Angle brackets and curly braces don’t hide implementation details.

I would say that the minimum amount of abstraction is provided by a standardized format specific to an application, for example, a standard format for online book sellers. (I’m using Roy Fielding’s definition of application which is basically “something the user wants to do with computers” such as buying a book. I’ve used “service” to denote a specific instance of an application.) If an interface can be exported by multiple distinct implementations it’s a sure sign that it’s not coupling the client to a specific one. Of course, such a format would limit the ability for a service to evolve as I described previously. For example, if the service changed to also support trading books or selling other types of goods, the application-specific format would likely not be sufficient for these new features.

Data encapsulation and service evolvability can be maximized by instead designing (or selecting) a format around the client’s needs. This is because such a format more closely matches the full space of applications that the client is able to support. If you want a single service to be accessible by two distinct clients that don’t use the same format(s) then simply support them both, using content negotiation or distinct URIs to allow a client to choose the right one.

This data encapsulation may come at the cost of some efficiency as compared to a format designed around the application. Also, it requires that the service designer, at implementation time, has some understanding of the clients that will use the service. It is up to the system designers to consider the tradeoffs and choose a format that meets their needs. But REST does not constrain you to using a broader, standard format – an application-specific format is RESTful as long as it meets the self-descriptiveness criteria described above. Roy Fielding touches on this in one of the postings already linked above, saying:

Sure, it is easier to deploy the use of a commonly understood data format. However, it is also more efficient to use a format that is more specifically intended for a given application. Where those two trade-offs intersect is often dependent on the application. REST does not demand that everyone agree on a single format for the exchange of data — only that the participants in the communication agree. Beyond that, designers need to apply their own common sense and choose/create the best formats for the job.

If we want to call one more RESTful than the other, then we have to take the goal of evolution into account. I would say it is more RESTful to use a specific standard type when applicable or to define a new type that is specific to a given purpose AND intended to be standardized for that application type (i.e., proprietary types are less RESTful than industry-wide standard types, but new standard types are not less RESTful than old standard types). But that is really only my personal preference, since the style does not constrain REST-based architectures to a single standard.

I’m talking about over-constrained, service-specific hypermedia formats that precisely represent a service’s resources and workflow. It’s hard to think of this as a “problem” — it’s what we’re used to doing in software interfaces. But this is certainly not how things work on the Web. Here we have a single format, HTML, used by a wide variety of services: Google, Facebook, Amazon, etc., that all do very different things. The markup language is not designed around the semantics of any of the resources exposed by these services. There is no <book> element used to represent a book on Amazon.com. The Web’s interface is uniform not only because of HTTP but also because of HTML.

Think of it this way: the fact that HTML is able to describe a wide variety of services not only allows a single client to work with the limitless set of sites on the Web, but also allows a single service to evolve. A site can change itself from one of the services describable by HTML to another without requiring the browser to be updated. However, if a hypermedia format is only able to express a very constrained range of services, then service evolution becomes impossible. If you provide a service for “sprocket management” and your hypermedia format is defined in terms of sprockets, collections of sprockets and link relations describing all the supported actions one might take on sprockets, then it’s hard to see how it could be used for anything else in the future.

You may be wondering how exactly you go about broadening the range of your service’s hypermedia format. Sure you can generalize the format (maybe a sprocket is just a type of widget and instead of sprocket collections, you can just have a generic collection element) but that will only get you so far. And more importantly, most folks don’t have the precognitive abilities required to know where their service is going to evolve to in the future. Format extensibility isn’t the answer either: clients would still need to be adapted to any extensions required to support changes to a service.

We can look to the Web for a strategy to deal with this.

The formats used on the Web aren’t designed around the services. They are designed around the client and what it does with the information. A browser is a program that presents information to human beings, primarily through a textual/graphical user interface, and the browser languages revolve around the needs of this task. HTML tags are used to provide structure to text and inline media as well as provide controls for interacting with the information via links and forms. CSS allows the presentation to be customized, and Javascript event handlers express handling for focus, mouse click, and other events if the default handlers aren’t appropriate for the application. These languages drive what the browser does with the information — they have nothing to do with the specific services being executed.

HTML is the language of the browser; it is the lens through which browsers see all resources. Services adapt their information and workflow descriptions (resources) into documents in this language (representations) when working with browsers. The HTTP Accept header tells them that they are working with a client that speaks the browser’s language — a browser or some other client, like an indexing spider that would like to see the resources from the browser’s perspective.

When building RESTful systems for other domains, why do we make the hypermedia format specific to the service? Instead, we should be defining hypermedia formats around the clients — if the goal is to allow the client to interact with the widest range of services possible, then it makes sense to use a format that is designed around the client semantics and bounded only by the capabilities of the client.

At first glance this seems to be binding the service to a specific client — but not completely due to the declarative nature of markup and the Principle of Least Power. Other, secondary types of clients like indexing spiders are able to analyse the markup and understand what it does to provide other capabilities such as search. So really the service is bound to the ecosystem of clients that understand that markup language. But still, this model does limit the clients that can be supported with a single markup language.

This is where the separation of resources and representations, and Content Negotiation (conneg) come into play. A single resource can have multiple representations. They can have a separate URIs or share a single one (using the conneg features of HTTP), but either way a client is able to get the resources represented in a format it supports. By supporting multiple hypermedia formats, your service can cater to multiple classes of client simultaneously. Support for new formats/clients can easily be added without disrupting existing ones.

For example, say that you are building services within a bank. Rather than designing a single hypermedia format around the resources and services such as accounts, balances and financial transactions, design (or re-use) separate formats for each type of client. The teller terminals would likely use HTML, the ATM machine might also use HTML or something similar to deal with the specifics of the interface, a hypermedia format might be designed for automated cheque processing machines. Inter-bank transactions are a tricky point — this might require a format designed around accounts, transactions etc. but this should be at least customized to the specific inter-bank use cases. However, an alternative approach might be to support multiple hypermedia formats at the inter-bank boundary for the specific client types. (Ok, so I’m not in banking myself, so maybe I’m a little off base here on the specifics but I think the principle is sound. If you want examples from my domain, I can simply refer you to VoiceXML and CCXML.) By customizing the hypermedia to the capabilities and needs of the individual classes of client, you provide the services with the ability to change and grow.

This model is based on the assumptions that the services outnumber and will evolve more quickly than the clients and that updating clients is more difficult/expensive than updating services. HTML and browsers do evolve — but there are far fewer browser implementations and versions of HTML than there are services (never mind versions of those services). Also, it is much more difficult to get millions of users to update their browsers than to update your web site. If these assumptions don’t hold for your domain, maybe designing your hypermedia format around your service does make sense — just don’t fool yourself into thinking that you’ve decoupled your clients from your services.

That’s enough for now. I’ve likely raised more questions than I’ve answered here but this is definitely not my last word on this. I plan to provide more details in upcoming posts. Please fire away with the questions and feedback though so I can focus on the right areas.

I’m certain that most of what I have to say here will be related to software architecture. It’s what I spend my days working on, and while you’d think that it would be the last thing I’d want to spend my nights writing about, it’s a topic on which I have a lot of opinions and insights to share. While I’ve had a variety of roles in my career, I’ve been largely focused on loosely-coupled, hyperlink-driven systems based on standard protocols and formats — in a nutshell: Web-inspired software architecture. I imagine that this will be the prominent theme here (though I don’t have long term plans for what I am going to cover beyond the first few posts). The title of this blog, “linked, not bound”, is meant to reflect the nature of the subject matter, though it incidentally describes the format as well.

Blogs are a very personal form of communication. Because the writing reflects the views of the author it often tells you as much about them as it does about the subject matter. I think that this, my first post, should elaborate on not only the subject of this blog but also my relationship to the topic. When I used the term Web-inspired software architecture above, I was not refering to the architecture of the Web itself — the world of HTML and desktop browsers, but to the Web’s architectural style. Unless you’ve been hiding under a digital rock, you’ll know that I’m referring to Representational State Transfer or REST.

And so at this point, I need to do a little work to keep your attention. Most folks these days are desperately trying to reduce their data intake by unfollowing or unsubscribing from low-value information sources. We’ve all got busy lives and if a blog doesn’t offer something new and unique then it’s not worth the reader’s (nor the writer’s) time. REST certainly isn’t new; in fact it’s probably in the waning stages of the hype cycle by now. But that’s the problem: the subject has not just been beaten to death, it’s been beaten beyond recognition. A lot of the content covering REST really misses the mark or is just plain wrong — a problem that snowballs as folks learn from this misinformation and then spread it in their own blogs, or worse, crank out half-baked APIs and mislabel them as RESTful.