HAYCORN — 15 July 2010

HTTP is one of my favourite things and I want to see it everywhere

First things first: I like HTTP. I really like HTTP. The other day I got to thinking about the architectural benefits of REST, and how many of the advantages seem to apply just as much to communication between the components of an application as between clients and server that communicate over the network. What are the implications of using HTTP (to pick the most common REST transport) for internal communication between components, as alternative to straightforward function calls?

There’s some obvious disadvantages, so let’s get those out of the way:

  • It’s more verbose and it’s more complicated. Instead of being able to directly call a method, you have to retrieve an HTTP client from a factory, configure it, build up the request, and parse the response. And the callee is likewise slightly more complicated than its non-HTTP equivalent.
  • It’s slower. Parsing the headers is slow (strings!!), and even if you cheat a little and, say, pass the headers and parameters around as associative arrays it will still be slower than passing arguments directly.

Against this, the advantages:

  • “Internal” calls (i.e. without network) look exactly the same as “external” calls (i.e. with network). If a server becomes overloaded, it’s easy to move an “internal” service to an “external” service to spread the load. If HTTP is already used in the application in some way (and it probably doesn’t make sense to add HTTP to a system that doesn’t already have traditional client-server HTTP somewhere), the difference between “internal” and “external” calls may already seem unnecessarily arbitrary.
  • The HTTP protocol has a number of useful, sophisticated, well-defined features, and there’s also a large number of people who know the protocol reasonably well. (For one technology to be both complex and widely known is quite unusual.) HTTP provides mechanisms for authorisation, content negotiation and influencing caching behaviour, to give a few examples of features that are typically difficult to provide, especially in a backward-compatible way. The degree of control over caching behaviour it offers seems particularly useful. (Since cache invalidation is one of the two “hard problems” of computer science (together with naming things), allowing the caller to, for example, invalidate the callee’s cache, would appear to be a helpful feature to have.)
  • HTTP is largely self-documenting. If you have a URL representing a resource, you probably don’t need to read the documentation to see how to perform a particular operation—you get the standard HTTP verbs, and the standard HTTP semantics. If an operation isn’t supported at all, or isn’t supported for a particular user, or isn’t supported right now, there’s probably a status code available to indicate the problem.
  • HTTP is stateless. This helps helps keep the interactions between components simple. It’s simply not possible to construct an object and pass it back and forth between client and server, transforming it all the while, in ways that turn out not to be quite the transformation you were expecting.

Using REST for internal communication might initially seem like a strange idea, but for the reasons described above, I think it’s one that makes sense. None of REST’s constraints and guiding principles are violated if there’s no network involved, which I think is especially interesting considering that REST was explicitly formulated by Roy Fielding as a solution to the problems faced by existing network-based software architectures.

There’s a few frameworks and libraries out there that go so way toward treating HTTP—both internal and external—as a fundamental, first class protocol. Kohana 3.0 has something called HMVC, and I’ve also done a proof of concept version of this feature for Zend Framework. Both allow you to create a standard HTTP client that is able to route and dispatch HTTP requests internally. Are there any other frameworks (in any language) with strong support for HTTP? I asked a related question on Stack Overflow recently but didn’t get many responses.

Finally, even if the disadvantages of pervasive HTTP outweigh the advantages for many applications, I think it’s interesting to imagine what it would be like to design an application with pervasive HTTP. For one thing, it helps clarify the difference between REST and RPC systems like SOAP—whilst there’s never any reason to use SOAP for internal calls, there may be reasons to use HTTP.