HAYCORN — 15 July 2010

HTTP is one of my favourite things and I want to see it everywhere

First things first: I like HTTP. I really like HTTP. The other day I got to think­ing about the ar­chi­tec­tural ben­e­fits of REST, and how many of the ad­van­tages seem to apply just as much to com­mu­ni­ca­tion between the com­po­nents of an ap­pli­ca­tion as between clients and server that com­mu­ni­cate over the network. What are the im­pli­ca­tions of using HTTP (to pick the most common REST transport) for in­ter­nal com­mu­ni­ca­tion between components, as al­ter­na­tive to straight­for­ward func­tion calls?

There’s some obvious disadvantages, so let’s get those out of the way:

  • It’s more verbose and it’s more complicated. Instead of being able to di­rectly call a method, you have to re­trieve an HTTP client from a factory, con­fig­ure it, build up the request, and parse the response. And the callee is like­wise slightly more com­pli­cated than its non-HTTP equivalent.
  • It’s slower. Parsing the headers is slow (strings!!), and even if you cheat a little and, say, pass the headers and pa­ra­me­ters around as as­so­cia­tive arrays it will still be slower than passing ar­gu­ments directly.

Against this, the advantages:

  • “Internal” calls (i.e. without network) look exactly the same as “external” calls (i.e. with network). If a server becomes overloaded, it’s easy to move an “internal” service to an “external” service to spread the load. If HTTP is already used in the ap­pli­ca­tion in some way (and it prob­a­bly doesn’t make sense to add HTTP to a system that doesn’t already have tra­di­tional client-server HTTP somewhere), the dif­fer­ence between “internal” and “external” calls may already seem un­nec­es­sar­ily arbitrary.
  • The HTTP pro­to­col has a number of useful, sophisticated, well-defined features, and there’s also a large number of people who know the pro­to­col rea­son­ably well. (For one tech­nol­ogy to be both complex and widely known is quite unusual.) HTTP pro­vides mech­a­nisms for authorisation, content ne­go­ti­a­tion and in­flu­enc­ing caching behaviour, to give a few ex­am­ples of fea­tures that are typ­i­cally dif­fi­cult to provide, es­pe­cially in a backward-compatible way. The degree of control over caching be­hav­iour it offers seems par­tic­u­larly useful. (Since cache in­val­i­da­tion is one of the two “hard problems” of com­puter science (together with naming things), al­low­ing the caller to, for example, in­val­i­date the callee’s cache, would appear to be a helpful feature to have.)
  • HTTP is largely self-documenting. If you have a URL rep­re­sent­ing a resource, you prob­a­bly don’t need to read the doc­u­men­ta­tion to see how to perform a par­tic­u­lar operation—you get the stan­dard HTTP verbs, and the stan­dard HTTP semantics. If an op­er­a­tion isn’t sup­ported at all, or isn’t sup­ported for a par­tic­u­lar user, or isn’t sup­ported right now, there’s prob­a­bly a status code avail­able to in­di­cate the problem.
  • HTTP is stateless. This helps helps keep the in­ter­ac­tions between com­po­nents simple. It’s simply not pos­si­ble to con­struct an object and pass it back and forth between client and server, trans­form­ing it all the while, in ways that turn out not to be quite the trans­for­ma­tion you were expecting.

Using REST for in­ter­nal com­mu­ni­ca­tion might ini­tially seem like a strange idea, but for the reasons de­scribed above, I think it’s one that makes sense. None of REST’s constraints and guiding principles are vi­o­lated if there’s no network involved, which I think is es­pe­cially in­ter­est­ing con­sid­er­ing that REST was ex­plic­itly for­mu­lated by Roy Field­ing as a so­lu­tion to the prob­lems faced by ex­ist­ing network-based soft­ware architectures.

There’s a few frame­works and li­braries out there that go so way toward treat­ing HTTP—both in­ter­nal and external—as a fundamental, first class protocol. Kohana 3.0 has some­thing called HMVC, and I’ve also done a proof of concept version of this feature for Zend Framework. Both allow you to create a stan­dard HTTP client that is able to route and dis­patch HTTP re­quests internally. Are there any other frame­works (in any language) with strong support for HTTP? I asked a related question on Stack Over­flow re­cently but didn’t get many responses.

Finally, even if the dis­ad­van­tages of per­va­sive HTTP out­weigh the ad­van­tages for many applications, I think it’s in­ter­est­ing to imagine what it would be like to design an ap­pli­ca­tion with per­va­sive HTTP. For one thing, it helps clarify the dif­fer­ence between REST and RPC systems like SOAP—whilst there’s never any reason to use SOAP for in­ter­nal calls, there may be reasons to use HTTP.