So I’m going to write about them.
In this context I’m really talking about REST APIs: those wonderful HTTP requests between you and some application that allow you to do all sorts of great things with your data. Most projects that I have the most fun with involve working with an API; reading about it, testing against it, building the solution with it and coming up with other crazy sh*t to do with it.
Quickly, about REST
REST APIs provide a simple interface to an application over – normally – HTTP. It follows the familiar standards of other architectures: create: POST, read: GET, update: PUT/PATCH and delete: DELETE. You use these methods to interact with ‘resources’ at different ‘endpoints’ of the service. These endpoints will return and/or accept data to achieve a desired result.
That’s the high level overview.
From there you will start encountering a wide range of variations and differences. Some APIs will allow XML with your requests, but not JSON. Some work with JSON, but not XML. APIs may require you to explicitly declare the media type you’re going to interact with. You might have one API that accepts HTTP basic authentication while others have a token based authentication workflow (like OAuth/OAuth2). There is a lot of variance in the designs from service to service.
Which brings us to the opinions on design
The APIs I do the most work with currently are for the JSS (of course), JIRA and HipChat, but I’ve also poked around in CrashPlan and Box on the side. There are a lot of things that I like about all of these APIs and, frankly, some things that really irk me. And, I mean, really irk me. Those experiences started me in the direction of learning what it was like to create my own.
If you know me at all you know that I have a real passion about Python. My current obsession has been the Flask, a microframework for Python that allows you to write web applications. I’ve been using it for HipChat add-ons that I’m developing, but I was really excited to get into Flask because I could start building my own REST APIs and dig into how they are designed.
Between working with established APIs and the reading and experimenting as I work on my own, I’ve determined there are a number of design choices I would want implemented in any API I worked with.
But it’s in the interface…
Two years ago I had the opportunity to attend Dreamforce. That year was a biggie as Salesforce was transitioning their development platform and announced their intention to be mobile first and API first. It was a pretty “phenomenal” keynote. There were tons of sessions during the mega-conference devoted to the plethora of new and revamped APIs that now made up the Salesforce platform. My big take away was a slide that provided an extremely high overview of the new stack. All of Salesforce’s apps and services sat above a unified API layer.
I can’t say why that stuck with me so much at the time since I didn’t even know how to write a simple Python script, but it did. This was the first big idea that I held onto about API design: implement your features at the API level first, document the implementation and then use that to build onto the end-user solution.
There are plenty of examples out there of services that segregate their user interface from their API and I’ve seen forums with a lot of developers or IT professionals asking why something was implemented in the GUI but inaccessible through their API which prevented an app/integration/automation from advancing. So, as Salesforce put it, API first.
Documented without the docs
I’ve seen a lot of great examples of API documentation out there. CrashPlan, JIRA and HIpChat are at the top of my “how to do it right” examples in that they provides representations of data for each supported request method for an endpoint, returned HTTP status codes and potential error messages with their causes. This is invaluable information for anyone who is writing a script or application against an API, but they all share the same weakness: they’re docs that exist outside the API.
A robust API can provide all the information a developer requires through through the same HTTP methods that – allowing for automated discovery of the API’s capabilities without scrolling around web pages and then flipping back to your console.
There’s an HTTP method I’ve read about, but not one I’ve seen in any of the docs for these APIs as supported. That would be the OPTIONS method. It’s a great idea! Want to know what you can do to a resource? Pass OPTIONS as the method and in the response there will be a header “Allow” that will list them.
This could be extended to be a contextual method based upon the level of access the provided credentials have. Say a resource supports GET, POST, PUT, PATCH and DELETE but our user account only supports creating and updating resources. An admin would return all five in the response header, but our user would only have GET, PUT and PATCH as valid options.
So ok, there’s an HTTP method in the REST standard that allows us to discovery how we can interact with our resources. Now how do we determine what the valid format of our data in our requests is supposed to be? JIRA actually implements a solution this for ‘Issues.’ Check out the following endpoints:
Text The ‘createmeta’ endpoint will return a wealth of data including available projects, issues types, fields and what is required when creating a new issue. That’s a goldmine of data that’s specific to my JIRA instance! Then it gets even better when parameters are passed to filter it down even further to better identify what you need to do. Like this:
That will return all of the required fields I require to create a new ‘Task’ within the ‘Information Technology’ project in my JIRA board. If I create a task and then want to update it I can call the second endpoint to reveal all of the fields relevant to this issue, which are required and acceptable values for input.
Despite how great the above is, that’s about all we get for the discovery through JIRA’s API. We still need to go back to the online docs to reference the other endpoints.
Something I read on RESTful API Design struck a note on this topic. The idea pitched here is to use forms to provide back to the client a representation of a valid request for the endpoint by passing the appropriate MIME type (for example: ‘application/x-form+json’). This isn’t something you could expect to have a uniform definition of, but that wouldn’t matter! You could still programmatically obtain information about any API endpoint by passing the the MIME type for the desired format.
Here’s an example of what a response might look like to such a request:
curl http://my.api.com/users -H "content-type: application/x-form+json" -X POST
They can do a lot more work for you
Usually if you’re making a request to an object there will be references, links, within the data to other objects that you can make calls to. Sometimes this is as simple as an ID or other unique value that can be used to build another request to retrieve that resource. Seems like an unnecessary amount of code to handle this on the part of the client.
There are two ways of improving this. The first is to include the full URL to the linked resource as a part of the parent.
curl http://my.api.com/users -H "content-type: application/json"
The second can build upon this by allowing parameters to be passed that tell the API to return linked objects that are expanded to include all of the data in one request. JIRA’s API does this for nearly every endpoint.
curl http://my.api.com/users?expand=computers -H "content-type: application/json"
'model': '13-inch Retina MacBook Pro',
Versions are a good thing
All APIs change over time. Under the hood bug fixes that don’t affect how the client interacts with the service aren’t much to advertise, but additions or changes to endpoints need to be handled in a way that can (potentially) preserve compatibility.
The most common kind of versioning I interact with has it directly in the URL. I’m going to reference HipChat on this one:
The v1 API was deprecated some time ago as HipChat migrated to their newer and more robust v2 API. While the v1 API is still accessible it has limitations compared to v2, is lacking many of the endpoints and is no longer supported which means that a lot of integrations that were written using v1 are steadily being phased out.
The differences between the two versions of the API are huge, especially when it comes to authentication, but even after its release the v2 API has had a number of changes and additions made to it. Unless you’re watching for them they would be easy to miss.
Going the route of maintaining the version of the API in the URL, I found this example:
my.api.com/ < Points to the latest version of the API
my.api.com/2/ < Points to latest version of the v2 API
my.api.com/2.0/ < Points to a specific version of the v2 API
On the backend the objects would need to track which version a field or endpoint was added (or even removed) and handle the response to a request based upon the version passed in the URL. Anything requested that falls outside of the version would prompt the appropriate 4XX response.
Another method of versioning is used with GitHub’s API. By default your API requests are made against the latest version of the API, but you you can specify a previous version by having it passed as a part of the ‘Accept’ header:
curl https://api.github.com/users/brysontyrrell -H "Accept: application/vnd.github.v3.full+json"
I’ve read about pros and cons for both approaches, but they serve the purpose of identifying changes in an API as it evolves while providing a means for compatibility with existing clients.
Multiple formats isn’t a sin
My personal preference for any REST API I work with is JSON. JSON is easy to me, it makes sense, it just works. I can think of one glaring example off the top of my head of an API I frequently work with that lets me read back objects in JSON but only accepts XML for POST/PUT requests. Frustrating.
Still, JSON is my preference. Plenty of people prefer XML. In some cases XML may be easier to work with than JSON (such as parsing in shell scripts) or be the better data set for an application. Structurally XML and JSON can be very interchangeable depending upon the data that is being accessed.
If the object can be converted to multiple formats then it may be a good idea to support it. By passing the appropriate MIME type the API can return data in the requested format. If no MIME type is passed there should be a default type that is always returned or accepted.
It’s late now and I’ve dumped a lot of words onto the page. There’s a PyCharm window open with the shell of my sample API project that attempts to implement all of the design ideas I describe above. Once I finish it I’ll throw it up on GitHub and see about incorporating some of the requests/responses to it into the article.