Developing an API product, from REST to running containers - part one: overall picture

How to develop, deploy and operate an API as a product?  In this first post in a series of blog posts we will discuss the goals we want to achieve with the proposed development and production setup.

In follow-up blog posts, we will discuss how we actually implemented the architecture, a development pipeline and production setup, and we will discuss some of the tradeoffs and assumptions we made.


Let’s start with the product we would like to build: a startups API. Three years ago, our colleague Omar Mohout started to collect data of the Belgian startup scene. At that time, nobody really had a good view on the startup scene in Belgium, let alone how it was evolving. Omar started gathering that data, and has since published many reports and insights about it. (You can find some of his analyses herehere and here.) As a result, Omar often receives requests to have access to that data, so they can do their own analyses. We also got the question to provide the community with a map and visual tools, all based on that data set. We thought it would be a good idea to expose that data through an official REST API, instead of sending excel files back and forth.

You can see the first app made on top of the API over at

Some requirements

Since we are bringing the startups API to the market as a real product, not just as some internal thingy, the idea was to do it “by the book”. This means the API would need:

  • An API portal where customers of our API can sign up for access to the API, manage their API keys, and where developers find information on how to use the API.
  • An API management component that allows the API owner to define plans, rate limits, etc.
  • A production setup that enforces the use of API keys, ensures that rate limits are respected, i.e. an API gateway.
  • Some simple dashboards and analytics on API usage.
  • A production setup that is robust, yet easy to maintain - did I mention that development and operations of the startup API would be done by one person and that the API isn’t supposed to be a full time job?
  • Implement the necessary security practices to ensure that access to our startup data is protected at all times.

To add to these requirements above, we wanted to take advantage of the state of the art in (cloud) software engineering and DevOps and make sure that we aim for:

  • developer parity: make the difference in setup between production and development as small as possible, which makes it easy to test, change and debug issues, should they arise.
  • git push based continuous delivery: a development workflow that allows for fast, agile iterations, and that leverages test driven development and CI/CD. Ideally, a simple git push should trigger building, testing and deploying next iteration of the application.
  • the whole setup should be fun and easy to work with. 


Architectural view of your API

Technological choices

With these requirements in mind, and with our own capabilities and preferences in mind, we made the following technological choices:

  • Implement the REST service in hapi (, an open source webserver built in Node.js by Wallmart. Hapi comes with out of the box support for APIs and has excellent testing support through the lab library. We're used to developing in Node.js, so no learning curve here, hapi is well documented, which is a big plus.
  • Mongodb as the data store: REST services produce mostly JSON these days, I therefore went with mongodb, since mongodb stores JSON (well technically BSON) directly and uses JSON for querying. Another plus is that we have some in-house experience with small scale mongo stuff.
  • as an API management platform. 3scale offers developer portals right out of the box, provides an API gateway service (both as a service as well as self hosted) and has a free plan to start with. Alternatives could be mashape or apigee.
  • Swagger for documenting the API. hapi has a swagger plugin that generates swagger documentation based upon the source code of the API itself, no extra work for me there (at least, that's the plan).
  • For deployment, Docker containers will be used, scheduled, managed and orchestrated via Rancher. Using Docker containers will make the app portable, so running the app locally or in production would more or less be the same.
  • We will use Amazon Web Services as the infrastructure provider, but we decided not to go overboard with it and mainly use EC2 (virtual machines) and ELB.
  • Let's Encrypt for everything ssl related. It's a free, open and automatic Certificate Authority that gets a lot of traction lately.
  • Use an internal, Sirris hosted GitLab repository for our source code, linked to a Jenkins as a continuous integration and delivery server that builds, tests and deploys our code.


In this blog post we sketched the overall architecture we have in mind for the startups API. In the next instalment in this series, we will discuss the startups REST API itself, highlight some of our lessons learned when implementing the API on top of hapi and we will take our first steps in containerising the application. 

This blog post is largely based on a post that recently appeared on the nebucom website.