Continuous Delivery of a startup

Hi! My name is Marcin Grzejszczak and I’m the lead of Spring Cloud Sleuth, Contract and Pipelines projects at Pivotal. If I have spare time I write on my blog toomuchcoding.com.

Today, I have this great privilege of having a conversation with Jakub Kubryński. We will be talking about software architecture, Continuous Delivery, and application testing.

– Can you tell us a few words about yourself and where you work?

I’ve been involved with professional software development since 2004. During this time I’ve been everything from a regular programmer to the head of software development. I’ve also done a lot of consulting, training and workshops for some of the biggest international companies. For the last 5 years, I’ve been focused on growing my own company, Devskiller.com, where I’m currently the CEO. We provide an online platform for verifying the technical skills of developers and system engineers. We do this by checking their expertise and ability to apply their knowledge to solving business and technological problems. If, for instance, you need someone who is fluent in Spring Boot and MongoDB, no problem we have great tasks to check for these skills 🙂

– Can you describe Devskiller’s architecture? What does the communication between services look like?

I don’t like buzzwords, so I’ll avoid using word “microservices”. Our system is built from autonomous, independently deployed components. Everything runs on Linux VMs hosted in the Azure Cloud. Depending on the specific requirements of a given task, we use synchronous communication based on REST services or asynchronous AMQP messages.

We use a continuous delivery approach, where most of the applications are deployed to production every day.

– What is the technological stack you use and why did you choose it?

The vast majority of the system is written in Java, a very popular language which performs really well and maintains a strong development velocity. For the technical parts of the platform, we also use Go which, while not as good as Java for implementing our “business model”, has a much smaller footprint despite performing really well.

– Can you elaborate on how the people doing frontend and backend cooperate?

For the last 4 years, we’ve used a consumer-driven contracts approach. This means we start designing our API by gathering requirements from our frontend. At the very beginning, we used sinonjs.org, but we have eventually switched to generating all our contracts using Spring Cloud Contract. By the contract I mean the request and response specification, eg. HTTP POST request sent to /users URL with a body containing a username and email will produce the response with status code 201 and Location header pointing to the created user. After establishing the contracts, we were able to start to develop the client side. During the process, we found a lot of improvements which could be applied to our API endpoints to make our front-end development easier and more effective. Thanks to the CDC, all we had to change were the DSLs with the contracts. When the API was almost ready, we started developing the backend. Spring Cloud Contract generates the tests and we simply continue to develop controllers until all the tests pass. When more changes are required, the frontend team usually creates a pull-request to the server repository (where we keep its contracts) with proposed changes, which are then implemented after the approval of our backend engineers (of course sometimes after exchanging some ideas and comments).

– With so many interactions, how do you ensure that API compatibility is maintained?

Continuous Delivery helps here. We do not version the internal API at all. We have only one version – current. So during every build, we check to see if the application is consistent with its current contract and the contract that is currently running on the production. This means that we cannot do any breaking changes. At the first glance, it looks like a giant limitation, however, it’s not. Any breaking change can be implemented as a set (in the context of the API, a set always means two) of non-breaking changes. For example, if you want to change the type of the field from Date to DateTime you use the following pattern:

1. Expose the `DateTime` field in the API. On production you now return both fields, however, the contract guarantees only the new one.
2. Wait until all client applications migrate to the new field. It’s pretty fast, as all tests will show only the `DateTime` field (contract protection) and without switching to the new field, you won’t be able to compile the application.
3. Remove the `Date` field.

It’s the same pattern that you use for zero downtime deployment, just applied to the API changes.

– Why did you decide to use Spring Cloud Contract as the framework to facilitate Contract Testing?

For the last 6 years, I’ve been focusing on improving the software delivery processes. Usually, this means that we want to get a feedback faster. Fast feedback means cheaper development and fewer errors. One of the biggest issues when working with distributed systems is how to guarantee the accuracy of communication. Usually, it’s done in a staging environment which makes this environment pretty unstable. I remember that we’d done some technical consultancy for a company and after many brainstorming sessions we’d reinvented consumer-driven contracts (described 10 years before by Martin Fowler). The tool supporting this approach, Pact, was so heavy that it was easier to develop custom solution than learn Pact. Accurest (currently Spring Cloud Contract) was deeply embedded in JVM technology, using the most popular build and testing tools and frameworks, which was much more accessible for Java developers.

– Do you publish your API for users? If that’s the case, how do you do that?

We have a public API which our users can use to automate the integration with our platform. To expose the public API, we use an edge service which translates the steady public API to a frequently changing internal API. Since the public API must be extremely stable and well documented, we use Spring Rest Docs to generate the Slate documentation. Starting with the second version, we plan to also publish the API stubs to allow customers to easily (even constantly) verify the integration.

– Are there any other tools from Spring’s toolbox that you are using and you would like to tell us about?

Of course – we use a lot of Spring projects, all Java applications are based on Spring Boot with common libs like Actuator, Security, Data, etc. We also use Session, Hateoas and Social libraries and a lot of Cloud projects, including Stream, Contract, Sleuth, Hystrix and Eureka. We’re also very close to a production release of an application based on a Cloud Gateway project.

– OK, so you’ve described how you build and test your applications. Can you describe your deployment process?

As mentioned before, we use continuous delivery with almost daily deployments. All components are deployed independently. Having made many improvements, our delivery pipelines are pretty simple: we run build with unit tests, fast integration tests, and the API compatibility check. When it finishes successfully, we deploy the version to the staging environment, where we run critical path acceptance tests. When those tests are green, the version is ready to be deployed to production. Deploying doesn’t mean releasing. We often use feature switches which let us decide when we want to expose a feature to our users, which also makes a fast and trouble-free rollback possible.

– Is there anything else you would like to add?

I’ve just wanted to mention two brand new open-source projects we’ve started. The first is a next step in the automation and encapsulation of the delivery process. Spring Cloud Contract makes it much easier to maintain communication between services but we’re missing a good tool to automate schema migration. We’re using Flyway but all migrations must be written manually. JPA2DDL based on an entity model in the application can dump the whole DB schema as well as generate automated Flyway migrations. https://github.com/Devskiller/jpa2ddl

The second project allows (thanks to easy Spring Web integration) us to simplify UUID identifiers (eg. c3587ec5-0976-497f-8374-61e0c2ea3da5) to shorter, one-click selectable Url62 form (eg. 7NLCAyd6sKR7kDHxgAWFPG), which is really important if we want to create a pretty and user-friendly REST API. https://github.com/Devskiller/friendly-id

Thanks so much for this conversation!