Many years ago, microservices appeared as the solution to all the problems that the monoliths had, such as scalability, maintainability, and availability. The companies adopt this type of architecture on their platforms which implies massive migrations to old systems using different strategies, like slicing/splitting into small pieces as an alternative to creating a new platform using the old one just as a guide. After some time, it could take months or years; the microservice needs to be deprecated for many reasons, like performance issues, the frameworks having many bugs and not existing a new version, or the company or else the community not supporting the language. This situation offers a unique problem because now you have a microservice that receives requests from many other places. Hence, the deprecation process implies you need to have the plan to remove the microservice to communicate it to all the consumers.
Why can a migration fail?
Let’s start with a real possible scenario; a travel company has a catalog microservice with all the information about countries, states, cities, airports, and many other entities. The microservice needs to be deprecated because it has some big problems related to performance issues, scalability, and maintainability, concentrating on many endpoints as well as the logic of multiple teams in one place.
The migration process to new microservices to replace the old one could be simple, but it’s not because you need to coordinate with multiple teams. Some strategies that could fail are:
- Notify all the consumers: Inform all the teams that a new microservice exists and that the old one will be deprecated, along with waiting for all the teams to respond to notify that the migration was successful. This approach has multiple problems because only some groups have the same availability on their backlogs to migrate. The migration could take weeks or months to end, implying that you need to maintain two different microservices spending time-solving issues.
- Not checking the new microservice: One common problem when you create a new microservice is to check if everything works fine in a real scenario, receiving a request for other microservices. This point is relevant because if the consumers start to migrate and problems appear in some of the environments, there will be a big chance that the consumers will stop the migration together with rollback the changes until you solve the problems. For some reason, it’s not strange that difficulties appear on the new microservice, but you need to reduce the risk that the issues arise during the migration using some mechanism to check if everything works fine or not.
- Continue receiving requests after the migration: You notified all the teams that use the microservices, and all of them migrated, but your microservices have a lot of requests, so you need to decide what you want to do with the old microservice, but you don’t know the real impact to stop it.
These are just a few problems you can find during migration, but many others depend on the company’s context and the consumers.
How to succeed in a migration?
There isn’t a way to migrate from one microservice to another without problems. Still, you could mitigate the risk of significant issues appearing during the process by doing a plan that considers many aspects, like the number of consumers using your microservice and the window of time to migrate.
The migration process to deprecate an endpoint/microservice
The migration process could have 3 phases, each of which needs to be executed in sequence. First, you need to create a plan for the migration; after that, migrate with the coordination of all the consumers, and last, deprecate the microservice. Let’s see each of these phases in a little more detail.
Creation of the plan for the rollout
The first step to deprecate an endpoint/microservice is to create a plan considering different aspects to communicate with all who will suffer the impact of the changes. Some of the elements that you need to consider are:
- Find the consumers: Many tools help you to solve this problem, like the APM (New Relic, ELK, Dynatrace), which shows you a service map with the relation between the microservices, so you only need to take this information and find which team or person are responsible for maintaining those microservices. Sometimes not all consumers appear on the APMs, which imply a possible risk of the deprecation process partially failing; an excellent alternative to solve this problem is to add and check on the logs of your application some information that helps you to validate the list of consumers, this depends of how you implemented your infrastructure because if you use Kubernetes and inject extra information on the headers of the different requests like the name of service the task to find the consumers, it’s simple.
APM – New Relic
Another thing to consider is if someone outside of the company uses the endpoint/microservice because this will affect the deadline to deprecate or remove it. Also, you need to consider alternatives if some of the external consumers won’t migrate to the new microservice.
- Define the strategy: This point is one of the most relevant parts you need to consider during the migration because you need to reduce the risk of something wrong happening with your new endpoint/microservice. There are at least two different strategies to consider:
- Both microservices exist but do not have any connection between them. The consumers migrate from one place to another, assuming all the risk that something could be wrong with the new microservice.
- The old microservice acts as a proxy between the consumers and the new microservice. This approach offers many advantages. You can use some library or tool that implement feature flags like FF4J, Togglz, or Unleash, which have clients to use with Java/Kotlin so you can redirect a percentage of the requests to the new microservices and check if everything works fine or no. If everything works fine, you can continue to increase the rate until you arrive to redirect all the requests to the new microservice. Still, if something terrible happens, you can send all the requests to the old implementation to fix the problem on the new one.
As a recommendation to choose which implementation of feature flags, it’s best to try to create an ADR (Architecture Design Record) to discuss the pros/cons of each implementation.
- Both microservices exist but do not have any connection between them. The consumers migrate from one place to another, assuming all the risk that something could be wrong with the new microservice.
- Define the early adopters: You need to define which of the consumers could be an excellent option to become the early adopters or the new microservice considering different aspects like the level of communication that you have with that team or if they are receiving the changes like something good and try to help you. You need to think about this particular point, like having allies that allow you to push these changes.
- Deadline: You can’t maintain both microservices forever because it implies that your team needs to fix bugs or issues in both. Also, these microservices could not have the same language or frameworks, so your team takes more time to solve the problems. The deadline needs to consider the number of consumers and the type of changes between both microservices. Suppose the changes only imply changing the URL using the same request/response. In that case, you can declare that the deadline for the old one could be in one month, but if the changes are complex, like adding new parameters on the request or the response changing all the structure could be an excellent option to have a deadline in a few months.
- Communicate your plan: The last part of the plan is to define the strategies to communicate the migration and follow the advances. As a suggestion, try not to use one particular way to inform all the consumers because it’s not the same as notifying all the consumers that are part of the same company that inform external consumers. Some approaches could be:
- Send the information on the headers: You can add the headers “Deprecation” and “Sunset” on all your requests with the information about the deadline to migrate, which is an approach that many companies like Paypal, IBM, and Clearbit use following the RFC8594 which introduces these headers.
Deprecation: Tue, 31 Dec 2024 23:59:59 GMT
Sunset: Wed, 31 Dec 2025 23:59:59 GMT
The main problem with this approach is that your consumers need to have some mechanism to intercept and notify that some endpoint will disappear, which only sometimes happens. By the way, adding headers could be simple if you use frameworks like Spring Boot or Quarkus, adding some interceptors which add the headers in all the responses.
- Send an email with the information: The idea is to send all the information about the deprecation process and the changes the other teams need to make. As a recommendation, the email includes questions about whether it is possible to do it until the deadline and if all the explanation is straightforward. This approach implies send and respond many emails, but it’s one of the best ways to do it.
- Create a group on your internal communication tool: If you use tools like Teams or Slack could be an excellent option to create a channel with the people responsible for the different teams involved in the migration to respond to the question just once.
- Send the information on the headers: You can add the headers “Deprecation” and “Sunset” on all your requests with the information about the deadline to migrate, which is an approach that many companies like Paypal, IBM, and Clearbit use following the RFC8594 which introduces these headers.
- Track the advance: Create a document to represent the advance of the migration and which teams migrated to the new microservice. If there are no significant advances during a period, you can send an invite to have a meeting and discuss what happens.
These aspects are only the basic ones, but other ones depend on the context of the situation.
Start the process of deprecation
After creating the plan with a specific deadline and having the new endpoint/microservice in the production environment, the next step is to start the deprecation process with the microservices that are part of your team because it’s an excellent way to detect problems with genuine requests. You could deploy and do a rollback in case something terrible happens without the need to notify another team. When you finish with the migration of all the microservices of your team and do not detect any problem for a period, you need to notify external consumers about the deprecation process.
This migration phase could take some time, weeks, or months, so the good idea is to monitor your APM or your logging tool periodically to check if the number of operations on the old microservice decreases each week. If nothing happens for a couple of weeks, try communicating with the consumers to understand what happens.
During this migration process, try to avoid including new features on the old microservice because this will imply that you will have them in the new one. An excellent strategy to force your consumers to migrate is to indicate that some new endpoints or features are only available on the new microservice. It’s not a unique way to motivate consumers to migrate, but you need to find some approach to seduce them to do it before the deadline.
At this phase, you must consider options in case some consumers will not migrate.
FINISH the process of deprecation
When the deadline that you defined on the plan arrives, if everything occurs like your idea, you will not have any problems. You can remove the microservices in different environments but only in some cases; this situation happens, so you need to define what you will do with the consumers. Here are some alternatives you could consider possible options:
- Communicate with the teams: You can meet with the teams that do not do the migration and try to obtain a tentative new deadline. If you adopt this approach, consider adding some mechanism to throw an exception when you receive a request for other microservices not declared in some whitelist. The main problem that could appear if you do not have a whitelist is that someone can assume that the microservice is active and try to use them, so you need to reduce the risk that only specific consumers for a while.
If you Spring Boot or Quarkus, a good option could be to create an interceptor that checks if the whitelist contains the consumer that requests to reject or accept it.Lastly, define a new deadline which is explícity declared on the microservice, and after that date, reject all the requests. - Remove the microservice: Another alternative is to remove the microservice in a non-productive environment. Hence, the entire platform continues working, but the consumers can’t continue using your microservice on the rest of the environment. It’s a way to force consumers to do their migration. This is especially useful when you have requests from unknown consumers.
- Redirect all the requests: Another possibility, if the changes of the endpoint or microservice do not have modifications on the request/response, it’s to create specific rules of routing all the requests to the new microservice and stop the old one. This implies that you need time from some DevOps to do it; the best scenario for this approach is that you have an API Gateway like Spring Cloud Gateway writing the rules without assistance.
Before choosing any alternative, you need to consider the tradeoff because it’s not the same that deprecates a microservice that only exposes information about cities and countries instead of another one that processes all the company payments.
WHAT’S NEXT?
There are tons of resources about the process of the deprecation of monoliths, but a few are related to microservices. The following is just a short list of resources that you can apply for both types of architectures:
- Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith by Sam Newman
- Respectful REST APIs – ‘Sunset’ and ‘Deprecation’ HTTP Headers by Horatiu Dan
- Evolution Patterns by Microservice API Patterns – This link will offer other strategies to use in the deprecation process.
Other resources that could be great reads to change the approach of connecting the microservices using a synchronic way like REST/SOAP to an architecture oriented to events.
- Building Event-Driven Microservices: Leveraging Organizational Data at Scale by Adam Bellemare
- Grokking Streaming Systems: Real-time event processing by Josh Fischer and Ning Wang
The following resources are connected with some topics that appear in the article:
- Fundamentals of Software Architecture: An Engineering Approach by Mark Richards and Neal Ford – In this book, you will find a great explanation of the use of ADR
CONCLUSION
Depreciating an endpoint or microservice implies many things, most of which you could consider in the migration plan. Still, communication with other people is key to success. Try always to migrate with your microservices or with teams with a certain level of affinity to do a test; this will help you.
In the future, consider other alternatives that the consumers connect directly with your microservices; there are alternatives like using an API Gateway or events. Create an API Gateway that acts as a proxy between the consumers and your microservices. This approach will help you reduce the interaction complexity and wait for other teams to migrate. Using this approach, you can migrate to the gateway without the need to notify all the consumers.
Author: Andres Sacco
Andres Sacco has been a developer since 2007 in different languages, including Java, PHP, NodeJs, Scala, and Kotlin. His background is mostly in Java and the libraries or frameworks associated with this language. In most of the companies he worked for, he researched new technologies to improve the performance, stability, and quality of the applications of each company.
In 2017 he started to find new ways to optimize the transference of data between applications to reduce the cost of infrastructure. He suggested some actions, some of them applicable in all the manual microservices and others in just a few. All this work concludes with the creation of a series of theoric-practical projects, which are available on the page Manning.com
Recently he published a book on Apress about the last version of Scala. Also, he published a set of theoric-practical projects about uncommon ways of testing, like architecture tests and chaos engineering.
He dictated internal courses to different audiences like developers, business analysts, and commercial people. Also, he participates as a Technical Reviewer on the books of the editorials: Manning, Apress, and Packt.
steinhauer.software