A talk I’ve given this year has often resulted in being asked to provide more details. This article is part of that effort. Keep an eye out for more from me on this subject.
There are many guides on achieving the right architectural balance and multiple thoughts about the ‘right’ way. In this article, I want to distil some of this thinking, apply it to Java API design, and use modern Java features. Historical patterns like AWT or RMI often influence older Java API design thinking, and it’s time to remind readers that modern Java features offer new, more powerful, and frequently safer ways to design APIs.
History
We’ve seen several approaches to API design over the years, and at their heart is the desire to create an architecture that balances simplicity over adaptability and future enhancement. As a Java developer, you’ll often hear references to some standard terms:
Separation of Concerns: This fundamental design principle advocates separating code into distinct sections, each addressing a separate concern.
Strategy Pattern: In design pattern terminology, particularly from the “Gang of Four” book, the Strategy pattern encapsulates different strategies in separate classes and allows them to be swapped easily without affecting the context
Inversion of Control (IoC): While typically associated with dependency management, IoC is also about decoupling the execution of a task. It creates a hard barrier between code, what the application does, and what the framework provides.
Command Pattern: This pattern separates the action’s requester from the object that executes the action. Command objects encapsulate a request as an object, letting you parameterise clients with different requests, queue or log requests, and support undoable operations.
Delegation Pattern: This pattern involves two objects where one object handles a request by delegating to a second object (the delegate).
There are many other patterns. In fact, this is a rich vein of discussion, and there are many opinions.
Lines in the Sand
All the great concepts above have tremendous value, but where those architectural lines begin to melt away is when we talk about ‘participation’. Any design requires thinking about how the user interacts with the API. How do we draw hard lines and create firm boundaries when code can be malleable, and developers are inventive? Often, our design requires or invites participation in unexpected ways. Ways we didn’t intend or expect.
In this article, I want to extract an underlying principle or two and look at what we have available to create robust designs. Designs that allow just the right sort of participation without too much opportunity to directly or inadvertently compromise our API.
Classes and Interfaces
Whatever design pattern you follow or the framework you use, you still write code.
The Java API design toolkit we use essentially consists of Classes and Interfaces. We might use Annotations to signal requirements to containing systems, but, as the name suggests, they are annotations and do not have to be honoured.
As an API designer, you will consider how someone consumes the API, what constraints you must apply, and where. There are many functional and nonfunctional elements to consider, such as performance, security, observability, etc., and you have to decide how the consumer of the API will participate in all of these factors.
Participation – it’s not a spectator sport.
Participation means assessing how much the API user is a consumer rather than a provider. Historically, the way we use Java classes and interfaces has remained unchanged.
Common approaches include
- Using the provider pattern to allow objects to be ‘magically’ instantiated. i.e. via IoC or a Java Service Provider mechanism.
- Providing an interface and Javadoc to explain expected behaviour for each method
- Providing an abstract class for others to subclass and fill-in-the-blanks.
- Providing a concrete class that someone can subclass to override behaviour
- Providing a concrete class that has a constructor with parameters
- Providing a concrete class and a selection of setter methods – the JavaBean pattern
- We might occasionally use the Builder pattern to allow the instantiation of a complex object.
Force Fitting
We assume and occasionally try to force how the end user will participate in all these standard techniques.
IoC tries to force the user to be only a consumer. Builder patterns or final classes also attempt to force consumption only, albeit with some flexibility in configuration. Meanwhile, abstract classes and plain interfaces invite the user to participate in the API’s internal behaviour as both a producer and a consumer.
The JavaBean pattern is a dreadful mix of consumer and producer but highlights the other aspect of participation: Understanding your place in the process. Ideally, once an object is instantiated, it’s fully configured, and any other method calls are to retrieve data or transform the object’s state. What the JavaBean model does is blur configuration with use. Any setter method can be called at any time so the implementation has to deal with the chance of its configuration being modified during use. That’s a challenge and a ready source of bugs.
The API design invites a particular form of user participation. JavaBeans are bad practice, and so is using subclassing as a consumption principle. Whether an abstract or concrete class, the requirement that the user use subclassing to participate must be revised.
This approach requires the user to know much more about the API’s work; the user code is now part of the API. The user code must understand its responsibilities and its place in the process. It needs to understand what state its parent is in, when it will get called, and what it’s allowed to do when accessing the parent state.
Overriding or implementing methods as an API design choice forces the user to become both producer and consumer, encouraging them to override other methods or dive into the parent class’s internals. The tie between user and API is now strongly coupled and personal.
Let’s talk dishwashing machines.
Examining one, we can see that simplistically we have a few buttons and knobs that form the control panel, and we interact with the machine by loading salt, coupling up power, water and drainage and then regularly loading the machine with the dishwasher tablets and, of course, dishes.
We can see three sections by distilling this physical design into software principles.
Configuration: Where the machine is plumbed in and connected to a power supply.
Policy: The control panel allows the user to describe what they want the machine to do.
Process: Loading, policy execution, washing the dishes, Unloading etc.
Note how the dishwasher users’ participation is controlled and anticipated. Of course, they must remember to load dishes or tablets and not overload the dishwasher. Many of the common mistakes are discoverable by the machine. Open the door, and the machine stops. No tablets or softener, etc., so a light goes on. Depending on how clever your machine is, it is more likely to defend against common mistakes.
At no point in these activities is there any expectation or ability for the user to step outside their part of the process. There is a clear separation between what the machine does and what the user does.
The user doesn’t get involved in the actual act of dishwashing. They support the machine in fulfilling its function. The user selects a policy to tell the machine what sort of washing to do, provides dishes to wash and then extracts the clean dishes.
Configuration, Policy and Process
This model is great for API designers to follow. Thanks to recent additions to the Java language, we now have the tools to deliver robust, secure APIs and control user participation.
Configuration is where the environment is discovered, and elements that affect all usage are gathered. This might be via a pre-instantiated builder class, loaded by IoC, etc., or the parameters passed into a constructor. Think of them as ‘rules of physics’ rules that are fixed for the API during its lifetime in the JVM.
Policy is like the washing machine control panel, the rules that this particular instance of the API will follow. Policy declares what you want the API to do, not how it does or what data it will process. A policy is a reusable but immutable object. Think of it akin to something like a declarative contract.
Process is the well, process of executing the policy on a particular data set. The process uses information from the policy and configuration to do its job. In Java terms, the policy might contain selection criteria for data as a predicate. The process uses this predicate internally but cannot interact with any other part of the process. It’s isolated and contained.
Clean Design
The separation of policy from process is an essential concept that promotes cleaner, more modular, and maintainable code. This principle contrasts with some of the older design paradigms in Java, where the lines between policy and process were often blurred, leading to complicated codebases that needed to be revised to maintain and extend.
Older Java designs, such as those found in the Abstract Window Toolkit (AWT), encouraged a blending of these two aspects. Developers often had to subclass abstract classes or override methods to integrate with an API. This design forces developers to engage deeply with the internal workings of the classes they are extending, leading to tightly coupled code that is difficult to manage and prone to errors.
Practical Implementation in Java – expanding the designer’s toolkit
I hinted a little about the use of predicates above, but that’s only a part of what is available to us in modern Java. Today, Java gives us classes, interfaces, lambdas, streams, enums and records. Add in inner classes, sealed classes and even sealed jars, and we have an extensive palette of tools that allows us to deliver better and cleaner APIs.
There’s too much to unpack in this article so let’s look at a working example.
Scenario
The scenario is that of an API that can visit a tree structure and stream the contents. There is an implicit visitor pattern here, but it’s hidden away. The consumer’s experience is more straightforward.
Policy
Let’s define the policy for navigating and traversing a tree structure. We need to know how to interpret a node in the tree as a parent and decide which nodes to visit. Beyond that, there might be particular types of trees we’re going to visit that will have other policy requirements. If the tree is a view over a website., where we’re streaming the URLs we discover, we’ll want to add rate limiting and possibly a depth limit. If we’re streaming a file system, we might want to keep the depth limit, but rate limiting is probably not useful.
Builder Pattern – Part of the API
Since this scenario is complex, our policy will be created via a builder pattern. Unless the policy is simple, using a builder gives us more flexibility to provide an interface that is easier to police for misconfiguration and easier for the consumer to understand.
import static NavigatorPolicyBuilder.builder;
NavigatorPolicy shortPolicy = builder()
.rateLimit(100, Duration.ofMinutes(1)) // play nice
.defaultReader(new HTMLRefNavigator())
.maxDepth(2)
.build();
In this example, we’re setting the policy on how to navigate a website. We’ve created a policy that we can reuse. In this example, the builder pattern (using fluent style ) is easy to understand and validate, resulting in a reusable but immutable policy object. The builder is part of the API but is not a process object.
Secret Sauce
In this scenario, NavigatorPolicy is a Java Interface. Until recently, that would allow anyone to implement it and subvert any restrictions we might put in place via the builder. With the advent of Java 17 and the arrival of the ‘sealed class’ feature, we now have control over who can implement the interface.
This is important because we want the NavigatorPolicy object to be read-only, so we want to remove any public ways of changing its data. Therefore, we must separate the mutator methods from the interface and hide them away.
Sealed classes let us define which can implement an interface and which can subclass a class. Only those in the permitted list can be implementors or subclasses.
The code for the Navigator Policy interface looks like this.
public sealed interface NavigatorPolicy permits AbstractNavigatorPolicy {
LinkReader handler(Connection.Response r);
}
This code indicates that only the AbstractNavigatorPolicy class can implement it. AbstractNavigatorPolicy definition includes
public abstract sealed class AbstractNavigatorPolicy implements NavigatorPolicy permits NavigatorPolicyBuilder.MyUriPolicy {
It implements the interface and is itself sealed, allowing only an inner class of the builder to subclass.
MyUriPolicy looks like this. Thats it. Note the final setting and private constructor.
public static final class MyUriPolicy extends AbstractNavigatorPolicy {
private MyUriPolicy() {
}
}
Class Relationships
This code pattern of a sealed interface + sealed abstract class + inner class concrete (hollow) class provides good code separation and strong controls on how they interact and can be used.
The API consumer has a NavigationPolicy that can be interrogated but not overridden or replaced. Without using reflection, there is no way for the consumer to discover policy internals.
From the API designer’s POV, the builder is part of the API, which is evident in its intention, yet the implementation details are hidden away and can not be changed. The builder might return MyUriPolicy this time, but maybe a change in the future means that the next time build() is called, it returns YourUriPolicy.
The consumer is not affected, and the API is rigorous yet evolvable.
Process Time – STREAMS and things
Here’s a related code set for using the API to do work.
‘base’ is the URI the API is going to visit. The policy object comes from above.
URITreeSteamVisitorBuilder.newInstance(base)
.policy(shortPolicy)
.select()
.on(Link.class)
.consume(this::handleLink)
.visit();
The first thing to note is that we have a reasonable DSL-like interface using a fluent style with judicious inner classes. Next is to see that the consumer’s involvement with the process is controlled. The consumer is effectively on the end of a stream of data.
In this case, the API allows some inline filtering for architecture and performance reasons. Still, it could have been written to return a stream and have the filtering and processing to be done entirely by the consumer.
We did not need to ask the consumer to provide a custom class other than the callback for received data.
Let’s expand our use of this builder to see how we can do so much with this style of API design.
URITreeSteamVisitorBuilder.newInstance(base)
.policy(shortPolicy)
.select()
.on(Link.class)
.consume(this::handleLink)
.on(Meta.class)
.consume(this::handleMeta)
.otherwise()
.consume(o -> {System.out.println("other:"+o);})
.visit();
Now, we have an additional on() clause and an otherwise() method. Hopefully, you can see how this mirrors a select expression. Again, the API provides ways for the consumer to participate in the process, but as an actual consumer, The internals are kept hidden away. The API is not just a simple class or interface. The API here is a fluent style, DSL-like mechanism that is flexible and extensible while allowing the API designer control over behaviour and interaction.
A bonus is that this style works well with IDEs; the same code seen in the IDE looks like this. Note all the additional info about the intermediate classes of this DSL-like construction. This really helps with understanding the available options as the DSL is used.
DSL-Like is your new API
From an API design POV using sealed classes to add extra controls on who can do what with your code, it is compelling. You could stop there, having already reduced the chances of your API being deliberately or accidentally compromised. Including the builder approach (with the same sort of DSL-like structure) takes the API design into new territory and transforms it from a static Java interface or class into something more dynamic. Now, your API can be a sophisticated builder that can be extended in ways that a simple Java interface or class would struggle to match.
Conclusions
I hope I’ve given you just a taste of what can be achieved by using the features of Java.
Using Inner classes, Sealed Classes, Lambdas, Streams, Enums, etc., and by thinking again about how we separate policy from the process, it’s possible to create APIs that provide clear, easy-to-use yet enforceable designs that are hard to compromise but can be extended and enhanced.
