As organizations move to API-lead approaches to architecture and design new API target states for the organization, the concept of a set of publishing a set of experience based API services no doubt come into play. API Management application vendors (Mulesoft, and others) have touted “Experience API”‘s as a necessary part of API-lead design and have provided in-built capabilities of a layered “Experience API” built into their platforms. However, these vendor’s didn’t invent the concept of specific APIs based on experiences as this practice has been around for quite a while and in multiple forms. These types of APIs are technology agnostic, so you don’t need any particular APIM platform to build, use, or consume them. Understanding how and when to abstract your monolithic APIs into a set of experience based APIs is key.
The idea of an Experience API is to provide a single API service endpoint to a very particular consumer, business context, or application. Instead of a one-size fit’s all approach, you create a single service for a single type of consumer that satisfies the needed or mandated technical, business, and user requirements of that consumer. This means that you would have a set of API’s which would provide a similar service but specific to the consumer each was designed for.
Consider a typical API (logical architecture) to handle a straight forward product lookup use case:
When looking at the above implementation, it seems reasonable right? You have all of your logic and implementation details for any client, regardless of the client, in a single re-useable API. So, let’s just call it a day, right? Well, not so fast. In the above implementation, you have many consumers. The enterprise has rolled out it’s own mobile apps for use by the public, it’s retail stores use legacy POS systems which also need to easily lookup and search for products in a multitude of ways, there is a public web app, and even a new, very recent, capability added which allows 3rd party’s to make use of and consume product data, inventory, and pricing as part of affiliate programs.
Over time, this API has evolved and more and more features have been added which has lead to “API bloat” while also requiring extensive testing to all of the consumers using the API whenever there was an API change that benefited only a minority of consumers.
When you think about the core requirements for all of these consumers they are pretty similar. However, there are some big differences. Consider the following:
- The legacy POS doesn’t need all of the product specs, it doesn’t need product images either. Its very simple use case is to lookup a product or do a quick search by name and check inventory. Unlike, the mobile app or e-commerce store which require images, product, specifications, and detailed inventory.
- The product API supports paging, but that is not even a capability that can be utilized by Legacy POS. In fact Legacy POS needs data in a comma delimited format and not JSON format which is the only one of the consumers needing this format, so the API is designed to return comma delimited data when it gets the appropriate request asking for data to be returned in that fashion.
- There are differences in the way the mobile app and e-commerce store utilize data as well. The web based e-commerce store requires additional data elements in the returned data response that is specific to the requirements of the e-commerce store platform it is built on.
- Now add the most recent capability to the mix which is the affiliate program. Now, partners, can build websites or applications and utilize product data as part of the affiliate program by authorizing to the API and making API requests. They do not need all of the data provided by the Products API and certainly some data should not be made available to them at all.
I’d call this a “Monolithic API”. It doesn’t necessarily mean the entire application is a monolith, and you could still have this same API getting the job done by orchestrating calls to multiple microservices but still supporting all of those consumers with a single inbound API endpoint. So, even with a microservice strategy, you could end up with an API service that is still quite monolithic from an experience perspective.
In this example, we don’t do “Product Lookup” particularly well or efficiently. All the logic to meet the core needs of all of the consumers is baked in to a single API endpoint with multiple levels of logic and processing in order to ensure you are giving the right consumer the right data back in the right format solely based on how the consumer crafted the query to get what the consumer needs. You could argue that this means the API is powerful – it can handle anything you throw at it – and it’s flexible. All of that is true. But, the following statements are also true:
- Tightly coupled logic for the end user experience
- Complexity and technical debt on the API side and on the client applications side
- Less flexibility to make specific changes to satisfy a single type of consumer later on – requires re-testing all consumers
- Less versioning flexibility
- More complexity, effort, and testing required for client development
- Developers tend not to like overly complex APIs which can impact developer adoption of your API
- Can’t scale well for a specific type of consumer, so scaling needs to be done for the holistic API for all consumers (also applies to DR, SLAs/Availability/etc) (Note: this is dependent on the capabilities of the API Management product at play)
- Can’t easily change the internal logic for a different consumer to use different sets of Internal APIs, services, or data sources
- No targeted business governance on the API level as the API handles multiple scenarios for different types of users under different business contexts
If these things above are not significant or important to you, then carry on with your monolithic ways if you must. Otherwise, consider a new facade in front of your APIs to handle requests from different types of consumers. This is your set of experience APIs or an “Experience Layer” and can form part of your core API capabilities for your enterprise.
How can we take this “Monolithic API” and transform it into a set of different experiences from an API perspective? Well, you first need to make some design decisions to understand how much separation of concerns from an experience perspective that the organization needs. If some APIs with different types of consumers are very similar, there may be a case you can make to keep a single API for that experience. In our example here, and in consideration of the API requirements indicated above, let’s consider that we plan to abstract the single monolithic API into four distinct Experience APIs as part of an omnichannel user experience strategy.
As the enterprise moves towards a new API target state which can include the use of Experience APIs you end up with experience specific APIs as a new API capability across the enterprise. You’ll get added flexibility to assign business owners and provide governance to each of your core experiences.
Here, we have met all of the constraints and requirements by abstracting the logic into four very specific consumable APIs implemented as part of an overall Experience API strategy instead of having all of the logic baked and bloated into a single API regardless of the application or consumer. Developers that write code to consume the APIs will no longer need to worry about the best and most useful way to navigate and utilize the API to get the data they need in the most useful way possible for their applications which greatly simplifies the implementation on the client side, and is a huge bonus when considering developer adoption of APIs provided to entities outside the organization. Because of the segregation of the APIs, you now have much better targeted control of scaling, availability service levels – you may put a much higher emphasis on scaling and availability for POS requests, followed by web and mobile, with less emphasis on affiliate traffic from a service level perspective.
As you expand the architecture out, you see how the Experience API’s sit on top (the facade) as the first point of contact for the consumer. Some of the original bloated logic in the “Monolithic API” has been moved to a new “Process API” which handles processing, business logic, and any further orchestration needed and is powerful enough to feed all of the data up the line as required and hand it off to the Experience API to implement “Experience” specific logic and return the appropriate, tailored, response back to the consumer. The Products (internal) API described in the logical diagram below could be a standard REST API service or even part of a microservices architecture which does further orchestration as needed.
Expanding out the architecture even further brings System APIs into play. This takes the data processing logic out of the Processing API (in this case our new Products internal API) and places that logic in the system API. To get System APIs in place, we’d put the corresponding data processing logic into each relevant System API and ensure enough API capabilities to provide the data back to the Processing API(s) that call it, use it, and then return it back to the Experience API that will take care of providing it to the consumer based on the consumer’s need.
As you move along in your journey towards effective enterprise API target state architecture, many significant decisions will need to be made at the enterprise level, business domain level, technical domain level, and solution level. As each organization and each API has different sets of requirements and constraints, defining a single universal architecture to be used across the lands doesn’t make sense. I’m also not advocating that an experience API is required for every single consumer or business context across the enterprise.
I like to loosely define architecture as a set of decisions that are made objectively based on business and technical need and in consideration of technical and business constraints. Architecting your API and defining API target states for the enterprise, business domains, and specific solutions will vary greatly depending on the organization. When considering your API architecture’s target state, consider how implementing a set of experience APIs and associated downstream API’s can positively impact your organization by facilitating better governance, creating a better user/developer experience, reducing complexity, lowering technical debt, and reduced lead time in promoting new functionality can lead to a more effective API-centric enterprise.