I had a chance to reconnect with my colleague Cesar Saavedra who recently attended the InterConnect conference in Las Vegas. If you weren’t there, or weren’t able to follow along with the livestream, you can find all the conference highlights, recordings, and materials right here.
For those of you who don’t know Cesar, he is an IT professional who has spent a good part of his career focusing on integration challenges – with hands on experience working with TIBCO, Oracle, and IBM solutions – and more recently he has been doing a lot of work in the fast evolving API Management space.
I was interested in getting his take on what is happening with API Management, how the competitive landscape is changing, and what we should expect going forward. I also wanted to get his take on the new API Connect offering announced at InterConnect.
So with that said, let the interview begin!
Hello, Cesar! Welcome back. I hope you survived Las Vegas with your wallet and your sanity intact. First, thanks for taking the time to share your thoughts on InterConnect with me. There were a number of very interesting announcements and it seemed from watching the livestream that the mood was pretty upbeat. What did you think?
The mood was exciting at InterConnect, which is IBM’s main customer-focused annual conference. There were many interesting sessions, hands-on labs and keynotes about our current products, recent acquisitions, partnerships as well as roadmap presentations. Personally, I was impressed by our continued innovations in the areas of IoT, DevOps, analytics, cognitive solutions, and hybrid cloud and integration, including our new IBM API Connect and IBM Application Integration Suite.
As I mentioned when we were planning this discussion, I wanted to focus our conversation on the integration space. I am particularly interested in getting your thoughts on what is happening in the world of API Management. But before we go there I think it would be useful to talk a little about what APIs are.
I’ve noticed that there has been some confusion between ‘services’, as we’re familiar with them in the Service Oriented Architecture sense, and APIs. Can you help clarify what we mean when we use the term APIs and how they differ from SOA Services?
As far as I can remember, the word “service” has been around since the early 1990’s when client/server architecture became popular. Protocols like RMI, CORBA and technologies like Java supported this architecture and, as others started to proliferate to support the Internet, it became a challenge to compose an application that depended on services that communicated on different protocols and data formats. So, Service Oriented Architecture was to address this challenge and technologies like WS, SOAP, HTTP, UDDI, etc. accelerated its adoption. Supporting SOA components were ESB, Registry, Repository, Data Services Platform, Policy and Security Managers, etc.
Since then the world has changed quite a lot. Always-connected digital devices such as smartphones, tablets, personal game consoles, watches, smart TVs, and the like created new business channels to sell and market products, which is the basis of what people today call the Digital Economy. Unlike SOA, where the architecture supports the inside-firewall side and applications need changes on a quarterly basis, this new channel introduced us to the world of apps that run outside the firewall and need changes on a weekly basis! Different operating systems, protocols, languages on each type of digital device complicate a developer’s life. Today we need to deliver an Omni-channel experience and developers have found in REST and JSON great fits to fulfill this due to their ease-of-use and non-binary nature respectively.
At the same time, organizations now realize that they can increase their revenues by making their services available to outside-the-organization app developers. All these events have brought about the need to manage all these services. Welcome API Management.
In API Management, you have outside-the-organization developers – API consumers – wanting to reuse readily available fee-based or free services to build apps. You need a way to securely and efficiently make these services available to these developers while managing their consumption. An API allows you create an indirect connection to your service implementation. Think of an API as a wrapper or a proxy that fronts the call to your business service (it could be a SOA service inside your firewall, for example). The API is what’s available to the external caller and is what you manage – apply security, throttling, caching, validation policies to it, for instance. The API does not contain any business logic or a long running process as this is what the business service does. All these policies are enforced by the API gateway, which must be a fast performing component handling thousands of API calls per second. This is another reason why you should never run business logic in your API gateway.
Although API consumers are usually outside the organization, you can think of your own in-house developers as API consumers as well. Hence, we are seeing a lot of companies adopting a dual approach to API Management, one to manage APIs for outside-the-organization consumption and one to manage APIs for in-house consumption.
Lastly, to make this process expedient, you need a consumption model that is lean and self-service and this is what a Developers’ portal does.
So, what does it mean when we hear people talk about a ‘Microservices Architecture’?
Microservices Architecture often comes up when discussing API Management and the reason is that there is a strong relationship between them. Service-oriented Architecture and Microservices Architecture are two different software architectures and one is not a substitute for the other one. Each one has its own characteristics, strengths and weaknesses. There are cases where having a SOA would be the best approach – Systems of Record applications – and there are cases where Microservice Architecture would be the best option – typically Systems of Engagement applications. A microservice is defined by the way you implement a service and is not a ‘small service’, which is a common misunderstanding. Obviously, a microservice must do one task and do it well but to describe it as a ‘small service’ does not do it justice. Also like in the case of SOA, a microservice is not defined by the language or protocol it uses. In fact, one of the main tenets of Microservice Architecture is that it allows for changes to happen in a microservice without the need to re-test an entire application, where the application is the composition of many microservices.
Microservice changes could include not only functionality changes but also implementation language or framework changes that the microservice is based on. The way this is possible is that one must implement a microservice in a way that is completely autonomous and independent of any other microservice. The only entry/exit point in/out of a microservice is its API. This means that your microservices cannot depend on or share an Enterprise Message Format or a set of database tables since this would violate their independence and autonomy. Further, transactions that span across microservices are not allowed, so compensating logic could be used to mitigate this, for example. So, an application that’s implemented using microservices can withstand a high rate of changes necessary in the Digital Economy. This is why Microservices Architecture is becoming so popular among application developers.
I’ve had discussions with people who view the API gateway as though it could be used in their organization as a lightweight ESB. What are your thoughts on that line of thinking?
An API gateway is not an ESB since they have different functions and solve different problems. An ESB is an intermediary for service calls and is a key component in SOA, whose focus is for behind-the-firewall applications. An API gateway, on the other hand, mediates API calls and is a good fit for outside-the-firewall applications.
To say that there’s no longer a need for an ESB is to deny that it serves a different purpose than an API gateway. In addition, there exist a huge number of corporate applications that are based on SOA and utilize an ESB and it’s an impossibility that customers will throw away their working SOA-based applications and re-write them or replace them with a Microservice Architecture solution. Just like the mainframe, SOA is here to stay.
I recently wrote about the importance of hybrid cloud and how leading organizations are adopting this approach as their core technology strategy going forward, and one of the ways they enable that strategy is by both exposing their technology assets as APIs and leveraging APIs available from other organizations.
Customers currently have working solutions that are running their businesses. They are not going to throw everything away and replace them with the latest technologies from one day to the next. Legacy applications will remain in production for years to come and will have to integrate with newer technologies. This is why hybrid is important. We cannot expect our customers to move to the cloud from one day to the next. And there are a myriad of reasons why customers may not want to move all of their systems to the cloud (government regulations, security and personal information requirements, etc.).
In other words, organizations need to support applications, which may have cloud-native components and on-premise components. APIs allow customers to make functionality in their legacy systems available in an easily consumable way using REST and JSON, for example. And cloud-native applications also support API access to their features. To consume APIs or to let others consume your APIs, you need a way to manage and secure them otherwise they become unwieldy. Not only does IBM API Connect provide these API management and security features but also microservice creation and running capabilities and it offers the capabilities on cloud or on-premise.
So, getting back to your question, the short answer is ‘yes’, API Connect is going to help organizations accelerate that process. That’s important too, when you consider how fast business environments are changing with new competitive pressures every day. You either innovate in today’s world or you quickly get left behind.
Are there any new capabilities that now become ‘must haves’ as organizations evaluate their API Management needs? I’m thinking beyond just having a repository and governance model as well as a portal for discovering APIs.
The biggest change we’re seeing now is that it’s no longer enough to have a solution that stops at managing APIs. In order to digitize their business and realize the promise of the API economy, organizations need the tooling and support to expose their services as APIs as well as the development tools and DevOps capabilities to create an API and their service implementations using languages like Java and Node.
So that brings us to IBM API Connect, which is the evolution of IBM’s API Management platform. What API Connect provides is the support for securing and managing APIs as well as creating and running their corresponding service implementations. The user interfaces have the same look-and-feel throughout and the ease-of-use of API creation and management has been extended to the creation and management of service implementation. For example, the platform supports the automatic creation of Swagger for services implemented in Node. Currently, Node and Java are supported for service implementation but support for more languages is coming in the future. As you would expect, API Connect also provides out of the box integrations with other parts of the IBM portfolio, such as IIB, WSRR, Cast Iron, mainframe, so it’s part of a cohesive hybrid cloud solution.
With API Connect it is worth mentioning that the service implementations run on their separate containers and most importantly do not run on the API gateway, whose function is to serve as the enforcement point for all policies applied to APIs and as such, it needs to have high performance to process thousands of API calls per second. The API gateway must not execute business logic in it since this is what the business service implementations do. IBM API Connect includes components to support the creation and running of business service implementations, and the securing and management of their respective APIs. All these components are fully integrated and supported by IBM.
What’s nice about the way the platform is set up is that if all you are interested in is just Node development then you would just purchase the number of cores you need to do Node. You don’t have to use the other components of the product if you don’t want to. When the time comes that you want to start managing APIs, then you can either run the other components on the cores you already own or you can purchase more cores of IBM API Connect or even pay by consumption.
So, there’s lots of flexibility with the platform and it’s all in line with the IBM philosophy of only having to pay for what you need. For a lot of our competitors you’re forced into buying their whole stack of software just to run the parts you need today. That’s the dirty little side of the hardened monolithic ‘software suites’ that those vendors – Oracle, Software AG and the like – tend to gloss over. That’s one of the prices you’re going to pay for software dependencies.
This article on the TechTarget site includes a comment that “the integration problem has moved from inside the organization out to the edge” while discussing APIs generally, but I think that prompts an important consideration around the enterprise gateway. Are there gateway capabilities that set API Connect apart from its competitors?
An API gateway is a key component in an API Management solution. It’s the enforcement point for API policies. To be enterprise-grade, a gateway must at least be high performance and secure. High performance because the API gateway must sustain high throughput traffic from API calls flowing through it. And in this Digital Economy, as APIs proliferate and are leveraged by apps, this traffic is only guaranteed to increase. This is why it is so important to consider an appliance – whether that is a physical or virtual appliance – as the API gateway. A purpose-built appliance API gateway has the advantage of running its logic directly on digitally signed and encrypted firmware with built-in optimizations, which insulates it from Java and OS-related vulnerabilities and gives it higher performance.
There are vendors in the marketplace claiming to have an appliance API gateway like CA and Axway, however, theirs is made up of software running on a Linux-based server, which is really not an appliance. There are other vendors whose API gateway is simply a software application you need to install on your server like Apigee and MuleSoft. These vendors will claim that’s less expensive than purchasing an appliance API gateway, however, I would strongly recommend working a TCO analysis of this type of solution to compare, since you will need a lot more cores to handle the same throughput vis-a-vis an appliance, resulting in higher maintenance and support costs, not to mention the ease-of-use, performance and productivity advantages of using a purpose-built appliance API gateway. Security is also a very important feature of your API gateway and having a locked-down appliance provides higher levels of security than a software solution running on a Linux server and Java. Again, my suggestion would be to compare the level and efficacy of security features offered by the software-based API gateway vendor versus the appliance API gateway vendor.
IBM offers a true purpose-built appliance API gateway, available in physical, virtual, Cloud, and Docker form factors, deployable to many IaaS providers, such as Amazon and IBM SoftLayer.
It seems that the API Management market has been a bit fragmented with a combination of ‘pure play’ vendors, traditional SOA vendors, and the cloud-only options like Google, Amazon, etc. What do you think happens with the competitive landscape for API Management solutions over 2016?
Everyone is getting on the API Management bandwagon so you will find vendors coming from all sorts of directions into API Management. For example, you have vendors like IBM, whose API Management is a fit-for-purpose solution using best of breed capabilities and technologies. Then you have vendors “spinning” their existing solutions into the API Management field, having only updated their marketing material to position their existing products within the context of API Management, like Forum Systems. And finally, you have vendors that have added API-Management-related functionality to their existing solutions, such as Akana (ex-SOA Software) and Oracle, who leverage their SOA-related legacy products in their API Management solutions.
Some of these vendors offer on-premise-only solutions or cloud-only solutions or both. Customers are not going to move all of their applications to the cloud from one day to the next so we need to support customers with an API Management solution that can support on-premise and on-cloud deployments. This is the direction of the market in 2016, and vendors that don’t support this model will suffer. This is what we call hybrid cloud, whose guiding principles are: choice with consistency, hybrid integration, powerful and accessible data and analytics, DevOps productivity, and cognitive solutions. At this point only IBM is delivering on all of those capabilities today.
From the market perspective, there has been some consolidation already -ProgrammableWeb got acquired by MuleSoft, Mashery got acquired by TIBCO, StrikeIron got acquired by Informatica who then went private in April 2015, SAP and Pivotal have partnerships with Apigee, Pivotal also partners with 3scale for their API Management solutions, and Oracle OEMs the Axway API Gateway as their Oracle API Gateway. There may be some more consolidation in the API Management market during 2016. Apigee went public in April 2015 and is currently facing investigation for potential securities law violations due to the huge drop in their stock price since going public, but it may be a target for acquisition by a larger company looking to obtain API Management capabilities. 3scale may follow the same fate. And who knows what SalesForce may decide to do with their investments in Informatica, Jitterbit, and MuleSoft – three vendors that offer different API Management solutions.
Any thoughts on who might be in trouble here given a too-narrow focus on what an API Management solution should provide, and what impact API Connect might have on that landscape?
So, let’s quickly look at some of IBM’s competitors.
Apigee supports the execution of Node on the API gateway but provide no integrated or supported Node tooling. Running Node on the gateway itself is not a good idea since this could potentially affect the gateway performance, e.g. imagine one of your developers writing a complex Node process on the API gateway! Akana supports BPEL in their virtual services but not Node. Oracle and Pivotal support Node but as a service in their respective cloud services completely unrelated to their API Management offerings.
Lastly, most IBM’s competitors rely on Amazon for their IaaS, which will affect customers’ production problem troubleshooting times since they will have to deal with the vendor plus Amazon to resolve production issues and that can potentially lengthen their downtimes. And even, if customers are insulated from Amazon as the IaaS provider for their API Management solution, that interaction with Amazon is still taking place even if they don’t see it. With IBM API Connect, customers get a fully integrated and supported solution to create and run microservices and to secure and manage APIs. For our cloud-option, you get support from a single vendor from software all the way to the IaaS.
What do you think it’s going to mean to organizations that have invested in API Management platforms that are struggling to gain market acceptance?
There are many API Management vendors in the market and customers should take into consideration the following criteria when making a decision as to what solution to select: 1) Solution viability 2) Standards 3) Completeness of solution 4) Choice of deployment.
The API Management market has not settled yet and is still very dynamic with respect to the number of players that currently exist. Small vendors are being acquired, some are going private, some are going public, and some are disappearing. API Management is here to stay so make sure to select a solution that is owned by a company that you know will still be around in 5 years.
There exist many standards that are used by API Management solutions, for example REST API description languages like Swagger, RAML, WSDL 2.0, OData, I/O Docs, API Blueprint, WADL, etc. are all supported by different vendors. For example, Mule uses RAML and Mashery I/O Docs . Swagger has gained a lot of popularity recently and it is becoming the de-facto standard for REST API description language. In fact, the Open API Initiative was created in November 2015 by some of the main API vendors and users in the market such as IBM, Microsoft, Apigee, 3scale, Google, Capital One, PayPal, among others, in an effort to standardize REST API description languages and selected Swagger.
Current API Management solutions let you create and manage APIs making the assumption that the business services that the APIs front already exist. What if you want to also implement the business logic associated to an API? Wouldn’t it be more efficient and productive to have the ability to implement the business logic behind the API in the same platform? This is what I mean by completeness of solution.
Lastly, choice of deployment is very important since nobody is going to move all of his or her applications to the cloud. There will always be the need to run applications, including API Management, on-premise and on cloud, in other words, in a hybrid environment, which we call hybrid cloud. Again, IBM API Connect successfully delivers on these four criteria.
Any last thoughts before we wrap up?
Yes. Under the hybrid cloud umbrella, hybrid integration is also becoming very important for customers that have a need to integrate on-premise applications and cloud solutions. At InterConnect, IBM announced the IBM Application Integration Suite, which addresses the needs of the shadow IT integrator and central IT integrator. IBM AppConnect, also announced at InterConnect, is a brand new product for use by the citizen integrator to integrate on-premise-to-cloud or cloud-to-cloud applications. But we can talk about this next time.
Thanks Cesar for a great discussion on API Management. I hope our readers found this helpful and perhaps we can pick this discussion up in a week or two.
We would love to hear your feedback so please leave a comment below, and remember like, subscribe, and share!