Integration Shift - API Spec with RAML / Swagger

Whether it is top down or bottom up approach, it is generally advisable to start on the mock servcies, api test plans, documentation along with the design. With modeling languages like RAML, Swagger and tools around these languages, you can have a live API experience while implementation is in progress, generate client / server code, create test cases and help in production monitoring.

Both Swagger and RAML provides api definition language, however it seems that they are distinguished by the vendor support and the tooling that support these specs. In its current state, both supports basic design elements of the RESTful like URIs, methods etc, however they may approach advance elements differently like Hypermedia, complex methods etc.

Companies like Apigee, Reverb  supports Swagger

- Swagger was released in 2011. has the most active developer community, extensive number of tools in most languages.

- Swagger creates a new section for each endpoint/verb which provides less level of nested indentation, which provides better readability but makes the API view granular at method level than at resource level.

- Node has a swagger module which creates a swagger.yml in the api folder of a node.js app. Project edit opens a swagger UI which dynamically changes the API view based on the changes in the swagger.yml file. 

- Swagger CodeGen is a template driven engine to generate client code in different languages.

- Swagger version is 2.0 and has been given to Open API initiative and has been renamed as Open API Specification supported my members like Apogee, Google, Microsoft, SmartBear etc

Companies like MuleSoft, SOA supports RAML 

- RAML is somewhat newer releases in 2013, has an active developer community and supports tools in most languages if not all.

- RAML is grouped around endpoints with drill down into verbs and keeps the needed visibility around resource level rather than verb level, however it does make it less readable with extensive nested indentation.

- As an offering in Anypoint Platform, MuleSoft has an  API Designer to create RAML specs. It also has an APIKit which generates Mule flows for the each URIs in a Mule Application.

- API Notebook is another tool by MuleSoft that is used to create client code to test use cases by composing different API calls and is based on Javascript.

- RAML version is 1.0 and supported by vendors like amazon web services, ca technologies, mulesoft.

Integration Shift - Http as a Uniform Contract

REST, an architecture style, when properly applied achieves a technology architecture that closely resembles the web. Although Restful API doesn’t mandate the usage of HTTP as a transport protocol, however in practice, http is the default choice for all restful apis.

REST leverages following elements of HTTP protocol  to provide uniform contract:

Uniform Resource Identifier (URI): HTTP URI provides a way of identifying resources within a service inventory. Generic Syntax for URIs is as follows:

{scheme}://{authority} {path}?{query}#{fragment}—->http://onlinestore.com/api/products?type=“toys”#page1

In this example, http is the scheme, online store.com is the authority,/api can identify an inventory or functional context , /products is the path that identifies the product resource within the inventory.

Some of the design guidelines while constructing resource identifiers:

- Path shouldn’t start with the variable part. For eg: avoid http://onlinestore.com/api/{products}

- Trailing slashes can indicate the collection of resources. For eg: “http://onlinestore.com/api/products/“ can give the list of products.

- Standardize the name of query params across the list of services or Service Inventory

- Use URI params for identifying the entity record rather than query param.

- Explicitly specify the query params that is need while constructing URIs. For eg: http://onlinestore.com/api/products{?type, price-range}

Method enables request- response communication with resources through URIs that includes methods, headers, response codes. 

Methods: Http provides set of generic methods such as GET, PUT, POST, DELETE, HEAD & OPTIONS that can be leveraged to interact with the resources and perform tasks like retrieving, creating, updating or deleting a resource. Service implementation at the server side controls what methods are allowed for a resource.

Headers: Http headers provides an additional information and can provide the metadata on how message should be processed by intermediary layers along the message path. They can be classified in several ways:

- According to their context: General Header,(Connection, Date, Cache-Control),  Request Header (Accept, Accept-Charset),  Response Header (Location, ETag , Age) , Entity Header (Allow, content-length, mime-type)

- According to Usage Category:

  1) Authentication (WWW-Authenticate, Authorization)

  2) Caching (Age, Cache-Control, Expires)

  3) Client Hints (Accept-CH)

  4) Conditionals (Last-Modified, ETag)

  5) Connection Management (Connection, Keep-Alive)

  6) Content - Integration (Accept, Accept-Charset)

  7) Controls (Expect, Max-Forwards) 

Response Codes: Http responses has a status code which specifies whether the request is successfully completed. They are grouped into five classes:

1) Information Responses (100 - 199) {100  Continue, 101 Switching Protocol}

2) Successful Responses (200 - 299) {200 OK, 201 Created, 202 Accepted}

3) Redirection Messages (300 - 399) {300 Multiple Choice, 301 Moved Permanently, 302 Found}

4) Client Error Responses (400 - 499) {400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found}

5) Server Error Responses (500-599) {500 Internal Server Error, 501 Not Implemented, 502 Bad Gateway, 503 Service Unavailable}

Media Types: This helps further specify the type of data that is be processed by an api. Media Type has the syntax  (type “/“ subtype *( “;” parameter). Some of the common media types are 

text/ plain; charset=utf-8 for simple representation such as integer or string data.

application/xhtml+xml for more complex lists, tables, human readable text, hypermedia links

application/xml for well-formed xml data that may or may not be schema specific

application/json for key-value pairs that can be parsed as javascript objects.

APIs usually processes “application” media type which can further qualified as vendor specific custom media types. Custom media types are usually identified with the notation:

application/vnd.organization.type+supertype (application/vnd.com.onlinestore.product+xml)

In the next post, I will write about some RAML / Swagger used for generating API specs & tooling around it.

Integration Shift - Restful Goals

Application of REST design constraints help in achieving following goals:

Performance: It is amount of time spent in getting the response for a single request. Assuming that network performance is absolute, some of the design decisions like keeping data close to where it is processed, number of request optimization for a use case, omitting irrelevant data, pagination can improve user perception.

REST constraint like uniform contract negatively impact performance as forcing single technical interface keeps it resource based which can result in increased round trips for a use case.

However constraints like cacheability  & stateless prevents extra request and keeps request simple & self contained.

Scalability: It is  the ability to support large number of instances or concurrent interactions. It is all about serving all your clients in an acceptable amount of time and prevent Denial of Service.

REST constraint like Layered System & Uniform Contract allows load balancers as an intermediary layer without changing the service contract. It is further simplified with the Stateless constraint as each request can be handled independently and there is no need to synchronize session state explicitly between service instances.

Simplicity: Rest is all about simplicity and is achieved by proper application of separation of concerns. Client requests the representation of resource through a URI (Uniform Interface) and content-type header and it is up to server to handle the state by correctly applying (Layered system, client-server) constraint and return an appropriate status code. Application of Stateless constraint keeps the interaction simple & self-contained without waiting & referring to earlier requests.

Modifiability: It is the measure of how easy it is to make changes to the system. With the constrains like:

Uniform Interface allows the ability to refactor / redeploy service without impacting clients.

Stateless constraint provides ability to add / modify functionality even when solutions are running.

Layered System constraint allows the add new solutions like security, logging analytics without modifying the services. 

Visibility: It is understanding of interaction between the service and its consumers and ability to monitor & mediate that communication.

Uniform Interface like http provides the generic information to the service agents which enables to mediate / control the traffic with features like load balancing, security, analytics etc.

Portability: It is the ease at which service and solutions can be moved from one deployed location to another.

Code on Demand supports portability by client dynamically implement the upgrades as service is moved from one cloud environment to another.

Uniform Interface & Layered system supports portability by keeping the same technical interface is part of service logic is moved to different technology  / environment.

Reliability: Reliability is the measure of the level of fault tolerance built into the system. It is the ability of the system to maintain availability, even when things go wrong.

Restful constraint like Stateless enables intermediate load balancers to easily route the request to a available node without worrying about state information as each subsequent request is independent. 

Layered System can help introduce monitoring & load balancing at different levels which can increase the fault tolerance of the solution.

Uniform contract can impact reliability if side effects are not considered properly while modifying the state of resource using PUT (not safe but idempotent) & POST (neither safe nor idempotent).

In my next post, I will write about http as a HTTP as Uniform Contract.

Integration Shift - Restful Constraints

REST stands for Representational State Transfer. It is resource based (in spite of action based) architectural style  which supports http protocol.

Resources are identified by URIs and it is the representation of resources  that is transferred between client & server. When client get the representation of a resource, it should have enough information to identify, modify, delete the resource on server, given it has appropriate permissions . Just for an example, user can be a resource which can be a database entity with 20 attributes and its representation can be userID, name, email address and one or combination of these attribute should be enough to uniquely identify a user.

REST has following design constraints:

Client-Server - This means that api solution should be implemented as client side (for eg: browser) which retrieves the data, and server side which hosts the business logic & data. It enables separation of concerns which keeps user interface design separate from service design. Implementation of this constraint assumes a disconnected system for eg: client side implementation may not have have direct connection to databases/ resources.

Stateless: This constraint specifies that each request is independent irrespective of the order and have enough context to process the message. Any session state is maintained at the client side.  For eg:

- Each request should have a token which mentions that user is authenticated.

- Each request has their pagination tag to specify the dataset expected in the response.

- 3 Legged OAuth is a stateful api, though built in a restful approach.

Uniform Interface - This means request from client to server is handled through a uniform interface. This can be explained as follows:

- URIs as a standard way of identify resources (/users, /products)

- Manipulation of resources through standard interface (GET, POST, PUT, DELETE) (users/1)

- Messages are self descriptive with standard http responses, caching and processing instructions.

- HATEOAS - Hypermedia as an engine of application state. Response from the API call has links which specifies how to use the API. An example may be API response contain key like “current_user_url” specifying link like “https://api.example.com/user”

Cacheable - This means that response message from the server specifies whether it’s cacheable. Caching response helps improve network & application performance.

- GET, PUT, DELETE request are idempotent and responses from these request are cacheable whereas POST changes the state on the server is not really cacheable.

- Usually server responses contain expiry time which specifies the cacheing duration.

- Etags specified in the request are checked whether the content has been changed and serve as caching signals.

- With caching, HEAD request can be enough to generate the response as client can use the cache data for response.

Layered System -  With this constraint, client cannot assume that it is directly interacting with the server. This allows API management companies to add load balancing, cacheing, logging, analytics without affecting the design of the API.

Code on Demand - It is only optional constraint through which server can extent client by transferring executable logic to client, for eg: in the form of javascript & applets.

In my next post, I will write about how these constraints help in achieving goals of REST architecture.

Integration Shift - SOAP Vs REST

I can start by saying SOAP is an older sibling of REST, however both provides a way of accessing Internet / http.

SOAP provides more of a standard way of access through WSDL whereas REST gives a simplistic approach by solely leveraging http standard.

SOAP is more of a messaging protocol that supports jms etc along with http whereas REST is http based and is more of an architectural style which forces implementation to have certain design constraints.

REST is more light weight as SOAP uses xml along with defined structure of soap envelope whereas Rest uses simple url along with json which is easily parseable as javascript object.

As a developer, I normally don’t get to decide when to choose soap vs rest, however I have my perspective.

Typically with the mobile apps, mostly apis are rest based as bandwidth is limited on the go. Mobile interfaces are mostly stateless.

Platform APIs almost always support Rest and typically provide language specific client side libraries to invoke the apis.

As SOAP has been the standard for implementing webservices for a decade, there has been extensive support for SOAP and still for asynchronous use cases, wsdl based contracts, soap may be better suited.

Any deviation / opinion as well scenario based use case feedback  / discussion is appreciated.

Example of SOAP vs REST request for retrieving user

<?xml version="1.0"?><soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope" soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding"><soap:body pb="http://www.acme.com/phonebook"><pb:GetUserDetails><pb:UserID>12345</pb:UserID></pb:GetUserDetails></soap:Body></soap:Envelope>

http://www.acme.com/phonebook/UserDetails/12345

As can been seen, in order to retrieve the details of the user, a xml request with <envelope> & <body> has to be created whereas using REST, URL is all we need.

In the next post, I will write about REST based architectural style and its constraints.

Integration Shift - To Web APIs

In this post, I want to explain why we need web apis ?

Some of the common usecase for developing APIs are mostly linked to way the enterprise & consumer data is shared & accessed.

Over the past few years, websites access through browser has been shirking and mobile access has been increasing exponentially.

Other shift is emergence of platform companies which are sharing / exposing  data to the developers who are using the APIs in Apps.

Lastly cloud has enabling enterprises to cross the firewall / network boundaries to leverage products / services over http and create composable enterprise use cases / processes.

In the next post, I would like to explain SOAP vs REST style of creating services…

Integration Shift - WebServices to Rest APIs

I have worked in an Integration space starting with WebServices which had been defacto standard for vendor neutral, technology independent way of exchanging the data & invoking back-end API.

As the Integration domain is expanding, these days REST APIs has been the emerged as a  integration approach not just on the web but also within an enterprise as an another viable option to soap/wsdl based webservcies. 

What is an Web API ? It is machine readable way of accessing the data over http in rest style approach. One way to think about this is that websites also access the same data, though the view layer is human readable, however with APIs its plain data in xml / json.

In my next post I will write about why we need APIs.....