Intelligent traffic shaping in WAS, WLS, JBoss and Tomcat


Traffic shaping is the ability of the server to intelligently handle user requests according to the rule set defined by the system administrator, instead of having a system where every request is treated equal and is processed with the same priority. When system is loaded lightly there is usually no need to perform traffic shaping. However, during the times of heavy loads when the number of requests for multiple applications exceeds the processing capacity of the system, traffic shaping can be very helpful and makes sure that requests of different priorities are serviced in accordance with the Service Level Agreements (SLA). Good example is a money making transaction, such as a stock buy or sell request, vs. the non-money making transactions, such as a stock “price quote” request. In addition to the Stock Trading application you may have HR, Tax and other applications running on your servers and requests for those can also be de-prioritized during the times of heavy trading.

Any application server, or any HTTP server for that matter can work with a 3rd party hardware load balancer (F5 is one example) or 3rd party software based load balancer (nginx is one example). However 3rd party load balancers usually lack features (discussed below) and introduce additional cost and vendors. In this article I will not discuss the use of 3rd party load balancers and will focus on advanced capabilities of the “native” load balancers provided by the app server vendors, yet some of those options still will cost you money, hence I am not sure if the cost of those “native” products is worth the features they bring.

There are three different ways to perform traffic shaping with WAS ND:

  • The Java based On Demand Router (ODR has been stabilized) – bundled with WAS ND free of charge (has been “stabilized” by IBM, but still supported);
  • IBM HTTP Server with the ODR enabled Web Plugin – bundled with WAS ND free of charge;
  • IBM DataPower with Application Optimization add-on module – sold separately;
  • 3rd party hw or sw based load balancers.

Oracle provides two options for managing the incoming traffic of requests:

Red Hat options for JBoss EAP:

Apache Tomcat options:

  • mod_jk plugin for the Apache HTTP server;
  • some folks have made mod_cluster work with Tomcat, but is not directly supported, hence not discussed here;
  • multitude of 3rd party modules for Apache HTTP server (not discussed here since they are not officially endorsed);
  • 3rd party hw or sw based load balancers.

With the IBM options listed above, you can define performance goals and bind them to specific subsets of the incoming traffic. The WAS ND runtime is able to support business goals in times of high load by making smart decisions about the work coming into the application servers. Not all work in your configuration is created equal. The WAS ND is able to support this concept by forwarding different flows of requests on for execution more or less quickly to achieve the best balanced result. WAS ND has the ability to prioritize requests based on administrator-defined rules to adjust server response times according to SLA and application priority. These rules can use information about the protocol, application version, URI, cookie, client IP, time of day, response timing, form data, and many other system, session and request parameters. The Java based ODR, which is part of WAS ND, can throttle and prioritize HTTP and SIP requests, while JMS and IIOP requests are prioritized without ODR – directly by the runtime engine. JMS messages can be classified by destination, that is, topic or queue. Service policies can be defined to prioritize messages differently based on the destination name. The WAS ND traffic shaping can be applied to non-IBM products and works on all major software and hardware platforms. For example, WAS ND can perform traffic shaping to improve the quality of service for Node.js, PHP servers, .NET, Apache Tomcat, JBoss, Geronimo, WebLogic Server and other web and application servers communicating over HTTP or HTTPS protocols.

Request prioritization and routing are not provided if you do not buy Oracle Traffic Director (OTD). WebLogic Server can do limited prioritization of HTTP requests within a boundary of a single application server JVM. WebLogic has no concept of JMS or IIOP traffic shaping. In order to get the HTTP traffic shaping function similar to the WAS ND ODR component, Oracle customers need to purchase and use OTD in conjunction with the Oracle Engineered Systems, such as Exalogic. As mentioned above, IBM traffic shaping works on all major platforms and is built into WAS ND product, therefore no additional software or hardware is necessary, unless you choose to use DataPower for added functionality as shown in the table below.

WAS ND classic (1) WAS ND Liberty (2) WAS ND + Data Power AO (8) Oracle WLS EE Traffic Director & Exalogic (7) JBoss EAP + EWP Apache HTTP + Tomcat
High performance some some Yes some some some some
WLM across static app server clusters Yes Yes Yes Yes Yes Yes Yes
SSL termination and HTTP compression Yes Yes Yes Yes Yes Yes Yes
Auto updates of configuration when cluster or app change Yes Yes Yes Yes Yes Yes no(9)
JVM maintenance mode Yes Yes Yes Yes Yes no no
Node (or host) maintenance mode Yes Yes Yes Yes Yes no no
Application edition-aware routing Yes no Yes Yes Yes no no
Dynamic clusters auto-grow or shrink based on workload Yes Yes Yes no no no no
Health policy support Yes Yes Yes no no no no
Auto-adjusts server weights based on resource use Yes no Yes no Yes no no
Traffic shaping and SLA enforcement for HTTP Yes no Yes no Yes no no
CPU and heap overload protection Yes no Yes no no no no
Support traffic shaping for 3rd party servers Yes no Yes no no no no
Custom rules for request routing (URI, IP address, etc.) Yes no Yes no Yes no no
Request rate limiting no no Yes no some no no
Number of client connections limiting no no Yes no Yes no no
Content based routing no no Yes no some no no
Protect against XML and SQL injection attacks no no Yes no no no no
XML processing (parsing, transformation, validation, etc.) no no Yes no no no no
Custom advisors no no Yes no no no no
DMZ ready Yes(5) Yes Yes no no Yes Yes
Static file serving and in-memory and disk page caching Yes(3) Yes(3) no no4 Yes Yes Yes
Replace hardware based load balancer(s) Yes(6) Yes(6) Yes no some no no
Traffic shaping and SLA enforcement for IIOP and JMS Yes no no no no no no
HTTP session rebalancing Yes no no no no no no

Summary

  • IBM provides considerable capabilities bundled with WAS ND free of charge and IBM traffic shaping can be applied to non-IBM back end products.
  • Oracle options provide less capabilities, yet cost extra money and only work with Oracle software and hardware.
  • Red Hat JBoss provides very limited capabilities for traffic shaping and cost extra (purchase JBoss EWP in addition to JBoss EAP).
  • Apache Tomcat is free and provides basic HTTP server connector (on top of the Apache HTTP server itself) and not much of traffic shaping capabilities. However one interesting aspect of using Apache HTTP server is that there are hundreds of 3rd party modules available, including ones for request rate limiting, request filtering, dynamic configuration updates, and about a hundred other Apache modules. Not all of these are relevant in the context of this discussion, nd not all of them are equally robust to the commercial offerings, howerer feel free to investigate if you feel adventurous.

 

Notes:

  1. This configuration is assumed to use Java ODR;
  2. This configuration is assumed to use IBM HTTP Server (IHS) with the ODR enabled Web Plugin;
  3. WAS ND (classic + Liberty) ships with IBM HTTP Server;
  4. Available in Oracle Web Tier, which is a separately chargeable product;
  5. WAS ND Java ODR is not DMZ ready, however WAS ND HTTP plugin + ODR enabled plugin can be used in DMZ. Alternatively one could put Edge Components (free with WAS ND) into DMZ in front of ODR;
  6. WAS ND ships with DMZ ready Edge Components at no charge, which can be installed in front of IHS or ODR;
  7. Oracle Traffic Director documentation;
  8. See DataPower rate limiter and other documentation;
  9. Apache Tomcat does not directly support dynamic updates of clusters, however 3rd party packages, such as JBoss mod_cluster can support this scenario in certain cases.

 



Categories: Technology

Tags: , , , , , , , , , ,

2 replies

Trackbacks

  1. Intelligent traffic shaping in WAS, WLS, JBoss and Tomcat | Jason Krohn's Tech Tips

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: