WebSphere Liberty auto scaling


Imagine a person who needs to go downtown every work day and buys a bus (instead of a small car or motorbike) to drive to and from work being the only passenger on that bus. The other 40+ seats on the bus are empty most of the time (except when relatives come to visit from out of town). In real life this is too ridiculous to be considered, yet in corporate data centers this happens all the time.

If you are lucky enough to get into a realm of solving scalability problem for your on-prem Java app (hopefully because your business is booming), chances are you are doing static JVM provisioning. For example, you configure and start hundreds of JVMs and keep them all running at all times regardless of the workload. If workload goes to 100% of its peak, then you are using resources to their full capacity (the “bus” is full). However workloads tend to come in peaks and valleys, and usually most of the time you are far from the peak load on the system. I’d argue that most systems average about 10% utilization rate compared to their total capacity. What this means is that your company paid big bucks for 100% of the hardware, networking, software, power, etc., only to use 10% of that investment. This is one of the reasons why pay per use models in the cloud are gaining momentum.

But what if you are not ready to move the workload into the cloud? What if you must run in your own datacenter behind your own firewall? One way to solve this is to utilize virtualization (PowerVM, KVM, VMware, etc.) This allows for much greater density of computing and more sharing of resources. One of the issues with virtualization is that 90% of the time it is memory bound. In other words your processors may be idle, but memory is at 100% and you are once again forced into buying new servers. Why is that?

This is where we come back to static clustering. Almost all of the deployments of WebLogic, JBoss, Tomcat, and other servers that I have seen are using this static provisioning model. Notice I did not even call it static clustering. Most of the JBoss and Tomcat deployments I have looked at are using standalone always-on servers with a simple load balancer in front to distribute the workload. WebLogic is mostly used in a true clustering environment, but still statically provisioned. Now imagine a large company that has 50 applications. Quite often these applications are deployed into their own JVM instances and not sharing the JVM (for security, compatibility, administration or other reasons). Imagine that most of those applications have peak workloads that require 100 (or more) JVMs to handle such peak. That gives us 50*100=5,000 JVMs. However at any given time only a small percentage of those applications are running at peak (perhaps even once or twice per year). Why keep all of those JVMs running? Assuming each JVM takes 2 GB of memory, it comes out to 10 TB of RAM. That is a lot of memory. Not to mention the footprint of the operating system and other supporting software. This could easily reach into 100s of terabytes of RAM. Despite the fact that RAM is relatively cheap, it still costs money, especially for server class hardware. How can you solve this memory problem – meaning serve peaks when they come (once every few hours, days or weeks), but not waste huge amount of resources?

I explained how WebSphere Application Server ND handles server consolidation (with SLA and other goodies) in this post. Today I would like to show how auto-scaling can be done with WebSphere Liberty Profile with little effort. Please note that not all of the Intelligent Management features of WAS ND are available in Liberty today – at least not yet. The current version of the Liberty Profile does provide the auto-scaling feature (see the docs here). Here is the video that shows setup and auto-scaling in action.

If you would like to build this autoscale environment yourself, here is what you do:

  1. Go to wasdev.net to download Liberty 8.5.5.6 GA with Java EE 7 support (or you can download Liberty Java EE Web Profile 7 with JDK)
  2. If you chose Liberty+JDK option in step 1, then skip to step 3. Otherwise install your favorite JDK on each of the hosts (host1, host2, etc.)
  3. Download my shell scripts for installation and configuration and unzip into the project folder
  4. Prepare four (or more) Linux hosts named host1, host2, host3, etc. and httphost (make sure they see each other on the network)
  5. Make sure that the install image of Liberty from step 1 and project folder from step 3 are mounted to the same location in all hosts
  6. Update setenv.sh and install_and_setup.sh with your own values (paths, host name, ports, etc.)
  7. Run install_and_setup.sh on each of the hosts (host1, host2, etc., but not on httphost)
  8. Finally run http_dynamic_routing_config.sh script on httphost to setup dynamic routing

You are all set. Now you can hit the apps (see video above for details).

How does it work?

The scripts I provided above not only install Liberty, but also automatically generate the proper environment variables and server.xml files for the Liberty Collective Controller and Collective Members. Here is the Collective Controller configuration:

<?xml version="1.0" encoding="UTF-8"?>
<server description="collective controller">
  <featureManager>
    <feature>jsp-2.2</feature>
    <feature>localConnector-1.0</feature>
    <feature>clusterMember-1.0</feature>
    <feature>dynamicRouting-1.0</feature>
    <feature>scalingController-1.0</feature>
    <feature>adminCenter-1.0</feature>
  </featureManager>

  <!-- Collective controller feature is included below -->
  <include location="collective-create-include.xml" />

  <httpEndpoint host="*" httpPort="${env.CONTROLLER_HTTP_PORT}"
httpsPort="${env.CONTROLLER_PORT}" id="defaultHttpEndpoint" />

  <httpEndpoint host="*" httpsPort="${env.DYNAMIC_ROUTING_PORT}"
id="dynamicRoutingEndpoint" />

  <basicRegistry id="basic" realm="ibm/api">
    <user name="${env.ADMIN_USER}" password="${env.ADMIN_PASSWORD}" />
  </basicRegistry>

  <administrator-role>
    <user>${env.ADMIN_USER}</user>
  </administrator-role>

  <scalingDefinitions>
    <defaultScalingPolicy enabled="true" min="1" max="30" />
    <scalingPolicy id="myScalingPolicy" enabled="true" min="1" max="-1">
      <bind clusters="${env.CLUSTER_PREFIX}*" />
      <metric name="cpu" min="30" max="80" />
      <in amount="1" units="instance" minInterval="30s" />
      <out amount="1" units="instance" minInterval="30s" />
    </scalingPolicy>
  </scalingDefinitions>
</server>

In the snippet above, note how scaling definitions are defined based on the average CPU load measured every 30 seconds. You can set scaling options based on host memory or Java heap. A more detailed discussion of the Liberty auto-scaling can be found in this InterConnect2015 presentation:

Some customers build their own scripts and frameworks for quick provisioning of resources. This can (and has been) done, but requires additional effort. Here is an interesting video that shows how WalMart uses WebSphere on a mainframe.

What about the cloud?

In a PaaS environment – if you to run your Liberty Java or Node.js applications on IBM BlueMix CloudFoundry cloud, you can take advantage of auto-scale capabilities natively provided by BlueMix.

In an IaaS cloud (such as SoftLayer, Amazon or Azure) – since Liberty auto-scaling does not use multi-cast, you can use auto-scaling by using the approach I outlined above. In my demo I used a static pre-install of Liberty servers on hosts. As you have seen it takes only a few seconds to install and start new instance of the Liberty Collective Member. This means you can quickly provision new hosts in a highly dynamic IaaS cloud environment where new hosts come and go often. However unlike BlueMix PaaS it will require additional scripting to be added to the process of provisioning of the new IaaS hosts.

PS. August 2016: Please see Liberty Docker Autoscale demo and script.



Categories: Technology

Tags: , , , , , , , , ,

8 replies

  1. Hi, I tried to download your scripts and your files are not in onedrive any more

    Like

  2. Considering that scaling policies can only be tied to heap, CPU, or memory thresholds this argument is ho hum at best. Not to mention that regardless of how well you can tune your dynamic sizing you end up licensing for maximum capacity. So you won’t save a dime on software licensing and support.

    And BTW, to build on your bus analogy, If you think that Red Hat JBoss EAP clusters are not used in production, you missed the reality bus.

    Like

    • Rich, I agree that scaling policies in the current release of Liberty are not quite as powerful as in full WAS ND profile, but it will be quickly improved.

      To your point on the licensing – yes, you do have to license for peak capacity if your workload peaks during the day. However if your workload peaks on a monthly basis you could buy licenses+support just for that month – something you can’t do with JBoss. However even more important that one saves on administration and has more efficient use of resources – this can potentially provide bigger savings than license itself.

      As for the reality bus – I guess you and I are in different realities. 90+% of customers I and my colleagues talked to (including those at Red Hat conferences) do NOT use JBoss clustering. I am sure somebody somewhere does make use of JBoss clustering…

      Like

Trackbacks

  1. Application auto-scaling in the BlueMix vs. OpenShift | WhyWebSphere Blog
  2. Dynamic clustering (auto scaling) in WebSphere, WebLogic, JBoss and Tomcat | WhyWebSphere Blog
  3. Which app server is best for consolidating unpredictable Java workloads? – IBM Advantage Blog
  4. WebSphere Liberty autoscale clustering with Docker – IBM Advantage Blog

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: