The combination of Tomcat, Apache and mod_jk can be a good choice on delivering a scalable and robust solution for a simple JSP/Servlet based web application.

Scalability and HTTP session failover can be achieved in a simple manner with a straightforward lightweight servlet container like Tomcat, which offers cluster functionalities.

Load Balancing is achieved through the mod_jk apache module, which distributes incoming requests with a simple weighed-round-robin mechanism through the AJP protocol.

(mod_jk seems to be the best supported and most widespread Tomcat – Apache connector)

In order to offer HTTP session failover (avoidance of session loss in case of failure of the Tomcat instance holding the session), session replication can be used.

Tomcat can share sessions through a database, a file or through in-memory replication.

To get an idea of the overall architecture, you will end up with something like this:

Tomcat uses a very simple clustering mechanism:

Clustering membership is established using multicast pings. Once a multicast ping has been received, the member is added to the cluster. Upon the next replication request, the sending instance will use the host and port info and establish a TCP socket. Using this socket it sends over the serialized data.

It is possible to have an initial starting point for this architecture in no time, and the following is a very brief proof of concept with Apache HTTP Server 2.2.11, mod_jk 1.2.28,Apache Tomcat 6.0.18 running on Mac OS X 10.5.6 (Darwin 9.6.0).

In this example, one Apache HTTP Server will be load balancing http requests between two Apache Tomcat instances through the mod_jk module.

(depending on your user rights, some sudo’ing and chmod’ing could be necessary)

1. JDK

Every version of Mac OS X comes with Java out of the box, and new Java releases are made available through Software Update.

Java is available as a Mac OS X framework , and all installed versions can be found at /System/Library/Frameworks/JavaVM.framework/Versions (Mac OS X 10.5) and are managed through the Java Preferences.

In Mac OS X, the JAVA_HOME environment variable should always be set to /Library/Java/Home.

This can be done by the setenv (csh) or export (bash) command, and can be run on startup for a specific user through the PropertyListEditor tool.

export JAVA_HOME=/Library/Java/Home

Java should already be part of the PATH environment variable, if not:

export PATH=${PATH}:${JAVA_HOME}/bin

2. Xcode

Xcode installation is an easy way to have all the tools needed for Tomcat and Apache compilation.

3. Apache Ant

Apache Ant is needed in order to build Tomcat, and can be installed first by downloading the source and then simply:

cd < directory_containing_Ant_distribution >
sh build.sh -Ddist.dir=< directory_containing_Ant_distribution > dist

This will create the binaries in the < directory_containing_Ant_distribution >/bin directory.

(I was required to additionally download the junit jar file and place it in < directory_containing_Ant_distribution >/lib/optional for compilation)

Remember to set up the environment variable:

export ANT_HOME=< ant_home >

and setup the path:

export PATH=${PATH}:${ANT_HOME}/bin

4. Apache Tomcat

After downloading the Tomcat distribution, simply:

    cd < directory_containing_Tomcat_distribution >
    ant download
    ant

After this, the binaries can be found at < directory_containing_Tomcat_distribution >/output/build and can be copied to a tomcat1 and a tomcat2 directory.

After chmod’ing the bin/*.sh files to executables and creating the < tomcat_home >/log directory, the conf/server.xml configuration file has to be adjusted for both tomcat1 and tomcat2 :

The shutdown port must be different for both tomcats:

< Server port="8005" shutdown="SHUTDOWN" >

The HTTP connector can be commented out, as it will be Apache HTTP Server who will handle HTTP traffic:

< !--< Connector port="8081" protocol="HTTP/1.1" connectionTimeout="20000"      redirectPort="8443" / > -- >

The AJP connector port must be different for each tomcat:

    < Connector port="8009" protocol="AJP/1.3" redirectPort="8443" / >

The attribut jvmRoute must be added to the Engine element, and must be different for each tomcat:

    < Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1" >

Uncomment the cluster element:

< Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/ >

Finally, a small test web application can be pasted into the webapps directory of each tomcat.

The web application contains a jsp that shows the session id and the tomcat instance that it is running on, and should be modified for each tomcat:

<%@page contentType="text/html"%> <html> <head>
<title>Session Example: session.jsp</title></head>
<body> TOMCAT 1: This is the session id <STRONG><%= session.getId()%></STRONG> <BR> <% if ( session.isNew() )
 { %>    This is a new session ! <BR> <%} else { %>    This is an
existing session<BR> <%}%> </body> </html>

5. Apache HTTP Server

Mac OS X has a built in Apache HTTP Server, which can be launched by /usr/sbin/apachectl and configured through /etc/apache/httpd.conf.

In case you do not want to mess around with the built in server (probably a good idea), it is quite easy to install Apache HTTP Server.

Simply download the unix source and issue the following commands:

     $ ./configure --prefix=PREFIX
     $ make
     $ make install
     $ PREFIX/bin/apachectl start

(where PREFIX is the directory where Apache will be installed)

When launching bin/apachectl, remember to use the complete path, otherwise the Mac OS built in Apache HTTP Server will be launched.

6. mod_jk

In order to compile and install mod_jk, download the source and then issue the following commands from within the downloaded distribution:

./configure –with-apxs=<apache_home>/bin/apxs
 cp ./apache-2.0/mod_jk.so <apache_home>/modules

Once installed, we can modify the Apache HTTP Server httpd.conf file and add the following lines:

LoadModule    jk_module  modules/mod_jk.so
JkWorkersFile "/conf/workers.properties"
JkLogFile     "/logs/mod_jk.log"
JkLogLevel    info
JkMount /test/* loadbalancer

This lines basically load the mod_jk module, configure the log, and redirect all incoming HTTP requests for /test/* URLs to the tomcat instance “loadbalancer”, which is defined in the workers.properties file.

The workers.properties file can be created at < apache_home >/conf:

(The tomcat instance identifiers in the worker.list property must be the same as specified in the jvmRoute attribute in the tomcat configuration files)

worker.list=tomcat1,tomcat2,loadbalancer

worker.tomcat1.port=8009
worker.tomcat1.host=localhost
worker.tomcat1.type=ajp13
worker.tomcat1.lbfactor=1

worker.tomcat2.port=8010
worker.tomcat2.host=localhost
worker.tomcat2.type=ajp13
worker.tomcat2.lbfactor=1

worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=tomcat1,tomcat2
worker.loadbalancer.sticky_session=0

The workers.properties defined the different tomcat instances available, and one special worker called “loadbalancer”, which is the group of tomcat instances that will be used for load balancing.

Another important property is the sticky session setting. If set to 0 (false) sessions will not stay on one tomcat instance once initiated, but rather be balanced.

7. Testing

Once everything is up and running (the two tomcat instances and apache), a browser pointing to http://localhost/test/test.jsp will show the test web application displaying the session id and which tomcat is the test.jsp running on.

The first time we access the page, a new session is created.

If the page is reloaded, the session is conserved (same session id) and the request is redirected to the second tomcat.

For each reload, the request is redirected in a round robin manner.

If we stop one of the tomcat instances, the session is in effect conserved, and the requests are redirected to the sole tomcat instance.

If the instance is restarted, it will be once again part of the cluster, and it will receive HTTP requests.

Conclusions

Obviously, the example explained here is just the tip of the iceberg, and does not dive into the topics that should be discussed when considering Tomcat clustering.

Some open questions that should probably be answered are:

Is in-memory replication adequate for your scenario?

Depending on the size of your session data, other storage can be considered, such as file based or database persistence.

Is your web application distributable? If not, what is the cost of adapting the code?

Simply adding the distributable element to the web application deployment descriptor will not make it magically and instantly ready for clustering.

It simply indicates that the web application is programmed appropriately to be deployed into a distributed servlet container: it is up to the software engineers to guarantee that the code will behave correctly in a distributable scenario.

The Java Servlet specifications (section SRV.7.7.2 Distributed Environments) and this great article discuss some of the main points to check:

• Instance variables and static variables should not be used to store application state
• ServletContext should not be used to store application state
• Any object placed into an HttpSession should be Serializable
• Consider that files may not exist on all backend machines; make sure all accessed files are replicated across all backend machines
• Consider that synchronization is not global and works only for the local JVM

Should all-to-all or backup only replication be used?

Tomcat can perform an all-to-all replication of session or perform backup replication to only one node. Depending on the number of nodes, one or the other will be more efficient.

Is there anything else to be considered?

Although the basic principles for Tomcat clustering are quite simple, it seems clear that there is no unique setup that will fit in any scenario: a lot of research and reading, thorough testing, tuning and fiddling will probably have to be done before finding the right combination that will suit your architecture and deployment needs.

No related posts.

Tags: Apache HTTP Server, Apache Tomcat, Load Balancing, mod_jk, Session Failover, Session Replication, Web Applications