Cloud: Cluster-Wide Configuration Using HTTP

Configuration Sharing: Cluster-Wide Configuration Using HTTP

In most cases, Resin can handle HTTP(S) URLs interchangeably with file paths. This provides the handy ability to pull configuration files from a central configuration repository over HTTP. Although it adds a possible single point of failure (the configuration server), for large clustered deployments the convenience of not having to synchronize configuration files to each machine can not be understated.

Note: Resin can also use cloud repository based configuration, which is stored in the Triad and shared between cluster members using Resin's internal .git implementation. While cloud configuration is slightly harder to work with than HTTP, it has the added advantage of having no SPOF, and requiring no additional HTTP server. An administrator can make the choice which works best for their architecture, or both can be used at the same time if desired.

Setup an HTTP Server

A single HTTP server will serve as your configuration repository for the entire cluster, or even multiple clusters/tiers if desired. You can use Resin for this also, but this server instance should have it's own local configuration and be entirely separate from other clusters.

For my example I've setup Apache on port 80 at "http://molson".

Setup Directory Structures

This can be as simple or complex as you desire. For this example, I've created a "resin" directory in my webserver's document root. Under that there is a directory for each environment: "dev", "qa", and "prod". Under each of these, there is a directory for each tier in the env: "web" and "app".

/Library/WebServer/Documents
  |-resin
  |---dev
  |-----app
  |-----web
  |---prod
  |-----app
  |-----web
  |---qa
  |-----app
  |-----web

This makes the URL to the app cluster in my dev environment "http://molson/resin/dev/app".

Move Configuration Files

Move the following files from the Resin conf directory to the appropriate directory in your webserver document root, in my case "resin/dev/app"

• admin-users.xml
• cluster-default.xml
• health.xml
• resin.properties
• resin.xml

So now I can directly reference resin.xml at "http://molson/resin/dev/app/resin.xml"

You should notice that Resin XML configuration often references "${__DIR__}" for imports. This EL variable always refers to the current directory where the XML resides, and it this still works with HTTP paths. Thus with resin.xml loaded from http://molson/resin/dev/app/resin.xml, ${__DIR__} will refer to "http://molson/resin/dev/app/", making loading the other files automatic.

Some of these files, or parts of them, the triad server addresses in particular, should be shared between clusters in the same environment. I'll discuss that below.

Local resin.xml

Now we add a basic local conf/resin.xml that simple imports resin.xml from the HTTP configuration repository:

<resin xmlns="http://caucho.com/ns/resin"
      xmlns:resin="urn:java:com.caucho.resin">
  
 <resin:import path="http://molson/resin/dev/app/resin.xml"/>
 
</resin>

Add Environment Variables

The local resin.xml above has the server url hardcoded. It would be more convenient to define environment variables on each machine, or on the startup command line, so that our resin.xml is not environment specific.

Any environment variables can be referenced directly in resin.xml with no special setup.

For my example on OSX/Linux, I'll export 3 environment variables that defined which environment this machine is a member of. This would most likely go in your ~/.bash_profile or /etc/profile file.

export RESIN_CONFIG_URL=http://molson/resin
export RESIN_ENV=dev
export RESIN_CLUSTER=app

Local resin.xml with variables

Now we can modify the local resin.xml to

<resin xmlns="http://caucho.com/ns/resin"
      xmlns:resin="urn:java:com.caucho.resin">
 
 <resin:import path="${RESIN_CONFIG_URL}/${RESIN_ENV}/${RESIN_CLUSTER}/resin.xml"/>
 
</resin>

Note: this will force all servers on the machine into the same environment and cluster. If you want to run multiple servers on the same machine for more than one cluster, you also have the option of setting variables on the command-line during startup, as show below.

$> RESIN_CLUSTER=app resinctl start

Environment Configuration Sharing

As I mentioned above, we can further eliminate copies of configuration files by sharing appropriate configuration between all clusters in the same environment.

These files make sense to share:

• admin-users.xml
• cluster-default.xml

So I'll move these up 1 directory in my webserver document root to "resin/dev".

And then modify resin.xml in resin/dev/app as such:

<resin:import path="${__DIR__}/../cluster-default.xml"/>
 ...
<resin:import path="${__DIR__}/../admin-users.xml" optional="true"/>

Finally, the web-tier must have a list of the app-tier servers for load balancing. Therefore it makes sense to create a separate file devoted to the server list alone. Comment out the servers from resin.properties, and create a file called servers.properties in the webserver root at resin/dev/servers.properties

web_servers      : 192.168.1.11:6810
app_servers      : 192.168.1.2:6800 192.168.1.12:6800 192.168.1.7:6800

Add add a reference to it in your resin.xml:

<resin:properties path="${__DIR__}/../servers.properties" optional="true"/>