Jira is a stateful application, it requires persistent storage for both database and the shared home folder. And actually it is more complicated than it looks, e.g local index file, index files for some addons, nodes records etc.
But it does not mean that you cannot run Jira in a dynamic environment, like AWS Auto Scaling Group. Obviously auto scaling group brings some benefits like multiple AZ HA, scalable cluster, rolling updates etc. Please bear in mind that Jira is a not Cloud native application, so the dynamic environment is only to a certain degree. We still need to keep it as static as possible even when using Auto Scaling Group.
We build Jira cluster with Auto Scaling Group (for Jira cluster nodes), EFS (for shared home folder) and RDS (for Jira database). And we run intensive performance testings to decide the right instance type for the Jira cluster, so it can handle heavy loads without scaling up even when with one less node. The only time we replace the Jira nodes are
- Linux Patching
- Jira Upgrade
For Linux patching, we simply bake a a new AMI with latest patches and then rolling update the existing cluster node one by one. We call it hot deployment, as it does not require any downtime.
For Jira upgrade, we bake a new AMI with the new Jira version, and scale down the current cluster to zero node, then scale up to only one node which uses the new AMI. This node will perform upgrade process. Once it is completed, we scale up the cluster with more nodes. We call this cold deployment, as it requires downtime. (Note: Jira supports zero downtime upgrade now, but it is only for minor version upgrade which does not require big changes in the database schema.)
Does the above make sense so far? It sounds not too bad, isn’t it? The evils are in the details, there are actually quite a few things that need to be done to ensure Jira can work smoothly in this dynamic environment. For example
- How to get index files to the local home folder? especially for the first node in the cluster.
- How to remove the stale node automatically?
- How to terminate Jira instance smoothly to ensure the node status is correct in database?
In the next a few posts, I will discuss those in details.
Jackie Chen's IT Workshop 