Wednesday, 17 July 2013

AMAZON - Install and Configure Varnish Cache

Varnish is an HTTP accelerator designed for content-heavy dynamic web sites. It sits in front of your web server. Varnish caches and stores web pages in memory so that web servers don't have to create the same web page over and over again. Varnish Cache serves pages much faster than any application server; giving the website a significant speed up.
We have seen use cases with Varnish , where pages can be delivered sub-second to the browsers.
Varnish is usually suitable for applications like Online Newspapers/magazines , Online Classifieds , Media sites which has more read pages and less of write operations (Publish once - Read many scenarios).

Install the varnish server and make corresponding configuration changes.


§  Install dependants for Varnish server installation
§  Download and Install Varnish server
§  START Varnish server and check for Varnish server run status
§  Make required configurations in default.vcl file

yum install automake
yum install autoconf
yum install libtool
yum install ncurses-devel
yum install groff
yum install pcre-devel
yum install pkgconfig
tar -zxvf varnish-3.0.2.tar.gz
cd varnish-3.0.2
sh configure
make check
make install

Now Varnish server is installed, to start varnish server we need to know the IP  of this server. To get the IP of the server execute command “ifconfig” and find the IP as shown in the image below,

Once we have received the IP of the instance now build the command for Varnish server launch

varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,4G -T -a -p thread_pool_min=500 -p thread_pool_max=4000 -p thread_pool_add_delay=2

Note: Make sure you provide the correct IP address while varnish server launch

Varnish Launch Command Explanation:

Varnishd –> Command
-f –> file location of VCL file
-s –> Backend storage specification. By default the storage is “file” we change it to “malloc” so that the information will be stored in memory
-T –> Telnet listen address and port. Host is set to localhost and port is some random port e.g. 2000
-a –> HTTP listen address and port. Host is this server’s IP and listening port is 80
-p –> Set parameter for service launch. 
There are few essential performance based parameters to be set while launch they are as follows
§  thread_pool_min is the minimum number of threads for each thread pool
§  thread_pool_max is the maximum total number of threads
§  thread_pool_add_delay – Reducing the add_delay lets you create threads faster which is essential - specially at startup - to avoid filling up the queue and dropping requests

Now we should edit the default.vcl ( located at : /usr/local/etc/varnish/default.vcl ) and uncomment the backend default section.

This backend default section is the section where we will configure the host and port of the server, where varnish will send the request; if the requested content is not available in varnish cache.

Here the host is set as localhost i.e. because, we have both web server and Varnish server running in the same Server Instance. If you wish to run the varnish in one server and the web server in another separate server, then you can do so but, make sure you provide the IP of the server instance where the web server is installed in the Varnish server’s default.vcl file’s “backend default” section accordingly.

By default in Red Hat server it has Firewall settings enabled which will block our setup from responding when requested from browser i.e. only port 22 will be open since our requests come via port 80 it will not be allowed. Hence, to allow access we block Red Hat’s Firewall settings to proceed further. Also we will configure in such a way that this firewall block will be effective even on server restarts.
Instead of blocking we can add configurations in iptables so as to allow port 80 also, but this is totally not required since our setup is in AWS. The security group settings of AWS are much secured and there is no need for another security level. Also, this additional security step will cause more latency in response so it is better to block Red Hat’s Firewall settings.
COMMAND’s for executing the above mentioned are as follows,

iptables  -L
ð  Checks for Red Hat Firewall status

/etc/init.d/iptables stop
ð  Stop Red Hat firewall

chkconfig iptables off
ð   Will make sure that the service won't start when server boots up. This removes the service from the start up list

Configure boot startup for Varnish Server

For the Varnish server to start automatically when the server reboots, we need to update the “rc.local” file in the server instance.
The commands for the same is as follows,

vim /etc/rc.local

Update these lines below in the file
/usr/local/sbin/varnishd  -f /usr/local/etc/varnish/default.vcl -s malloc,4G -T -a -p thread_pool_min=500 -p thread_pool_max=4000 -p thread_pool_add_delay=2

save the changes made

No comments:

Post a Comment

Thank you so much for providing your valuable feedback. I will will look into them and update my skills & technologies accordingly.