Archive for September, 2021


Set up Apache Varnish for a Laravel API

I needed to cache a Laravel API due to high use.  I needed it cached before it touched PHP or MySQL, and the server I was on had to stay on Apache.  I looked at mod_cache too, but that’s a lot more suited for static websites, while this was a dynamic API.  So I decided to set up Varnish.  For this tutorial, I was using Varnish 6.2. It will store responses in memory for as long as dictated by the Cache-Control header.

This is how I went about installing and configuring Varnish. This is assuming you already have Apache set up. Varnish only works on HTTP so you first have to set it up to work through the 80 port and then use a reverse proxy so Apache handles the SSL and then passes the request to Varnish.

So first, you have to change Apache from the 80 port to 8080. Essentially you just have to change 80 to 8080 in ports.conf and your VHosts file, but here’s a simple command that’ll do it for you

1
2
sudo sed -i -e 's/80/8080/g' ports.conf 
sudo sed -i -e 's/80/8080/g' sites-available/exampl.com.conf

Then restart Apache

1
2
apachectl configtest
sudo /etc/init.d/apache2 restart

Now you’re ready to set up Varnish on port 80.  First, install and enable it

1
2
3
sudo apt install -y varnish
sudo systemctl start varnish
sudo systemctl enable varnish

Then adjust Varnish to use port 80, and increase its ram.  I’ve seen it recommended to set it to use 75% of your server’s total memory, but I set it for a more conservative 30%.

1
2
cd /etc/default/
sudo nano varnish

change uncommented

1
2
DAEMON_OPTS="-a :6081 \
     -s malloc,256m"

to

1
2
DAEMON_OPTS="-a :80 \
     -s malloc,5G"

Do essentially the same change here

1
2
cd /lib/systemd/system/
sudo nano varnish.service

change this line

1
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

Just change the 6081 to 80 and 256 to 5G

Then reload the setting and restart Varnish

1
2
sudo systemctl daemon-reload 
sudo systemctl restart varnish

Now it should be all set on port 80 and caching your HTTP content! You can make sure Varnish is running with this command

1
sudo netstat -plntu

Next we gotta make it work on SSL, first install the neccesary Apache modules

1
2
sudo a2enmod proxy_balancer 
sudo a2enmod proxy_http

In the VHost, comment out the document root, and add this

1
2
3
 ProxyPreserveHost On
 ProxyPass / http://127.0.0.1:80/
 ProxyPassReverse / http://127.0.0.1:80/

This will make Apache pass the request to Varnish on port 80. And Varnish running on HTTPS is all set!

But, to get it to cache the API requests, I had to make a few changes to the Varnish configuration.

1
sudo nano /etc/varnish/default.vcl

First, add this to set a Hit/Miss header so you can easily debug the configuration.

1
2
3
4
5
6
7
8
sub vcl_deliver { #just need to add it to the existing enclosure, don't make a new one
    if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed
      set resp.http.X-Cache = "HIT";
    } else {
      set resp.http.X-Cache = "MISS";
    }
    return (deliver);
  }

Add this inside of vcl_rev to ignore if the request has cookies. Varnish by default will not cache requests with cookies. My API used keys so this wasn’t an issue.

1
2
unset req.http.cookie; 
unset req.http.Accept-Encoding;

In vcl_backend_responce add this to unset the laravel cookie session header

1
unset beresp.http.Set-Cookie;

Another adjustment to allow purging of individual cached pages via the PURGE method works by adding this inside vcl_recv

1
2
3
4
#allow to purge saved URLs
if (req.method == "PURGE") {
return (purge);
}

That will allow you to remove any cached page using this command

1
curl -v -k -X PURGE https://url-to-purge

And to setup the headers so it’ll only cache in Varnish, and not in the browser they need to be:

1
2
Cache-Control:s-maxage=60,max-age=0,private,must-revalidate,no-cache,no-store
Surrogate-Control:*

s-maxage will set the max-age only for the server, and surrogate-control will make Varnish ignore no-cache/no-store

And, in case you haven’t done it yet, you can add a cert to your secure site using these commands

1
2
sudo certbot --apache -d example.com
sudo /etc/init.d/apache2 restart

To control the length of time that Varnish caches a page for all you need to do is set its Cache-Control header.  That essentially needs to be set to

max-age=<seconds>, public

and Varnish will do the rest.

And for me that was it!  All good!

Sources:

https://www.howtoforge.com/how-to-install-and-configure-varnish-with-apache-on-ubuntu-1804/
https://bash-prompt.net/guides/apache-varnish/
https://stackoverflow.com/questions/65013805/cant-purge-entire-domain-in-varnish-but-can-purge-individual-pages-do-i-have
https://stackoverflow.com/questions/48988162/disable-cache-on-browser-without-disabling-it-on-varnish

Permalink » No comments