Work with nginx deny rules and HAproxy

Greeting folks,

In this post I will explain what is needed to restrict certain IPs from accessing a web-app on nginx. The trick is to get the real-IP of the user rather then the IP of the loadbalancer.

Imagine the following setup:

Your Internal Domain directly points to your haproxy (e.g. 10.0.0.120).
Your external Domain hits the router and goes via DNAT to the external IP of the haproxy (10.0.0.121)

Since HAproxy will use the first adapter (eth0 in most cases) which is (can be) the internal IP to contact the NGINX (typo in my drawing :D) it will happen that even if you block IP 10.0.0.121 your visitors can still access the restricted sites.

On the other hand blocking your internal IP will lead to the problem that your internal client(s) can't access those sites. So we need a way to get the real IP of our visitor.

First lets look at the haproxy sample I prepared:

There is nothing special to it, except one thing: send-proxy-v2
This parameter must be added to the backend directing to nginx. It tells haproxy to use the proxy protocol when directing traffic to nginx.

In my example above it resides on the same machine as haproxy under a different port not accessible from inside and outside.

---

Now let's have a look at the nginx part of things:

The interesting parts here which shouldn't be present at your vhost config are set_real_ip_from,real_ip_header, real_ip_recursive and proxy_protocol

set_real_ip_from 10.0.0.120 - Specifies the IP addresses HAproxy uses. Basically trusted IPs which will always send correct replacement addresses to nginx.
real_ip_header proxy_protocol; - Specifies that the ip in the header should be replaced by the real IP of the user and not the loadbalancer IP
real_ip_recursive on; - Used to specify to always use untrusted IPs for replacement. Trusted are the one listed via set_real_ip_from.
proxy_protocol - Tells nginx that request are recieved via the proxy protocol.

With those parameters used you are good to go and can secure sites easily. There is an example location block called /admin where you take a look.
If you have HAproxy directly exposed to the web, then just use the external IP address instead via set_real_ip_from.

Over and Out,

Aebian