What is this language?!

Yeah, I usually write my blog posts in French but this time as I think the people who can be interested by this topic are more to be English readers than French readers, it's in English.

What's the problem?

nginx for several reasons has been removed from base between 5.6 and 5.7, and like many people I prefer to use software present in base rather than in ports/package. So the replacement is httpd. My problem, mmmh I mean my first problem (:p) was that httpd doesn't support SNI and as I have two certificates for all my vhost, it couldn't work.

The solution

The solution I wanted was something which would terminate the TLS connection then forward it to the httpd.

relayd !!!

As httpd code base comes for a big part from relayd, if httpd doesn't support it yet, you can guess that relayd doesn't neither. As for httpd, support is planned though.

nginx in reverse proxy

If I don't want to use it anymore for the http daemon part, it's mainly to not to have it on my system.

haproxy

I've heard many times that it was a cool piece of software so I wanted to use it for a while but I never had any reason to use it.

I read that haproxy works fine as a TLS termination proxy so this is the one I chose.

Use haproxy

Or try to

I installed haproxy on my system, tried to have a config parsable but it didn't want the keyword ssl. After looking at the net/haproxy/Makefile, I saw that haproxy wasn't compiled with libressl.

Patch the port

jca@ gave me some advice to make a diff I could post to ports@ to add tls support. By the time gonzalo@ (who isn't marked as maintainer but who has been updating haproxy for a while) sent another diff that took in account my diff and updated haproxy to 1.5.11 the latest stable version.

It has been commited in -current though I backported the diff to 5.7 -stable and it works fine.

Write haproxy.cfg

General part

There are a couple of general things fine for the two use-cases I'll talk about. They're mainly from the haproxy.cfg that comes with the port.

global
        log 127.0.0.1   local0 debug
        maxconn 1024
        chroot /var/haproxy
        uid 604
        gid 604
        daemon
        pidfile /var/run/haproxy.pid
        tune.ssl.default-dh-param 2048
        ssl-default-bind-options no-sslv3 no-tls-tickets
        ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
        ssl-default-server-options no-sslv3 no-tls-tickets
        ssl-default-server-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS

defaults
        log     global
        mode    http
        option  httplog
        log-format %ci:%cp\ %ft\ %b/%s\ %ST\ %hr\ %hs\ %{+Q}r
        option  dontlognull
        option  redispatch
        retries 3
        maxconn 2000

I added some ssl-default options with some cipherlists I found somewhere on the web. I tested it with ssllabs and got an A (if I ignore my self-signed cert otherwise it's a T yeah). I have no doubt that you can find something better :)

I tweaked the log-format so it's not too verbose.

First use case (basic)

My first use case was I have httpd which will answer to http request and haproxy which will terminate the TLS connection if needed. Before that I have sslh which listens to the 443 to enable me ssh my server on port 443 while running an httpd server on that port too.

Here's a schema:


      80    +                        +------------+       +----------+
            |                     80 |            |  8080 |          |
   +--------------------------------->            +------->  httpd   |
            |                        |  haproxy   |       |          |
            |  +--------------+      |            |       |          |
            |  |              | 8443 |            |       +----------+
   +----------->     sslh     +------>            |                   
            |  |              |      |            |                   
      443   +  +-----+--------+      +------------+                   
                     |                                                
                  22 |                                                
                +----v-----+                                          
                |          |                                          
                |  sshd    |                                          
                |          |                                          
                |          |                                          
                +----------+                                          

In addition to the global and default section, I had:

frontend http
        bind *:80
        bind 2001:910:1322:1:dead:beef:cafe:1:80
        http-request redirect scheme https if { hdr(host) -i somesikritdomain.chown.me } !{ ssl_fc }
        default_backend httpd

frontend https
        bind *:8443 ssl crt /etc/ssl/pki/server-haproxy.pem crt /etc/ssl/pki/wild-haproxy.pem accept-proxy
        rspadd Strict-Transport-Security:\ max-age=31536000
        default_backend httpd

backend httpd
        option forwardfor
        option httpchk GET /check/index.html HTTP/1.0
        server www 127.0.0.1:8080 check

Haproxy listens both on http port 80 and https port 8443 (because that's the port I used with sslh, 443 is perfectly fine if there's nothing between) and then it forwards the traffic to httpd.

I also added some HSTS header for https. For a specific domain I redirect automatically to https (because the page has an authentication method and I don't want my password to be sent on clear).

To get the server.pem it's just cat server.crt server.key > server.pem.

Second use case

While I was setting up the first use case, someone pasted a link on irc from someone's blog saying that he used haproxy as a replacement for sslh. So let's try to remove another package from the server.

What I wanted to achieve this time:


   80  +           +------------+       +----------+
       |           |            |  8080 |          |
+------------------>            +------->  httpd   |
       |           |  haproxy   |       |          |
       |           |            |       |          |
       |           |            |       +----------+
+------------------>            |                   
       |           |            |                   
  443  +           +------+-----+                   
                          |                         
                       22 |                         
                   +------+------+                  
                   |             |                  
                   |    sshd     |                  
                   |             |                  
                   +-------------+    

So in addition to the global and default section, I added:

listen front
        bind *:443
        bind 2001:910:1322:1:dead:beef:cafe:1:443
        mode tcp
        option tcplog
        tcp-request inspect-delay 2s
        acl is_ssl req.ssl_ver gt 0
        tcp-request content accept if is_ssl
        use_backend loop_ssl if is_ssl
        server local 127.0.0.1:22

backend loop_ssl
        mode tcp
        server ssl 127.0.0.1:1443 send-proxy

frontend http
        bind *:80
        bind 2001:910:1322:1:dead:beef:cafe:1:80
        http-request redirect scheme https if { hdr(host) -i somesikritdomain.chown.me } !{ ssl_fc }
        default_backend httpd

frontend https
        bind 127.0.0.1:1443 ssl crt /etc/ssl/pki/server-haproxy.pem crt /etc/ssl/pki/wild-haproxy.pem accept-proxy
        rspadd Strict-Transport-Security:\ max-age=31536000
        default_backend httpd

backend httpd
        option forwardfor
        option httpchk GET /check/index.html HTTP/1.0
        server www 127.0.0.1:8080 check

A listen section is equal to a duo backend/frontend. The listen section here listens for connections and then checks if it's a TLS one. If it is, it will send it to the backend loop_ssl. If it's not a TLS connection, then it assumes it's an SSH one and forward it to sshd.

From the backend loop_ssl we forward it to the frontend https with the "PROXY" protocol (that's what the two keyworkds send-proxy and accept-proxy are there for). The backend loop_ssl is here because we can't go directly from a listen to a frontend, it should first go to a backend, seems logical, doesn't it?

What about the check?

Of course, they're not mandatory but I prefer to have some.

There's two type of check: tcp and http. There's a bug in httpd of 5.7 which will make it segfault if the check is tcp. I reported it and it has been fixed so if you plan to use tcp check, you should backport this commit. Edit: an errata was finally released

Since I had this problem, I moved to http check. Then, as haproxy accesses every two seconds the httpd, access.log get filled. So I simply added in the vhost:

location "/check/*" {
    no log
}

The end.

I'm not talking about everything, so if you can't find something, go read the documentation it's pretty eye-candy but stays handy.

Many thanks to vr for his kind help setting up this and for all the explanations he provided me.

And please remember that if you backport any diff from -current to -stable, you do it at your own risk.