TLS Acceleration for Multiple Domains with relayd
Please see the TLS acceleration guide with relayd before consulting this guide. Concepts from the earlier guide will be expanded upon in this guide.
Contents
Check Server in Plaintext
Before using relayd, make sure your server serves its content in plaintext. If the server doesn't respond to plaintext requests, relayd won't work, either.
WARNING: If you are using relayd for TLS acceleration for openhttpd, make sure openhttpd does not have a listener on port 443. Look for blocks like the ones below:
server "example.com" { listen on * tls port 443 tls { certificate "/etc/ssl/example.com.fullchain.pem" key "/etc/ssl/private/example.com.key" } location "/pub/*" { directory auto index } location "/.well-known/acme-challenge/*" { root "/acme" request strip 2 } }
Notice the line listen on * tls port 443
and the tls {...}
block. If httpd is listening on port 443 while relayd is running, the reverse proxy will fail to forward for android, iOS, and other devices!
In the configuration below, we assume you are following the openhttpd hosting guide and your http services are listening on port 80.
Request SSL Certs
Make sure you have the SSL certs you need for the domains you want to provide TLS acceleration for. Request them using acme-client if you have not already.
By default, relayd searches /etc/ssl/name:port.crt
and /etc/ssl/name:port.key
for the public/private keypair. So, we will create symlinks:
$ doas ln -s /etc/ssl/example.com.fullchain.pem /etc/ssl/example.com:443.crt $ doas ln -s /etc/ssl/private/example.com.key /etc/ssl/private/example.com:443.key
Make sure to replace example.com with your actual domain.
Edit relayd.conf
Let's create /etc/relayd.conf. This configuration will provide TLS acceleration for three services: one that listens on port 8001, a second which listens on port 8002, and a third which listens on port 80. Here is what we will put, one block at a time:
ip4="192.168.1.1" ip6="2001:db8::" table <service1> { 127.0.0.1 } table <service2> { 127.0.0.1 } table <www> { 127.0.0.1 } log connection
Replace ip4 and ip6 with the actual IPv4 and IPv6 address you want to listen on. Make sure the IPv4 is DDoS-filtered if you have that option.
http protocol https { match request header append "X-Forwarded-For" value "$REMOTE_ADDR" match request header append "X-Forwarded-By" \ value "$SERVER_ADDR:$SERVER_PORT" match request header set "Connection" value "close" tcp { sack, backlog 128 } tls { keypair service1.example.com } tls { keypair service2.example.com } tls { keypair www.example.com } tls { keypair www.sub.example.com } match request header "Host" value "service1.example.com" forward to <service1> match request header "Host" value "service2.example.com" forward to <service2> match request header "Host" value "*" forward to <www> }
relayd will inspect the headers of the HTTP requests that users send. If the header says service1.example.com, it will forward to port 8001 for service1 to handle. If it says service2, then it will forward to port 8002 for service2. And finally, all the remaining domains are forwarded to port 80 for openhttpd to handle.
We also define how to handle the http protocol. We add X-Forwarded-For, X-Forwarded-By, and Connection headers to HTTP requests before forwarding it to openhttpd.
We turn on selective acknowledgments and set the maximum queue to 128 connections in the tcp block.
We then define the keypair names. Here's where relayd searches for them:
Name | Public Cert | Private Key |
---|---|---|
name | /etc/ssl/name.crt | /etc/ssl/private/name.key |
service1.example.com | /etc/ssl/service1.example.com.crt | /etc/ssl/private/service1.example.com.key |
service2.example.com | /etc/ssl/service2.example.com.crt | /etc/ssl/private/service2.example.com.key |
www.example.com | /etc/ssl/www.example.com.crt | /etc/ssl/private/www.example.com.key |
www.sub.example.com | /etc/ssl/www.sub.example.com.crt | /etc/ssl/private/www.sub.example.com.key |
The last two lines forward to the proper service based on the Host HTTP header.
relay wwwtls { listen on $ip4 port 443 tls protocol https forward to <service1> port 8001 check icmp forward to <service2> port 8002 check icmp forward to <www> port 80 check icmp } relay www6tls { listen on $ip6 port 443 tls protocol https forward to <service1> port 8001 check icmp forward to <service2> port 8002 check icmp forward to <www> port 80 check icmp }
We create two relays, one for IPv4 and another for IPv6. Both of them listen on port 443 using TLS. They handle use the protocol template for https and forward to the proper service on ports 8001, 8002, and 80 (see the above openhttpd hosting guide). Both check ICMP to see if the service is available.
Complete relayd.conf
Here is the entire /etc/relayd.conf without commentary:
ip4="192.168.1.1" ip6="2001:db8::" table <service1> { 127.0.0.1 } table <service2> { 127.0.0.1 } table <www> { 127.0.0.1 } log connection http protocol https { match request header append "X-Forwarded-For" value "$REMOTE_ADDR" match request header append "X-Forwarded-By" \ value "$SERVER_ADDR:$SERVER_PORT" match request header set "Connection" value "close" tcp { sack, backlog 128 } tls { keypair service1.example.com } tls { keypair service2.example.com } tls { keypair www.example.com } tls { keypair www.sub.example.com } match request header "Host" value "service1.example.com" forward to <service1> match request header "Host" value "service2.example.com" forward to <service2> match request header "Host" value "*" forward to <www> } relay wwwtls { listen on $ip4 port 443 tls protocol https forward to <service1> port 8001 check icmp forward to <service2> port 8002 check icmp forward to <www> port 80 check icmp } relay www6tls { listen on $ip6 port 443 tls protocol https forward to <service1> port 8001 check icmp forward to <service2> port 8020 check icmp forward to <www> port 80 check icmp }
Login class permissions
If you have a large number of TLS certs, you will need to increase the maximum number of files that relayd can open. Add this to the bottom of /etc/login.conf:
relayd:\ :openfiles=4096:\ :stacksize-cur=96M:\ :stacksize-max=96M:\ :tc=daemon:
Then we must make sure there is no login.conf.db
database:
$ doas rm /etc/login.conf.db
Starting relayd
$ doas rcctl enable relayd $ doas rcctl start relayd
WARNING: Make sure that packet filter is enabled! relayd will not run if pf is disabled. You can enable it by typing:
$ doas pfctl -e
To test relayd, we'll use openssl:
Troubleshooting
If relayd fails to start, you will see this message:
relayd(failed)
First, we check the conf file to see if there are any errors:
$ doas relayd -n
When properly configured, relayd will say configuration OK
.
Sample errors:
- /etc/relayd.conf:NN: cannot load keypair example.com for relay wwwtls
Check line NN. Your keypair may be missing, have the wrong permissions, or are not labeled correctly. - /etc/relayd.conf:NN: syntax error
Check line NN for syntax errors.
To turn on debugging, first stop any running instances of relayd then run it in the foreground:
$ doas rcctl stop relayd $ doas relayd -dvv
-d is for debug and -v is to increase verbosity.
log connection
WARNING: This may produce a verbose output which can dramatically increase the size of your /var/log/daemon, especially on busy networks. To avoid this, simply have your syslogd send all relayd messages into its own file. To that, see here.
In addition to splitting relayd logs to its own file, you may wish to create a new entry in your /etc/newsyslog.conf to handle log rotation for your relayd.
common errors
- Make sure httpd is not listening on port 443
- Make sure all domains have real dns records
- Make sure nsd is set up properly
- Make sure packet filter is turned on