diff --git a/config.toml b/config.toml index dddefcd..9cab283 100644 --- a/config.toml +++ b/config.toml @@ -90,11 +90,11 @@ logoText = "Hello there!" logoHomeLink = "/fr/" [menu] -# [[menu.main]] -# identifier = "blog" -# name = "Blog" -# url = "/posts" -# weight = 1 +[[menu.main]] + identifier = "blog" + name = "Blog" + url = "/posts" + weight = 1 [[menu.main]] identifier = "about_me" diff --git a/content/en/posts/dns-challenge.md b/content/en/posts/dns-challenge.md new file mode 100644 index 0000000..3e0a191 --- /dev/null +++ b/content/en/posts/dns-challenge.md @@ -0,0 +1,400 @@ +--- +title: "How to do HTTPS at home (when your infrastructure is private)" +date: 2024-07-02T21:00:50+02:00 +draft: true +toc: true +images: +tags: + - self-hosting + - sysadmin +--- + +## The problem of having a self-hosted infrastructure + +I've been maintaining a personal homelab and self-hosted infrastructure for a few years +now, but one of the most infuriating pages when starting such project is this dreaded +**Warning: Potential Security Risk Ahead** page that appears when you're using a +self-hosted certificate, or when trying to use a password on a website or app that is +served through plain HTTP. + +![A screenshot of a warning from Firefox indicating that the website that is being accessed is not secure.](/images/dns_article_firefox_warning.png) + +While acceptable if you're alone on your own infrastructure or dev environment, this +poses several issues if many other contexts: + +- It is not acceptable to publicly expose a website presenting this issue +- It's not advisable to say "hey look, I know that your browser gives you a big red +warning, but it's okay, you can just accept" to friends/family/etc. It's just a very +bad habit to have +- After a while, it really starts to get on your nerve + +Thankfully a free solution for that, which you will probably know already, has existed +for almost ten (10) years now: [Let's Encrypt and the ACME protocol](https://letsencrypt.org/) + +{{< callout type="note" >}} +I promise this is not yet another Let's Encrypt tutorial, well it is, but for a more +specific use-case +{{< /callout >}} + +## The Let's Encrypt solution + +### What is Let's Encrypt + +[Let's Encrypt](https://letsencrypt.org/) is a nonprofit certificate authority founded +in November 2014. Its main goal was to provide an easy and free way to obtain a TLS +certificate in order to make it easy to use HTTPS everywhere. + +The [ACME protocol](https://letsencrypt.org/docs/client-options/) developed by Let's +Encrypt is an automated verification system aiming at doing the following: + +- verifying that you own the domain for which you want a certificate +- creating and registering that certificate +- delivering the certificate to you + +Most client implementation also have an automated renewal system, further reducing the +workload for sysadmins. + +The current specification for the ACME protocol proposes two (2) types of challenges +to prove ownership and control over a domain: [HTTP-01](https://letsencrypt.org/docs/challenge-types/#http-01-challenge) and [DNS-01](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) challenge. + +{{< callout type="note" >}} +Actually there are two (2) others: [TLS-SNI-01](https://letsencrypt.org/docs/challenge-types/#tls-sni-01) which is now disabled, and [TLS-ALPN-01](https://letsencrypt.org/docs/challenge-types/#tls-alpn-01) which is only aimed at a very +specific category of users, which we will ignore here. +{{< /callout >}} + +### The common solution: HTTP challenge + +The [HTTP-01](https://letsencrypt.org/docs/challenge-types/#http-01-challenge) challenge +is the most common type of ACME challenge, and will satisfy most use-cases. + +![A schema describing the HTTP challenge workflow for the ACME protocol and the interactions between the application server, Let's Encrypt, and the DNS server, all of them public.](/images/dns_article_http_challenge.svg) + +For this challenge, you need the following elements : + +- A domain name and a record for that domain in a public DNS server (it can be a self-hosted DNS server, your providers', etc) +- Access to a server with a public IP that can be publicly reached + +When performing this type of challenge, the following happens (in a very simplified way): + +1. Your ACME client will ask to start a challenge to the Let's Encrypt API +2. In return, it will get a token +3. It will then either start a standalone server, or edit the configuration for your +current web server (nginx, apache, etc) to serve a file containing the token and a fingerprint of your account key. +4. Let's Encrypt will try to resolve your domain `test.example.com`. +5. If resolution works, then it will check the url `http://test.example.com/.well-known/acme-challenge/`, and verify that the file from step 3 is served with the correct +content. + +If everything works as expected, then the ACME client can download the certificate and key, and you can configure your reverse proxy or server to use this valid certificate, +all is well. + +{{< callout type="help" >}} +Okay, but my app contains my accounts, or my proxmox management interface, and I +don't really want to make it public, so how does it work here? +{{< /callout >}} + +Well it doesn't. For this type of challenge to work, the application server **must** be +public. For this challenge you need to prove that you have control over the application +that uses the target domain (even if you don't control the domain itself). But the +DNS-01 challenge bypasses this limitation. + +### When it's not enough: the DNS challenge + +As we saw in the previous section, sometimes, for various reasons, your application +server is in a private zone. It must be only reachable from inside a private network, +but you still want to be able to use a free Let's Encrypt certificate. + +For this purpose, the [DNS-01](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) challenge is based on proving that you have control over the **DNS +server** itself, instead of the application server. + +![A schema describing the DNS challenge workflow for the ACME protocol and the interaction between Let's Encrypt, the public DNS server and the private application server](/images/dns_article_dns_challenge_1.svg) + +For this type of challenge, the following elements are needed : + +- A public DNS server you have control over (can be a self-hosted server, or your DNS provider) +- A ACME client (usually it would be on your application server), it doesn't need to be public + +Then, the challenge is done the following way : + +1. Your ACME client will ask to start a challenge to the Let's Encrypt API. +2. In return, it will get a token. +3. The client then created a `TXT` record at `_acme-challenge.test.example.com` derived from the token +and your account key. +4. Let's Encrypt will try to resolve the expected `TXT` record, and verify that the content is correct. + +If the verification succeeds, you can download your certificate and key, just like the other +type of challenge. + +It's important to note that **at no point in time did Let's Encrypt have access to the +application server itself**, because this challenges involves proving that you control +the domain, not that you control the destination of that domain. + +As someone trying to use a valid certificate for my proxmox interface, this is the way I +would want to go, because it would allow me to have a valid certificate, despite my server +not being public at all. So let's see how it works in practice. + +## DNS challenge in practice + +For this example, I will try to obtain a certificate for my own domain +`example.internal.faercol.me`.As this name hints, it is an internal domain and should not +be publicly reachable, so this means I'm going to use a DNS challenge. I don't really want +to use my DNS provider API for this, so I'm going to use a self-hosted [bind](https://www.isc.org/bind/) +server for that. + +### Configuring the DNS server + +The first step is configuring the DNS server. For this, I'll just use a [bind](https://bind9.readthedocs.io/en/v9.18.27/) +server installed from my usual package manager. + +```bash +# example on Debian 12 +sudo apt install bind9 +``` + +Most of the configuration happens in the `/etc/bind` directory, mostly in `/etc/bind/named.conf.local` + +```text +root@dns-server: ls /etc/bind/ +bind.keys db.127 db.empty named.conf named.conf.local rndc.key +db.0 db.255 db.local named.conf.default-zones named.conf.options zones.rfc1918 +``` + +Let's declare a first zone, for `internal.example.com`. Add the following config to +`/etc/bind/named.conf.local` + +```text +zone "internal.example.com." IN { + type master; + file "/var/lib/bind/internal.example.com.zone"; +``` + +This simply declares a new zone which is described in the file `/var/lib/bind/internal.example.com.zone` + +Let's now create the zone itself. A DNS zone has a base structure that you must follow + +```dns +$ORIGIN . +$TTL 7200 ; 2 hours +internal.example.com IN SOA ns.internal.example.com. admin.example.com. ( + 2024070301 ; serial + 3600 ; refresh (1 hour) + 600 ; retry (10 minutes) + 86400 ; expire (1 day) + 600 ; minimum (10 minutes) + ) + NS ns.internal.example.com. + +$ORIGIN internal.example.com. +ns A 1.2.3.4 +test A 192.168.1.2 +``` + +This file declares a zone `internal.example.com` which master is `ns.internal.example.com`. +It also sets the parameters (time to live for the records, and the current serial for the +zone config). + +Finally, two (2) A records are created, associating the name `ns.internal.example.com` to +the IP address `1.2.3.4`, and `test.internal.example.com` (the domain for which we want +a certificate) to a local IP address `192.168.1.2`. + +A simple `systemctl restart bind9` would be enough to apply the modification, but we still +have one thing to do, which is allowing remote modifications to the zone. + +### Enabling remote DNS zone modification + +To allow remote modification of our DNS zone, we are going to use [TSIG](https://www.ibm.com/docs/en/aix/7.3?topic=ssw_aix_73/network/bind9_tsig.htm) +which stands for **Transaction signature**. It's a way to secure server to server operations +to edit a DNS zone, and is preferred to access control based on IP addresses. + +Let's start with creating a key using the command `tsig-keygen ` + +```shell +➜ tsig-keygen letsencrypt +key "letsencrypt" { + algorithm hmac-sha256; + secret "oK6SqKRvGNXHyNyIEy3hijQ1pclreZw4Vn5v+Q4rTLs="; +}; +``` + +This creates a key with the given name using the default algorithm (which is `hmac-sha256`). +The entire output of this command is actually a code block that you can add to your bind9 +configuration. + +Finally, using `update-policy`, allow this key to be used to update the zone. + +```text +update-policy { + grant letsencrypt. zonesub txt; +}; +``` + +{{< callout type="note" >}} +Doing so allows users to update everything in your zone using this key. In fact +you would only need to update `_acme-challenge.test.internal.example.com` as seen +in the DNS challenge description. + +If you want a better restriction, then you can use the following configuration instead + +```text +update-policy { + grant letsencrypt. name _acme-challenge.test.internal.example.com. txt; +}; +``` + +{{< /callout >}} + +This means your entire `named.conf.local` would become something like this + +```text +key "letsencrypt" { + algorithm hmac-sha256; + secret "oK6SqKRvGNXHyNyIEy3hijQ1pclreZw4Vn5v+Q4rTLs="; +}; + +zone "internal.example.com." IN { + type master; + file "/var/lib/bind/internal.example.com.zone"; + update-policy { + grant letsencrypt. zonesub txt; + }; +}; +``` + +{{< callout type="warning" >}} +Be **very cautious** about the `.` at the end of the zone name and the key name, they are +easy to miss, and forgetting them will cause issues that would be hard to detect. +{{< /callout >}} + +With that being done, you can restart the DNS server and everything is ready server side, +the only remaining thing to do would be the DNS challenge itself. + +### Performing the challenge + +Start by installing the certbot with the RFC2136 plugin (to perform the DNS challenge). + +```shell +apt install python3-certbot-dns-rfc2136 +``` + +It's handled using a `.ini` configuration file, let's put it in `/etc/certbot/credentials.ini` + +```ini +dns_rfc2136_server = +dns_rfc2136_port = 53 +dns_rfc2136_name = letsencrypt. +dns_rfc2136_secret = oK6SqKRvGNXHyNyIEy3hijQ1pclreZw4Vn5v+Q4rTLs= +dns_rfc2136_algorithm = HMAC-SHA512 +``` + +Finally, run the challenge using certbot (if it's the first time you're using certbot on +that machine, it might ask for an email to handle admin stuff). + +```shell +root@toolbox:~# certbot certonly --dns-rfc2136 --dns-rfc2136-credentials /etc/certbot/credentials.ini -d 'test.internal.example.com' + +Saving debug log to /var/log/letsencrypt/letsencrypt.log +Requesting a certificate for test.internal.example.com +Waiting 60 seconds for DNS changes to propagate + +Successfully received certificate. +Certificate is saved at: /etc/letsencrypt/live/test.internal.example.com/fullchain.pem +Key is saved at: /etc/letsencrypt/live/test.internal.example.com/privkey.pem +This certificate expires on 2024-09-30. +These files will be updated when the certificate renews. +Certbot has set up a scheduled task to automatically renew this certificate in the background. + +- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +If you like Certbot, please consider supporting our work by: + * Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate + * Donating to EFF: https://eff.org/donate-le +- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +``` + +And that's done, you have a certificate, and a no point in time did you need to +actually expose your application to the outside world. + +Now because I like to go way too far, I can propose two (2) improvements to this +setup: + +- Using ACL in addition to the TSIG key to secure operations on the DNS server +- Using a second DNS server only locally accessible for your private records, and +using the public server to only perform challenges + +## Bonus 1: adding a second layer of authentication to connect to the DNS + +In our setup, we used **TSIG** to secure our access to the DNS server, meaning that +having the key is necessary to perform the operations. If you are paranoid, or if you +want to do a little bit more, then you could add a second layer of authentication based +on [Access Control List (ACL)](https://bind9.readthedocs.io/en/v9.18.1/security.html). + +**ACL** allow to filter allowed operations based on several characteristics, such as +IP address, TSIG key, subnet. In our case, we will use an IPV4 subnet from inside a +Wireguard tunnel between the application servers (DNS clients) and the DNS server. It +could be any form of tunnel, but Wireguard is easy to configure and perfect for +point-to-point tunnels such as what we are doing here. + +### Wireguard configuration + +First, let's create the [Wireguard](https://www.wireguard.com/quickstart/) tunnel. + +We start by creating two wireguard key pairs, which can be done this way + +```shell +# Install wireguard tools +apt install wireguard-tools + +# Create the keypair +wg genkey | tee privatekey | wg pubkey > publickey +``` + +Private key is in the `privatekey` file, and public key in the `publickey` file. + +Then we can create the server configuration, create a file `/etc/wg/wg0.conf` on +the DNS server. + +```ini +[Interface] +PrivateKey = +Address = 192.168.42.1/24 +ListenPort = 51820 + +[Peer] +PublicKey = +AllowedIPs = 192.168.42.0/24 +``` + +Then on the client side you can do the same + +```ini +[Interface] +PrivateKey = +Address = 192.168.42.2/24 + +[Peer] +PublicKey = +Endpoint = :51820 +AllowedIPs = 192.168.42.1/32 +``` + +Then you can start the tunnel on both sides using `wg-quick up wg0`, check that ip works +by pinging the server from the client + +```shell +root@toolbox:~ ping 192.168.42.1 +PING 192.168.42.1 (192.168.42.1) 56(84) bytes of data. +64 bytes from 192.168.42.1: icmp_seq=1 ttl=64 time=19.2 ms +64 bytes from 192.168.42.1: icmp_seq=2 ttl=64 time=8.25 ms +``` + +Basically, we created a new network `192.168.42.0/24` which links the DNS server and our client, +and we can restrict modification to the DNS zone to force them to be from inside the +virtual network, instead of allowing them from anywhere. + +{{< callout type="note" >}} +The ACL that we are going to use here can have many other purposes, such as hiding +some domains, or serving different versions of a zone depending on the origin of +the client. This is not our topic of concern here though. +{{< /callout >}} + +## Bonus 2: completely hiding your private domains from outside + +![A schema describing the DNS challenge workflow for the ACME protocol using a public and private DNS servers](/images/dns_article_dns_challenge_2.svg) diff --git a/static/images/dns_article_dns_challenge_1.svg b/static/images/dns_article_dns_challenge_1.svg new file mode 100644 index 0000000..e81e1ad --- /dev/null +++ b/static/images/dns_article_dns_challenge_1.svg @@ -0,0 +1,8 @@ + + + + + + + + Private zonePublic zoneApplication serverLet's EncryptDNS server1. Start DNS challenge andobtain secret2. Create a TXT recordwith the secret3. Check secretUserGet applicationaddress from DNS \ No newline at end of file diff --git a/static/images/dns_article_dns_challenge_2.svg b/static/images/dns_article_dns_challenge_2.svg new file mode 100644 index 0000000..adab647 --- /dev/null +++ b/static/images/dns_article_dns_challenge_2.svg @@ -0,0 +1,8 @@ + + + + + + + + Private zonePublic zoneApplication serverLet's Encryptpublic DNS server1. Start DNS challenge andobtain secret2. Create a TXT recordwith the secret3. Check secretUserGet applicationaddress from DNSprivate DNS server \ No newline at end of file diff --git a/static/images/dns_article_firefox_warning.png b/static/images/dns_article_firefox_warning.png new file mode 100644 index 0000000..50b4318 Binary files /dev/null and b/static/images/dns_article_firefox_warning.png differ diff --git a/static/images/dns_article_http_challenge.svg b/static/images/dns_article_http_challenge.svg new file mode 100644 index 0000000..7ca1ef3 --- /dev/null +++ b/static/images/dns_article_http_challenge.svg @@ -0,0 +1,21 @@ + + + + + + + + Public zoneApplication serverLet's EncryptDNS server1. Start HTTP challenge2. Check DNS record3. Perform HTTP challenge \ No newline at end of file