Add article on DNS challenge

config.toml

@@ -90,11 +90,11 @@ logoText = "Hello there!"
 logoHomeLink = "/fr/"
 
 [menu]
-# [[menu.main]]
-#   identifier = "blog"
-#   name = "Blog"
-#   url = "/posts"
-#   weight = 1
+[[menu.main]]
+  identifier = "blog"
+  name = "Blog"
+  url = "/posts"
+  weight = 1
 
 [[menu.main]]
 identifier = "about_me"

content/en/posts/dns-challenge.md

@@ -0,0 +1,493 @@
+---
+title: "How to do HTTPS at home (when your infrastructure is private)"
+date: 2024-07-02T21:00:50+02:00
+draft: true
+toc: true
+images:
+tags:
+  - self-hosting
+  - sysadmin
+---
+
+## The problem of having a self-hosted infrastructure
+
+I've been maintaining a personal homelab and self-hosted infrastructure for a few years
+now, but one of the most infuriating pages when starting such project is this dreaded
+**Warning: Potential Security Risk Ahead** page that appears when you're using a
+self-hosted certificate, or when trying to use a password on a website or app that is
+served through plain HTTP.
+
+![A screenshot of a warning from Firefox indicating that the website that is being accessed is not secure.](/images/dns_article_firefox_warning.png)
+
+While acceptable if you're alone on your own infrastructure or dev environment, this
+poses several issues if many other contexts:
+
+- It is not acceptable to publicly expose a website presenting this issue
+- It's not advisable to say "hey look, I know that your browser gives you a big red
+warning, but it's okay, you can just accept" to friends/family/etc. It's just a very
+bad habit to have
+- After a while, it really starts to get on your nerve
+
+Thankfully a free solution for that, which you will probably know already, has existed
+for almost ten (10) years now: [Let's Encrypt and the ACME protocol](https://letsencrypt.org/)
+
+{{< callout type="note" >}}
+I promise this is not yet another Let's Encrypt tutorial, well it is, but for a more
+specific use-case
+{{< /callout >}}
+
+## The Let's Encrypt solution
+
+### What is Let's Encrypt
+
+[Let's Encrypt](https://letsencrypt.org/) is a nonprofit certificate authority founded
+in November 2014. Its main goal was to provide an easy and free way to obtain a TLS
+certificate in order to make it easy to use HTTPS everywhere.
+
+The [ACME protocol](https://letsencrypt.org/docs/client-options/) developed by Let's
+Encrypt is an automated verification system aiming at doing the following:
+
+- verifying that you own the domain for which you want a certificate
+- creating and registering that certificate
+- delivering the certificate to you
+
+Most client implementation also have an automated renewal system, further reducing the
+workload for sysadmins.
+
+The current specification for the ACME protocol proposes two (2) types of challenges
+to prove ownership and control over a domain: [HTTP-01](https://letsencrypt.org/docs/challenge-types/#http-01-challenge) and [DNS-01](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) challenge.
+
+{{< callout type="note" >}}
+Actually there are two (2) others: [TLS-SNI-01](https://letsencrypt.org/docs/challenge-types/#tls-sni-01) which is now disabled, and [TLS-ALPN-01](https://letsencrypt.org/docs/challenge-types/#tls-alpn-01) which is only aimed at a very
+specific category of users, which we will ignore here.
+{{< /callout >}}
+
+### The common solution: HTTP challenge
+
+The [HTTP-01](https://letsencrypt.org/docs/challenge-types/#http-01-challenge) challenge
+is the most common type of ACME challenge, and will satisfy most use-cases.
+
+![A schema describing the HTTP challenge workflow for the ACME protocol and the interactions between the application server, Let's Encrypt, and the DNS server, all of them public.](/images/dns_article_http_challenge.svg)
+
+For this challenge, you need the following elements :
+
+- A domain name and a record for that domain in a public DNS server (it can be a self-hosted DNS server, your providers', etc)
+- Access to a server with a public IP that can be publicly reached
+
+When performing this type of challenge, the following happens (in a very simplified way):
+
+1. Your ACME client will ask to start a challenge to the Let's Encrypt API
+2. In return, it will get a token
+3. It will then either start a standalone server, or edit the configuration for your
+current web server (nginx, apache, etc) to serve a file containing the token and a fingerprint of your account key.
+4. Let's Encrypt will try to resolve your domain `test.example.com`.
+5. If resolution works, then it will check the url `http://test.example.com/.well-known/acme-challenge/<TOKEN>`, and verify that the file from step 3 is served with the correct
+content.
+
+If everything works as expected, then the ACME client can download the certificate and key, and you can configure your reverse proxy or server to use this valid certificate,
+all is well.
+
+{{< callout type="help" >}}
+Okay, but my app contains my accounts, or my proxmox management interface, and I
+don't really want to make it public, so how does it work here?
+{{< /callout >}}
+
+Well it doesn't. For this type of challenge to work, the application server **must** be
+public. For this challenge you need to prove that you have control over the application
+that uses the target domain (even if you don't control the domain itself). But the
+DNS-01 challenge bypasses this limitation.
+
+### When it's not enough: the DNS challenge
+
+As we saw in the previous section, sometimes, for various reasons, your application
+server is in a private zone. It must be only reachable from inside a private network,
+but you still want to be able to use a free Let's Encrypt certificate.
+
+For this purpose, the [DNS-01](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge) challenge is based on proving that you have control over the **DNS
+server** itself, instead of the application server.
+
+![A schema describing the DNS challenge workflow for the ACME protocol and the interaction between Let's Encrypt, the public DNS server and the private application server](/images/dns_article_dns_challenge_1.svg)
+
+For this type of challenge, the following elements are needed :
+
+- A public DNS server you have control over (can be a self-hosted server, or your DNS provider)
+- A ACME client (usually it would be on your application server), it doesn't need to be public
+
+Then, the challenge is done the following way :
+
+1. Your ACME client will ask to start a challenge to the Let's Encrypt API.
+2. In return, it will get a token.
+3. The client then created a `TXT` record at `_acme-challenge.test.example.com` derived from the token
+and your account key.
+4. Let's Encrypt will try to resolve the expected `TXT` record, and verify that the content is correct.
+
+If the verification succeeds, you can download your certificate and key, just like the other
+type of challenge.
+
+It's important to note that **at no point in time did Let's Encrypt have access to the
+application server itself**, because this challenges involves proving that you control
+the domain, not that you control the destination of that domain.
+
+As someone trying to use a valid certificate for my proxmox interface, this is the way I
+would want to go, because it would allow me to have a valid certificate, despite my server
+not being public at all. So let's see how it works in practice.
+
+## DNS challenge in practice
+
+For this example, I will try to obtain a certificate for my own domain
+`example.internal.faercol.me`.As this name hints, it is an internal domain and should not
+be publicly reachable, so this means I'm going to use a DNS challenge. I don't really want
+to use my DNS provider API for this, so I'm going to use a self-hosted [bind](https://www.isc.org/bind/)
+server for that.
+
+### Configuring the DNS server
+
+The first step is configuring the DNS server. For this, I'll just use a [bind](https://bind9.readthedocs.io/en/v9.18.27/)
+server installed from my usual package manager.
+
+```bash
+# example on Debian 12
+sudo apt install bind9
+```
+
+Most of the configuration happens in the `/etc/bind` directory, mostly in `/etc/bind/named.conf.local`
+
+```text
+root@dns-server: ls /etc/bind/
+bind.keys  db.127  db.empty  named.conf         named.conf.local    rndc.key
+db.0    db.255  db.local  named.conf.default-zones  named.conf.options  zones.rfc1918
+```
+
+Let's declare a first zone, for `internal.example.com`. Add the following config to
+`/etc/bind/named.conf.local`
+
+```text
+zone "internal.example.com." IN {
+  type master;
+  file "/var/lib/bind/internal.example.com.zone";
+```
+
+This simply declares a new zone which is described in the file `/var/lib/bind/internal.example.com.zone`
+
+Let's now create the zone itself. A DNS zone has a base structure that you must follow
+
+```dns
+$ORIGIN .
+$TTL 7200 ; 2 hours
+internal.example.com IN SOA ns.internal.example.com. admin.example.com. (
+                            2024070301 ; serial
+                            3600       ; refresh (1 hour)
+                            600        ; retry (10 minutes)
+                            86400      ; expire (1 day)
+                            600        ; minimum (10 minutes)
+                            )
+                    NS ns.internal.example.com.
+
+$ORIGIN internal.example.com.
+ns      A     1.2.3.4
+test    A     192.168.1.2
+```
+
+This file declares a zone `internal.example.com` which master is `ns.internal.example.com`.
+It also sets the parameters (time to live for the records, and the current serial for the
+zone config).
+
+Finally, two (2) A records are created, associating the name `ns.internal.example.com` to
+the IP address `1.2.3.4`, and `test.internal.example.com` (the domain for which we want
+a certificate) to a local IP address `192.168.1.2`.
+
+A simple `systemctl restart bind9` would be enough to apply the modification, but we still
+have one thing to do, which is allowing remote modifications to the zone.
+
+### Enabling remote DNS zone modification
+
+To allow remote modification of our DNS zone, we are going to use [TSIG](https://www.ibm.com/docs/en/aix/7.3?topic=ssw_aix_73/network/bind9_tsig.htm)
+which stands for **Transaction signature**. It's a way to secure server to server operations
+to edit a DNS zone, and is preferred to access control based on IP addresses.
+
+Let's start with creating a key using the command `tsig-keygen <keyname>`
+
+```shell
+➜ tsig-keygen letsencrypt
+key "letsencrypt" {
+ algorithm hmac-sha256;
+ secret "oK6SqKRvGNXHyNyIEy3hijQ1pclreZw4Vn5v+Q4rTLs=";
+};
+```
+
+This creates a key with the given name using the default algorithm (which is `hmac-sha256`).
+The entire output of this command is actually a code block that you can add to your bind9
+configuration.
+
+Finally, using `update-policy`, allow this key to be used to update the zone.
+
+```text
+update-policy {
+  grant letsencrypt. zonesub txt;
+};
+```
+
+{{< callout type="note" >}}
+Doing so allows users to update everything in your zone using this key. In fact
+you would only need to update `_acme-challenge.test.internal.example.com` as seen
+in the DNS challenge description.
+
+If you want a better restriction, then you can use the following configuration instead
+
+```text
+update-policy {
+  grant letsencrypt. name _acme-challenge.test.internal.example.com. txt;
+};
+```
+
+{{< /callout >}}
+
+This means your entire `named.conf.local` would become something like this
+
+```text
+key "letsencrypt" {
+ algorithm hmac-sha256;
+ secret "oK6SqKRvGNXHyNyIEy3hijQ1pclreZw4Vn5v+Q4rTLs=";
+};
+
+zone "internal.example.com." IN {
+  type master;
+  file "/var/lib/bind/internal.example.com.zone";
+  update-policy {
+    grant letsencrypt. zonesub txt;
+  };
+};
+```
+
+{{< callout type="warning" >}}
+Be **very cautious** about the `.` at the end of the zone name and the key name, they are
+easy to miss, and forgetting them will cause issues that would be hard to detect.
+{{< /callout >}}
+
+With that being done, you can restart the DNS server and everything is ready server side,
+the only remaining thing to do would be the DNS challenge itself.
+
+### Performing the challenge
+
+Start by installing the certbot with the RFC2136 plugin (to perform the DNS challenge).
+
+```shell
+apt install python3-certbot-dns-rfc2136
+```
+
+It's handled using a `.ini` configuration file, let's put it in `/etc/certbot/credentials.ini`
+
+```ini
+dns_rfc2136_server = <you_dns_ip>
+dns_rfc2136_port = 53
+dns_rfc2136_name = letsencrypt.
+dns_rfc2136_secret = oK6SqKRvGNXHyNyIEy3hijQ1pclreZw4Vn5v+Q4rTLs=
+dns_rfc2136_algorithm = HMAC-SHA512
+```
+
+Finally, run the challenge using certbot (if it's the first time you're using certbot on
+that machine, it might ask for an email to handle admin stuff).
+
+```shell
+root@toolbox:~# certbot certonly --dns-rfc2136 --dns-rfc2136-credentials /etc/certbot/credentials.ini -d 'test.internal.example.com'
+
+Saving debug log to /var/log/letsencrypt/letsencrypt.log
+Requesting a certificate for test.internal.example.com
+Waiting 60 seconds for DNS changes to propagate
+
+Successfully received certificate.
+Certificate is saved at: /etc/letsencrypt/live/test.internal.example.com/fullchain.pem
+Key is saved at:         /etc/letsencrypt/live/test.internal.example.com/privkey.pem
+This certificate expires on 2024-09-30.
+These files will be updated when the certificate renews.
+Certbot has set up a scheduled task to automatically renew this certificate in the background.
+
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+If you like Certbot, please consider supporting our work by:
+ * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
+ * Donating to EFF:                    https://eff.org/donate-le
+- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+```
+
+And that's done, you have a certificate, and a no point in time did you need to
+actually expose your application to the outside world.
+
+Now because I like to go way too far, I can propose two (2) improvements to this
+setup:
+
+- Using ACL in addition to the TSIG key to secure operations on the DNS server
+- Using a second DNS server only locally accessible for your private records, and
+using the public server to only perform challenges
+
+## Bonus 1: adding a second layer of authentication to connect to the DNS
+
+In our setup, we used **TSIG** to secure our access to the DNS server, meaning that
+having the key is necessary to perform the operations. If you are paranoid, or if you
+want to do a little bit more, then you could add a second layer of authentication based
+on [Access Control List (ACL)](https://bind9.readthedocs.io/en/v9.18.1/security.html).
+
+**ACL** allow to filter allowed operations based on several characteristics, such as
+IP address, TSIG key, subnet. In our case, we will use an IPV4 subnet from inside a
+Wireguard tunnel between the application servers (DNS clients) and the DNS server. It
+could be any form of tunnel, but Wireguard is easy to configure and perfect for
+point-to-point tunnels such as what we are doing here.
+
+### Wireguard configuration
+
+First, let's create the [Wireguard](https://www.wireguard.com/quickstart/) tunnel.
+
+We start by creating two wireguard key pairs, which can be done this way
+
+```shell
+# Install wireguard tools
+apt install wireguard-tools
+
+# Create the keypair
+wg genkey | tee privatekey | wg pubkey > publickey
+```
+
+Private key is in the `privatekey` file, and public key in the `publickey` file.
+
+Then we can create the server configuration, create a file `/etc/wg/wg0.conf` on
+the DNS server.
+
+```ini
+[Interface]
+PrivateKey = <server_private_key>
+Address = 192.168.42.1/24
+ListenPort = 51820
+
+[Peer]
+PublicKey = <client_public_key>
+AllowedIPs = 192.168.42.0/24
+```
+
+Then on the client side you can do the same
+
+```ini
+[Interface]
+PrivateKey = <client_private_key>
+Address = 192.168.42.2/24
+
+[Peer]
+PublicKey = <server_public_key>
+Endpoint = <dns_public_ip>:51820
+AllowedIPs = 192.168.42.1/32
+```
+
+Then you can start the tunnel on both sides using `wg-quick up wg0`, check that ip works
+by pinging the server from the client
+
+```shell
+root@toolbox:~ ping 192.168.42.1
+PING 192.168.42.1 (192.168.42.1) 56(84) bytes of data.
+64 bytes from 192.168.42.1: icmp_seq=1 ttl=64 time=19.2 ms
+64 bytes from 192.168.42.1: icmp_seq=2 ttl=64 time=8.25 ms
+```
+
+Basically, we created a new network `192.168.42.0/24` which links the DNS server and our client,
+and we can restrict modification to the DNS zone to force them to be from inside the
+virtual network, instead of allowing them from anywhere.
+
+{{< callout type="note" >}}
+The ACL that we are going to use here can have many other purposes, such as hiding
+some domains, or serving different versions of a zone depending on the origin of
+the client. This is not our topic of concern here though.
+{{< /callout >}}
+
+### DNS configuration
+
+Using ACLs, we are going to split the DNS zone into several [views](https://kb.isc.org/docs/aa-00851)
+based on the source IP. Basically our goal is to say that
+
+- Users coming from inside our wireguard network `192.168.42.0/24` can modify DNS
+records in our zone using the TSIG key defined earlier.
+- Users coming from any other IP can read the DNS zone, but nothing else, so they can't
+update it, even using the correct key.
+
+ACL can be defined inside `named.conf.local` using the following syntax.
+
+```text
+acl local {
+  127.0.0.0/8;
+  192.168.42.0/24;
+};
+```
+
+This means that local addresses, and addresses coming from our wireguard network
+will be considered as `local` and can be referenced as such in the rest of the
+configuration.
+
+Then, a view can be created like this:
+
+```text
+view "internal" {
+  match-clients { local; };
+  zone "internal.example.com." IN {
+    type master;
+    file "/var/lib/bind/internal.example.com.zone";
+    update-policy {
+      grant letsencrypt. zonesub txt;
+    };
+  };
+};
+```
+
+Basically this means that the view `internal` is only used for clients that match
+the `local` ACL defined above. In this view we define the zone `internal.example.com`,
+which is the zone we defined ealier.
+
+We also need to declare the zone for non-local users who wouldn't match the `local` ACL.
+It's important to note that **you cannot use the same zone file twice in different zones**,
+so we cannot define the public view exactly the same way. Our public view will be
+defined the following way:
+
+```text
+view "public" {
+  zone "internal.example.com." IN {
+    in-view internal;
+  };
+};
+```
+
+This way, in the `public` view, we define the `internal.example.com` zone, and we
+define this zone as being inside the `internal` view. This way, we will serve the
+exact same DNS zone whatever the origin, but the *update policy* only applies to user
+from local addresses, and they will be the only ones able to edit the zone.
+
+In summary, our `named.conf.local` file should now look like this.
+
+```text
+acl local {
+  127.0.0.0/8;
+  192.168.42.0/24;
+};
+
+key "letsencrypt." {
+  algorithm hmac-sha512;
+  secret "oK6SqKRvGNXHyNyIEy3hijQ1pclreZw4Vn5v+Q4rTLs=";
+};
+
+view "internal" {
+  match-clients { local; };
+  zone "internal.example.com." IN {
+    type master;
+    file "/var/lib/bind/internal.example.com.zone";
+    update-policy {
+      grant letsencrypt. zonesub txt;
+    };
+  };
+};
+
+view "public" {
+  zone "internal.example.com." IN {
+    in-view internal;
+  };
+};
+```
+
+And now
+
+## Bonus 2: completely hiding your private domains from outside
+
+![A schema describing the DNS challenge workflow for the ACME protocol using a public and private DNS servers](/images/dns_article_dns_challenge_2.svg)