Typing any random subdomain gave me a blank white page, so I built a catchall page
Because of wildcard DNS and the Apache default vhost, nonexistent subdomains landed on my blog and rendered a blank page. How I settled file order, the wildcard cert, status codes and noindex while replacing it with a landing page.
#Apache #Infrastructure #Reverse proxy #DNS #Operations
My server has close to ten subdomains attached to it. Each service gets one, like blog, coin, travel, couple, game, taro and fishing, and Apache looks at the incoming domain and passes the request to the right port. I wrote up that setup in an earlier post . One day I mistyped my blog address on my phone. I ended up on blgo.jaeyonging.com, and instead of an error like "This site can't be reached" I got a blank white page. The tab title, though, was my blog's title. Just to check, I tried something random like asdfas.jaeyonging.com and got the same thing. Any address I typed opened the shell of my blog with nothing inside. In this post I go over why that happened, and what I thought through while changing it so that unregistered subdomains now land on a page I prepared in advance. Why did any address open The cause was in two places. The first was DNS. My domain is managed at Gabia, and since adding a record every time I brought up a service was a chore, at some point I set up a single * record so every subdomain points to my server IP. So even subdomains that don't exist anywhere end up at my server. I masked the server IP, but all three return the same address. $ dig +short jaeyonging.com xxx.xxx.xxx.xxx $ dig +short asdfas.jaeyonging.com xxx.xxx.xxx.xxx $ dig +short nonexist123.jaeyonging.com xxx.xxx.xxx.xxx The second was Apache. Name based virtual hosts look at the Host header of the request, or the SNI for HTTPS, and look for a vhost whose ServerName or ServerAlias matches. But when nothing matches, Apache doesn't return an error. It sends the request to the first vhost it read for that port . That is the default vhost. Which one gets read first is decided by the file name order inside sites-enabled . In my case the files starting with blog were at the front. I never once chose the blog as the default, but just because b comes early in the alphabet, the blog was taking every request that didn't match anything else. Running apache2ctl -S shows the default server for each port, so if you run several subdomains, I recommend checking it once. So why was the page blank If the blog had just shown up as usual it would have been less strange, but the page was empty. When I opened the dev tools, the HTML had arrived but the JS file was a 500. GET https://asdfas.jaeyonging.com/assets/index-e215cf4d.js 500 (Internal Server Error) It was the CORS setup on the blog server. The scripts Vite builds are loaded with type="module" , and module scripts are requested with an Origin header even when they are same origin. The Express cors middleware throws an error when an Origin not on the allowlist comes in, and that turns straight into a 500. A mistyped host would never be on the allowlist, so not a single JS file got through and only the shell was left. $ curl -s -H 'Origin: https://asdfas.jaeyonging.com' \ http://localhost:5000/assets/AdminBlog-3568ff89.js | head -c 120 !DOCTYPE html html lang="en" head meta charset="utf-8" title Error /title /head body pre Error: Not allowed by CORS curl doesn't send an Origin, so if you just run it plainly you get a 200. It only breaks in the browser, which is why I spent a while finding the cause at first. I laid out a few options and weighed them The easiest one was adding *.jaeyonging.com to the CORS allowlist. But then instead of a blank page, the blog would open on an endless number of addresses. I happened to be in the middle of an AdSense review, so I wanted as few addresses showing the same content as possible. Also, the blog being the default was an accident in the first place. If I later added a service whose name starts with a, from that day on every mistyped address would go to that service. A setup where adding one config file changes something unrelated is bound to confuse me someday. Removing the wildcard DNS record was another option. Typos would end at the DNS step, which is the cleanest. In exchange, every time I brought up a service I would have to go into Gabia, add a record and wait for it to propagate. I switched to the wildcard precisely because that was a chore, so I didn't want to go back. And for someone who made a typo, it ends with nothing but a browser error page. I also thought about a 301 redirect to the blog or the root domain. But then infinitely many addresses would all act like aliases of a real address, and there's no guarantee that someone who mistyped a subdomain was heading for the blog. They might have fumbled while typing coin. So I settled on a landing page that makes it clear the address doesn't exist and shows the list of services that actually do . The person knows best where they were trying to go, so I figured it was right to let them pick. Making a default vhost on purpose The Apache side is simpler than it sounds. If the default vhost is decided by file name order, all you need is a vhost whose name sorts first. # /etc/apache2/sites-available/000-catchall.conf # Default vhost that takes every request no other vhost matches # (mistyped subdomains, direct IP access, etc). # It has to be loaded first in sites-enabled to become the default vhost, # so don't change the 000- prefix in the file name. VirtualHost *:80 ServerName catchall.jaeyonging.com ServerAdmin admin@jaeyonging.com DocumentRoot /var/www/catchall Directory /var/www/catchall Require all granted Options -Indexes AllowOverride None /Directory ProxyRequests Off # "/" is the landing page, any other path gets the same page with a 404 status DirectoryIndex index.html ErrorDocument 404 /index.html # keep the endless mistyped subdomains out of search indexes Header always set X-Robots-Tag "noindex, nofollow" ErrorLog ${APACHE_LOG_DIR}/catchall_error.log CustomLog ${APACHE_LOG_DIR}/catchall_access.log combined /VirtualHost You could say the 000- at the front of the file name is the whole setup. Digits sort before letters, so this file is always read first. I might look at it later and rename it to something prettier, so I nailed it down with a comment. ServerName catchall.jaeyonging.com isn't an address anyone actually uses. If ServerName is left empty, the vhost inherits the ServerName from the global server config, and on my server that's jaeyonging.com, so it would look like it overlaps with the root domain vhost. To avoid confusion I gave it a separate name that isn't used anywhere. All the other vhosts hand requests to a Node server with ProxyPass , but this one has Apache serve an HTML file straight from disk. I did that on purpose. If the landing page depended on a Node process, it would go down whenever the server was having trouble. As it is, the page still shows up even if every PM2 process is down. ProxyRequests Off keeps this vhost from being used as a forward proxy. It's Off by default so it doesn't need to be written, but I spelled it out. In fact, requests like GET http://azenv.net/ do show up in the log. They are probes checking whether my server can be used as a relay to someone else's site. Where I got confused once: file order I ran ls to check the order, and it came out differently than I expected. $ ls /etc/apache2/sites-enabled | head -4 000-catchall.conf 000-catchall-ssl.conf blog.jaeyonging.com.conf blog-ssl.conf $ LC_ALL=C ls /etc/apache2/sites-enabled | head -4 000-catchall-ssl.conf 000-catchall.conf blog-ssl.conf blog.jaeyonging.com.conf ls sorts by locale rules, which mostly ignore punctuation like hyphens and dots when comparing. Apache, on the other hand, sorts by raw byte values. A hyphen comes before a dot, so in reality blog-ssl.conf is read before blog.jaeyonging.com.conf . To see the order Apache sees, you need to add LC_ALL=C . This time the port 80 files and the port 443 files didn't overlap, so the result was the same, but if you build a setup that relies on name order you should know about this difference. Trusting plain ls to line up the order could easily leave the wrong vhost as the default. HTTPS with the wildcard certificate At first I thought handling port 80 alone might be enough. But the root domain config has HSTS set with includeSubDomains . Any browser that has visited jaeyonging.com even once will open every subdomain over https only from then on. For those browsers an http landing page is simply never seen. So I made a catchall for 443 as well. # /etc/apache2/sites-available/000-catchall-ssl.conf (excerpt) VirtualHost *:443 ServerName catchall.jaeyonging.com SSLEngine on SSLCertificateFile .../_wildcard_.jaeyonging.com_20260201F68CA.all.crt.pem SSLCertificateKeyFile .../_wildcard_.jaeyonging.com_20260201F68CA.key.pem DocumentRoot /var/www/catchall DirectoryIndex index.html ErrorDocument 404 /index.html Header always set X-Robots-Tag "noindex, nofollow" /VirtualHost The certificate is the same *.jaeyonging.com wildcard the other services use. Whether it's asdfas or helo, any single level subdomain matches the certificate. But a wildcard covers exactly one level. Add one more dot and the certificate no longer matches, so the request never even reaches the landing page. $ curl -sS https://a.b.jaeyonging.com/ curl: (60) SSL: no alternative certificate subject name matches target host name 'a.b.jaeyonging.com' There's no way around this short of getting a separate certificate, so I decided to leave it. Two level typos almost never happen anyway. While checking this, though, I found something else. The blog config has ServerAlias www.blog.jaeyonging.com in it, and that is also two levels, so for the same reason it can't be reached over https. For months I had an alias that nobody could ever get into. It's a line that came along when I made the config by copying another vhost, and I plan to remove it the next time I touch Apache. What status code to return This is the part I spent the longest on. The address doesn't exist, so returning 404 for everything seems right. But the subdomain itself is in DNS and the server responds normally. I saw the root of this subdomain as closer to a "landing page" than a "page that doesn't exist". So the root ( / ) returns the landing page with a 200, and requests with any path below it return a 404 while still showing the same landing page as the body. The single line ErrorDocument 404 /index.html does that. When a file that doesn't exist is requested, Apache keeps the 404 status code and only swaps the body for index.html. Of course, returning 200 at the root makes it something like a soft 404 from a search engine's point of view. With wildcard DNS, addresses can be made up endlessly, so once they start getting indexed there's no cleaning it up. That's why I added X-Robots-Tag: noindex, nofollow as a response header. I also put a robots meta tag in the HTML, but setting it as a header also covers responses that aren't HTML, like the favicon. There's a reason I wrote it as Header always too. Without always , the header isn't added to error responses like a 404. $ curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" https://asdfas.jaeyonging.com/ 200 https://asdfas.jaeyonging.com/ $ curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" https://asdfas.jaeyonging.com/abc 404 https://asdfas.jaeyonging.com/abc $ curl -sI https://asdfas.jaeyonging.com/ | grep -iE '^(HTTP|x-robots|content-type)' HTTP/2 200 x-robots-tag: noindex, nofollow content-type: text/html The page is a single static HTML file The layout is simple. A 404 notice, the address you came in on, and cards for the services that are actually running. Showing the address looks minor, but it's quite useful. People who made a typo usually don't know what they typed, so putting the address right in front of them shows them at once where it went wrong. The address isn't inserted by the server but by JS. It's a static file, so there's no server side template. Turning on mod_include would let the server insert it, but I didn't want to enable another module just for this one page. div class="host" id="host" jaeyonging.com /div script (function () { var el = document.getElementById('host'); if (el location.hostna