This is the core of the matter. The issue was less one of the router knowing what hostname it's known by, and more one of other hosts knowing how to resolve it's hostname. The fact is that there is nothing on my LAN telling MY computer about the router's hostname/IP.
There's no local DNS server on my LAN, and the router certainly doesn't use Bonjour or any other broadcast method to teach other hosts on the LAN about it's hostname and IP address.
My problem was one of forgetting my fundamentals!
I have not tried your workaround, but one of the techs in my MSP is a Linux head and caught this right away. The ER would need to be the DNS proxy (which is was) but it would ALSO need to know to send requests for it's hostname to it's LAN IP, rather than sending them on to Google's (or the ISP's) DNS servers for resolution - which of course they could never do.
And rather than having some complex DNS routing table, we figured that you'd be able to tell the ER to reference it's own host file, but couldn't quite find the solution. It looks like what you posted might be it.
Beyond that, my tech was actually able to shell into the ER and install a full DNS server in the Debian environment - which actually really impressed us both! These boxes are really versatile and totally open! But in the end, the process of actually configuring the DNS server proved not worth this simple luxury of not typing the router IP when on the LAN
. I mean, it's not like the router changes LAN IP addresses frequently.
Nevertheless, thanks for the exploration!