|I think you misunderstand what DNS is and what it does.|
Our DNS assigns hostnames for some internal servers in the office.
Wrong. DNS provides hostname to IP address resolution it doesn't 'assign' anything to anything. Say your server's name is Server1 and it's IP is 192.168.1.100
DNS resolves "Server1" to "192.168.1.100" so you can, in a command prompt window, type something like ping Server1 and it will respond to the ping. Also, when mapping a drive you would map to \\Server1\share. You could also map: \\192.168.1.100\share but "Server1" is easier to remember than an IP address.
What I want to do is connect to the VPN, and have those hostnames available to me from home.
Providing the servers are in the same physical/logical network and their DNS host records are correct (ie: show the correct server name to the correct IP address) then it should work properly. I think maybe you have some misconfiguration with your DNS server and it's host records pertaining to other servers within your environment.
I know that where I work, once you've VPN'd into our network, all servers and resources available to a user (based on their domain username and what, if any, resources they have available to them) are available to the user as if they were logged into their work computer in their office.