|"Am I just being paranoid?"|
No, just cautious. You say you were 100% it was turned off. There could be a few explanations: 1) When you clicked the setting to turn it off, perhaps you closed the web interface or went to another property page without "applying" the changes, 2) perhaps the router was reset at some point in time to factory defaults? or 3) you (or someone else) changed that setting.
You should change your password for your router to a very strong password. And, if you don't need it, ensure that remote administration is disabled. It may be listed under a different name, but it is the setting that allows connections from the internet to administrate your router.
By the way, your ISP should be able to know exactly where those requests were coming from based upon the IP address making the request. Perhaps the "DHCP request success" is your router making a DHCP request from your IP and this is nothing to be worried about. And, if it is coming from another subscriber, it may be unintentional. They may be infected with something that is trying to access network resources.