|"I also have a wireless modem connected to my router."|
Can you tell us about this? How is it configured because you really should not have more than on router on a network. It could be that it is configured into bridge mode so that it is just being a Wireless Access Point and not a router.
As for your Internet Light, it should be on if your DSL modem is communicating to your ISP. The fact that it is off even when you have a connection could indicate that the light is burned out which is a shame because it would let us know if the problem is with the phone lines or your network. You could try to ping the modem from one of your computers when the internet is out. If you can ping the modem but not a web site like yahoo.com then the problem is in your phone lines.
Remember DSL is sensitive to other phone on the same line as your DSL modem. If you have voice service in addition to your DSL try disconnecting the phones in your house and see if the internet comes back. If it does try find the particular phone that might be causing the issue and double up the DSL filter on it.