|Plug an Ethernet cable from the router into one of the ports of the switch. If the lights light then you are in business. All ports have auto-up-link capabilities.|
This assumes that what you have is in fact a router with NAT and DHCP built in and not just a modem. If it doesn't have NAT it wont work.
A model numbers of the router would be useful as then a definitive answer could be given.