|Yep, you could use normal Ethernet patch cable (straight cable) to connect from computer's network port to router's LAN port. After that you need to do some configuration on network adapter and router (if it's not configured yet) in order to make them work together. |
If you need more ideas, could have a look on this wired home network setup tutorial.
Hope this helps.