|run a speedtest.net test when good|
run a speedtest.net test when bad
compare the two.
Then call your isp with the info. I suspect it has nothing to do with your network but its the isp.
What is up with your usage of the word "routers" as in many? There should only be one.
Answers are only as good as the information you provide.
How to properly post a question:
Sorry no tech support via PM's