|"I guess I will order a new CPU then and see if that works"|
Why would you buy a CPU? You're not even sure if it's the problem. You already bought a bunch of unnecessary stuff. You need to work on your troubleshooting skills.
Start by examining the board. Look for swollen or leaking capacitors. If you find either, the board is shot. Here's some examples: bulging caps, leaking caps
There's no easy way to test a power supply. Multimeters are only slightly helpful. They can confirm that voltages are within spec (or not) but not amperage. Just because a PSU is putting out enough power to spin a few fans doesn't necessarily mean it's putting out enough to power an entire system. Try connecting a "known to be good" PSU.
And try stripping the system down to the bare necessities & doing a benchtest: How to Bench Test, Build, and Troubleshoot Your Computer