|Sorry to hear about your woes Lotts.|
Things like this are difficult to pin down without actually seeing the machine in action, but having had similar issues in the past, here's what I have found to most often be the case:
1. A component is overheating. For instance, have you dislodged the CPU cooler when you were installing the new hardware?
2. You never mentioned anything about the mobo. Given that you have replaced other devices (and your original graphics card did bite the dust), I wonder whether the problem might be in the mainboard itself?
3. There is a configuration issue. Perhaps your components are not designed to work together? Check each component against the spec in the manual (PSU and mobo included). Then make sure that the BIOS utility is set up to boot with settings designed for these devices.
As I say, there could be any number of issues here. If you can save yourself from tearing your hair out then you will find the answer.
Take everything out of the PC besides the absolute minimum to get it running, and I include graphics card if your mobo has in-built (sorry, haven't checked your mobo to determine this). When you are comfortable that the basic devices are booting up, place each of your additional components in 1-by-1 until the problem recurs.
Note: Also check your power cable and use an alternative is possible. It's a long shot, but has happened to me before.
Good luck and please post back with your findings.