|Your overclock settings are a mess. What happens if you load the BIOS defaults or use the clear CMOS jumper? Does the system settle down & run stably?|
As for the RAM, the memory controller in AMD AM3 CPUs supports up to DDR3-1333. You're not losing anything by running your DDR3-1600 or 1800 RAM at that speed. In fact, by underclocking your RAM, you should be able to tighten up on the timings & pick up a little performance boost.