|The front panel connections shouldn't matter although it's odd it turns on when you connect it to a wall socket.|
Most power supplies provide a small charge to the motherboard even when the PC is off. Since you don't have the power cord always connected maybe that's a factor but I can't say for sure. I think that's just for the 'power on' options but it might also hold a charge if the cmos battery is bad.
The cmos battery only holds the motherboard settings when the PC is off. When it's on the battery is irrelevent as that circuit is charged by the power supply. So if you have that same problem every time you turn it on then the first thing to suspect is a bad battery.
Or possibly the battery is not making a good connection or the solder joints on its holder are broken. Or maybe something is shorting it. Is the motherboard contacting the metal case someplace it shouldn't?
The power button on the front case only has two wires. Can't you locate those and attach them to the power switch pins on the motherboard?
Unless the hp case has a proprietary power supply that's not easily replaced there wasn't any reason to move all your hardware to another case. You could have just put in a different power supply.
You're not really green until you're soylent green.