|It could be the PSU or it could be the motherboard. |
On the motherboard you will see some coils of fairly thick copper wire wound round an iron core. These are called chokes and and help to eliminate interference. There are some in the power supply as well.
If the windings is a bit lose then they can start to vibrate at a high frequency causing the whine. The noise is likely to get louder as the computer warms up and the copper expands. You can test for this if you touch them with an insulated screwdriver, the whining should stop. You can also get a similar noise front the core of transformers. As the noise stops when you pull the plug from the wall suggests that it is the power supply at fault.
If you can stand the noise you can ignore it as it isn't doing and harm.