|In general I agree, the E6320 would be better. Here is your side-by-side on the two:|
I would recommend that you verify that your system can use (support) this processor through the manufacturer's CPU support list or by contacting their tech support since you did not mention the machine and/or the board. Further if you are still using the onboard graphics you would gain as much or better performance just by adding a gaming grade video card that fits your board's graphics expansion slot, changing both should make the system significantly better. Further, if you have not yet, adding more memory (depending on how much you have) can also improve performance greatly if you have only near the minimum required like many systems sold at the time yours probably was. Understand that IF you change/add a graphics you may also need to upgrade your power supply to support it depending on what you choose and what was supplied. For detailed information on this you need to post your exact system specs including any upgrades you may have already made.
You have to be a little bit crazy to keep you from going insane.