If your case will take a standard ATX style mobo, you can put nearly anything in it. You will likely need to upgrade your power supply to handle the more modern cpus. Most power supplies for ATX come as certified and ready for most any cpu today, including Athlon XP and pentium 4. Even should you decide to go with something via an auction site, like eBay, you can get some good deals.
If you are thinking that you might wish to try Linux at some time, You will need to go to a distro's website and check hardware compatibility, starting with mobo and cpu. That way you will at least have those 2 components as linux ready. I recently got a Gigabyte GA-7IXEH F2 mobo, which has ata100 onboard eide. It supports Durons to at least 1.2 Ghz and Athlons to at least 1.4 Ghz. Both are socket A. I got my 1.1 gig athlon via eBay for $65 US and arctic silver 3 paste the same way. I currently run win98se and Mandrake Linux 8.1 Powerpack on it with no problems.
after you are upgraded, and all runs smoothly, you can sell your old mobo and vid card on eBay or other method. I got my mobo locally, new at retail.
ATX has those ports on the back, while AT has normally only the AT style keyboard port on the mobo, with all else via cable. That will tell you if you need to get a new case as well. If you need to, you may as well get something with a lot of drive bays. Mine has 6 x 3.5 and 6 x 5.25, ATX full tower. You may also need to get new ram, especially if yours has a small chip on them (compaq proprietary and won't work in standard boards). If your ram is pc66, you will definitely need new ram. Be sure of what you get, as there are alot of choices out there. I also have a Tyan Trinity K7 (S2380) Slot A for classic athlon only and a 900 mhz athlon to go with it. Still need a few things to get a 2nd system together. It is a sweet board and cpu. I prefer to have a mobo with a full complement of expansion slots, not including CNR and AMR, which limits my choices. Don't rope yourself into a board with builtin sound and/or video. ATX max of 7 slots and AT max of 8 slots. Depending on HP's OS distro policy, you may have to get a new OS (windows), as some OEMS only give you a restore/rescue CD, not the full OS. In any case, copy all of your critical files to cd-rw or cd-r, or a 2nd HD, in case you need to install the OS from scratch.
A sure method to sort out your choices, is to settle on video card and cpu, then find a good mobo to match, then ram. My Gigabyte mobo has 1 AGP 4x, 5 pci, and 1 ISA shared, for 6 usable slots, but it only handles 200 Mhz FSB AMD Socket A cpus (another factor), though it uses pc133 sdramm dimms, up to 1.5 GB. Some Celerons are limited to pc66 ram, which most newer boards do not support. You can still get win98se in retail pack, and oem units via eBay.
In any case do some research and look around before deciding on everything. Save money where you can do so safely. Don't forget to measure your PSU to see if it is a standard size, IF ATX type (if you can use current case). If you got one of those dinky HP cases, get a new one, to leave room for future addons. Remember this OEM rule: Upgrade = NEW computer! (to generate sales).
Have fun and very happy PC building!
Charles