Yep, for webmail you are quite correct. They are deleted at the provider end and not stored on the computer at all.
For Windows Live Mail, Thunderbird, the old Outlook Express, etc (onboard programs) they are both are held at and deleted from the computer. Most servers have a duplicate webmail arrangement too. You have a local option to retain email copies at the server's webmail inbox as well (from any machine) but that is not usually the default setting - so emails normally vanish from the server's inbox once downloaded onto the computer.
The "retain at server" option is useful if you have more than one computer with onboard email sharing the same email address. For example, you could only let the main computer delete emails at server but still allow your portable to download them to look at or answer when you are out and about.
This lot underlines the importance of knowing what the poster is using.
Servers keep "hush-hush" email copies of their own (both directions) in case the cops need them - policies varying from country to country. I doubt they would ever be made available to the customer tho - never tried.
Sorry if I've just told you what you already know (a failing of mine LOL).
Always pop back and let us know the outcome - thanks