|Certainly not a pen drive as they are not reliable enough and are therefore only suitable for short-term storage.|
DVD media is the best option.
The number of pics you have is not the real issue, it's their total size in megabytes which will determine how many blank DVDs you will need, but for arguments sake let's say each JPEG file is 500 kilobytes:
500 Kb x 4000 = 2,000,000 Kb = 1,954 MB = 1.9 GB
In the above example, your 4000 pics will easily fit on a single-layer 4 GB DVD.