|With a CSV file it is assumed the records are of random |
length. That precludes using random access. I would convert
it to random access by finding the largest size of each field
and creating a random file by padding each field shorter than
the largest value. Using random number generator select the
records Records reads 36,020,000 writes 18,000,000 versus
est. 180,000,000,000 reads if random access file not created..
Additional 20,000 writes needed to save results.
This is done by using a program to read, write file and select
records. There is almost a 1 chance in 1000 of getting a
duplicate record. Selecting a unique record will increase the
time depending upon the selection criteria.