I would appreciate, if some one help me on this.
I have a flat file (fixed length) and I have got position 20 thru 28 SSN number (9char).
I need to take all the SSN number and eleimnate duplicate and put it into a new output file. Please note this is a huge file. So I am looking the most efficient way to do this. Thanks again

Hi:

I'm assuming you want only the 9 characters in a new file. If your datafile is already sorted:

cut -c20-28 datafile|uniq > newfile

If your datafile is not sorted:

cut -c20-28 datafile|sort -u > newfile

Regards,

Nails

Try:

sort -u -k20,28 inFile > outFile

I assume that you don't want a listing of sorted SSAN's. You want a list sorted by SSAN. The above solution will provide you with a sorted list of the values in columns 20-28. If that is what you want, then that is a good solution.

You can also try:

cut -c20-28 inFile | sort -u > outFile

if all you want is a sorted list of SSAN's.

Thanks a lot! This really worked. Now another related question. Now that I got my unique key into a file, I want to take that key and go against multiple files and extract record from them. Note, the files that I am trying to match may contain my key in a fixed position. Say, for file A the key is from 12 to 20 and for file B it is in 0 to 9.
I know I can use grep agains the key file for matching some thing like:

fgrep -f record_key data_out >data_updated

but how do I go against maching those particular fields. Also is there an easy way to go against both the files same time ?
Many thnaks again!!!

See my solution to this last question in your newer posting "Matching files against the key".