Hi.Hope someone can help me. Situation: I have a CSV file which contains 90 columns. I want to delete all columns but retain only 4 columns i.e. col 41,col 9.col 80 and col 74. I also want to rearrange these 4 columns so that they are in that sequence. What I have so far: for i in *.csv do cut -d, -f41,9,80,74 > done/"$i" done When I run the above, nothing happens. Hope someone out there can help me to get this script working. Thanks in advance.

It's not doing anything because you didn't give it input file name $i: cut -d, -f 4,8,9 $i > done/"$i"

Thanks nails. It works now, it removes the unwanted columns and retains only the column I specify but it doesn't rearrange the columns in the sequence I want. I tried this and but nothing happens: awk -F, -v OFS=, '{ print $4,$1,$3,$2 }' Can I combine the awk command above into the loop? so that the script does two things while looping: 1.removes unwanted columns 2.rearranges remaining columns in specified sequence 3.move output to done folder

The OFS should be set in the BEGIN section of the awk script: awk -F, ' BEGIN { OFS="," } { print $4,$1,$3,$2 }' datafile > done/newfile Another way is to leave the OFS alone, and use the printf statement to do the formating: awk -F, ' { printf("%s,%s,%s,%s\n", $4,$1,$3,$2) }' datafile > done/newfile

for i in *.csv do sed '1d' $i cut -d, -f 9,41,70,84 $i awk -F, ' { printf("%s,%s,%s,%s\n", $4,$1,$3,$2) }' $i > done/"$i" done I am very new to shell scripting and you have been a great help. Thank you for helping and guiding me along. So,would the above script do the following? 1.loop through a directory and process all files ending in .csv 2.delete the first line in each csv 3.delete all columns except columns 9,41,70 and 84 in each csv 4.rearranges columns 9,41,70,84 into 84,9,70,41 in each csv 5.outputs processed csvs into /done folder Thanks nails.

No, your script is NOT right. The old unix tools like sed, cut, and awk can not edit files in place. That means you have to use pipes, |, so the output of one file becomes the input of another: for i in *.csv do sed '1d' $i | cut -d, -f 9,41,70,84 | awk -F, ' { printf("%s,%s,%s,%s\n", $4,$1,$3,$2) }' > done/"$i" done Or you have to use temporary files: for i in *.csv do sed '1d' $i > 1.tmp cut -d, -f 9,41,70,84 1.tmp > 2.tmp awk -F, ' { printf("%s,%s,%s,%s\n", $4,$1,$3,$2) }' 2.tmp > done/"$i" done Finally, have you thought about solving the entire problem in awk? If the number of fields is greater than 3 (skip the blank lines) print the four fields in whatever order you want. the only problem I see is if your awk version will not support that large number of fields: # untested for i in *.csv do awk -F, ' { if(NF > 3) printf("%s,%s,%s,%s\n", $9,$41,$70,$84) }' $i > done/"$i" done

Thanks nails. I tried it but it jumbles up the columns wrongly. The script did delete the first line of every csv file, kept only the wanted columns i.e. 9,41,70,84 but it did not rearrange the columns correctly. It didn't rearrange them into the correct column sequence which is 84,9,70,41(i.e. $4,$1,$3,$2 )