since I have several files to run this through I notice that when I use For it crashes.

DIRS = "at_1 be_1 ch_1"
for j in $DIRS
do
gawk -F\| 'BEGIN {
while ((getline < ${j}_main.txt) > 0)

does not like ${j}_main.txt in the script- will not go pass this line-- everything from there on is not parsed right or something

the original script works fine on one file but I need to run it on multiple files.

gawk -F\| 'BEGIN {
while ((getline < "main.txt") > 0)
{w4=sprintf("%2.2d",$4)
mainw5[w4 ":" $2] = $5}
#print "===== mainw5 array ====="
#for (i in mainw5)
# print i, mainw5[i]
#print "======================"
OFS="|"
}

{w2=sprintf("%2.2d",$2)
key=w2 ":" $3
if (key in mainw5)
print $1,$2,mainw5[key],$4,$5,$6,$7,$8
else
print
}' new.txt > new.modified

How do I use the above script to continue to the next file once it has finished the first file

Thanks

the gawk's option --assign might be helpfull

bash/ksh/sh paramater substitution

gawk -F\| 'BEGIN

' - no shell $x substitution
" - shell substitution; but, will probably result in other problems with gawk script

Variable assignment in most shells is without spaces around the equal-sign:
DIRS="at_1 be_1 ch_1"

You will notice that my "main.txt" is in double-quotes so that awk will see it as a literal string. Unquoted, awk will treat it as an undeclared awk variable.

There are several ways to feed a shell variable into awk. One way is:

while ((getline < "'${j}'_main.txt") > 0)

It appears that we are enclosing ${j} within single quotes, but in fact we are placing it OUTSIDE of the single-quoted awk program, thus allowing the shell to evaluate it on the command line before awk executes. If we look at the big picture all on a single line:

awk 'some code'${j}'some more code' new.txt

then you can see that we have a single quoted string followed by ${j} followed by another single quoted string.

But your for-loop is not going to do what you want. It will run awk 3 times, and each time awk will read new.txt and will create new.modified, each time replacing the previous new.modified. The new.modified that you end up with will be the one processed on the 3rd for-loop with ch_1_main.txt.

Making awk append each time to new.modified (>> new.modified) will just give you a new.modified that has 3 copies of new.txt in it, one copy processed with at_1_main.txt etc.

I think you are wanting new.txt to be processed once, pulling conversion info from all main.txt's, correct? If so, one way would be:

rm -f combined_main.txt 2> /dev/null
DIRS="at_1 be_1 ch_1"
for j in $DIRS
do
cat ${j} >> combined_main.txt
done

gawk -F\| 'BEGIN {
while ((getline < "combined_main.txt") > 0)
etc

And if you do not want to create a temporary combined main.txt, it could be coded so that awk reads all main.txt's individually.

Report Offensive Follow Up For Removal