Solved Adding data columns in c shell

March 26, 2012 at 01:02:37
Specs: Windows 7
I'm trying to write a c shell script to add and subtract columns. I'm sure there's an easy way to do this, but I'm not getting much help from google, and my attempts are just producing errors. Could someone point me in the right direction?

This is what I have just as a test run. Note: first row of test.dat is text headers.

set col1 = `awk '{print $1}' test.dat`
set col2 = `awk '{print $2}' test.dat`
set col3 = `awk '{print $3}' test.dat`
echo $col1
echo $col2
echo $col3
set var1[1] = $col2[1]
@ var1[2] = $col2[2] + $col3[2]
echo $var1

guest


See More: Adding data columns in c shell

Report •

#1
March 26, 2012 at 10:05:08
✔ Best Answer
If you just want to skip line one, how about something like this:

#!/bin/csh

if ( -e test.dat) then
   set x = 0
   foreach line ("`cat test.dat`")
      # skip the first line
      @ x = ($x + 1)
      if ( $x == 1 ) then
         continue
      endif
      set col1 = `echo "$line"|awk '{print $1}'`
      set col2 = `echo "$line"|awk '{print $2}'`
      set col3 = `echo "$line"|awk '{print $3}'`
      echo $col1
      echo $col2
      echo $col3
      @ myvalue = ($col2 + $col3)
      echo $myvalue
   end
endif

Probably a better way exists for parsing the line in csh but I don't have time to research it. (BTW, the csh is poor. Use it only when you have to).


Report •

#2
March 26, 2012 at 19:29:08
That's great! Thanks!

It works well, however if the numbers in the variables col2 or col3 are decimals then I get an error. Any idea how to correct this?

guest


Report •

#3
March 27, 2012 at 10:47:10
The csh does not support float arithmetic. Use another unix tool for arithmetic such as bc or awk:

#!/bin/csh

set a = 5.21
set b = 7.33

# add two vars together
set mystr = "$a $b"
set myvalue = `echo "$mystr" | awk ' { printf("%.2f", $1 + $2) } '`

echo $myvalue


Report •

Related Solutions

#4
March 28, 2012 at 19:52:07
Hi, thanks again for your help! It's been very useful to help write my program.
I thought I'd share my final solution.

This code will perform column arithmetic and output the result with the same headers as used in the input file. (Note: not the entire program, hence why $file isn't defined, etc.)

The MATHS alias is just something I copied and pasted off another website, so I don't understand what it's doing, only that it works. :P Essentially I'm using it to replace set or @, but with the ability to handle decimal numbers.

alias MATHS 'set \!:1 = `echo "\!:3-$" | bc -l`'

set x = 0

foreach line ("`cat $file`")
set col1 = `echo "$line"|awk '{print $1}'`
set col2 = `echo "$line"|awk '{print $2}'`
set col3 = `echo "$line"|awk '{print $3}'`
if ($x > 0) then
MATHS col2 = $col2 + $col3
endif
set data = ($col1 $col2 $col3)
echo $data
@ x = ($x + 1)
end

guest


Report •

#5
March 29, 2012 at 01:42:58
Final question:

I've got my entire program (not just the above code) working completely, 100% debugged, and my only complaint is regarding a rather excessive computational time. Is there any way to bring this down, or is this as good as it's going to get? I believe it's the contents of the foreach loop above that is the major contributor to the running time.

guest


Report •

#6
March 29, 2012 at 07:37:45
The reason the script is slow because you are calling awk 3 times. This script would be easy if you weren't using the csh (IMO the csh totally sucks!)

If it where me, I would do the addition in awk and cut down on the number of calls:

set data = `echo "$line" | awk ' { print $1 $2 $3 } '`

You could put that if statement within awk call.


Report •

#7
April 3, 2012 at 01:01:58
I believe you've made a typo in that code but I don't know enough about awk to be able to find it. Could you correct it for me?

*edit* Never mind. It's the ) that has to go. I was having trouble because of other issues which were making diagnostic difficult.

guest


Report •

#8
April 3, 2012 at 06:58:04
You are right and I apologize for that. I edited the code and removed the parenthesis.

Let me know if you have any other awk questions. I will try NOT to mess up!


Report •

#9
April 4, 2012 at 01:47:26
"I will try to mess up!" lol. NOT. I think you meant "I will try NOT to mess up!" lol at the irony. But anyway, I've made some significant modifications to the code, pretty much rewrote the entire thing in the end, adding in some of your suggestions, added in some functionality, and at the end of it found I had about a 50-70% reduction in computational time! The main part of the program had been reduced down to a single line as well!

Below is my final solution (again, not the entire program, just this section of it).

This will write files with values taken from and computed from those in the original file, and with the same headings as used in the original.

set x = 0
set out1 = ${file}+err
set out2 = ${file}-err

# Ensures data is written to new file, rather than appended to old.
if (-e $out1) then
rm -f $out1
endif
if (-e $out2) then
rm -f $out2
endif

foreach line ("`cat $file`")
if ($x == 1) then
echo "$line"|awk '{printf($1 " %.1f " $3 "\n",$2 + $3)}' >> $out1
echo "$line"|awk '{printf($1 " %.1f " $3 "\n",$2 - $3)}' >> $out2
else
echo $line >> $out1
echo $line >> $out2
set x = 1
endif
end

echo "Data file generated:" $out1
echo "Data file generated:" $out2

end

guest


Report •

#10
April 4, 2012 at 11:19:05
To make up for my clumsiness, I have two suggestions for you:

First, combine the two awk commands into one by printing to seperate files in one awk script:

#!/bin/csh

set out1 = "file1.err"
set out2 = "file2.err"

echo "7 8 9" | awk ' { printf($1 " %.1f " $3 "\n",$2 + $3) >> "'"$out1"'" ; prin
tf($1 " %.1f " $3 "\n",$2 - $3) >> "'"$out2"'" } '

Above, the shell variable file names are embedded in the awk script. This should speed up your script.

This link describes how shell variables can be passed to awk:

http://www.tek-tips.com/faqs.cfm?fi...

Next, it's a small thing, but there is no reason to check for the existence of the file before you delete it:

if (-e $out1) then
     rm -f $out1
endif

Simply remove it:

rm -f $out1

The -f option of the rm command suppresses any error message such as the file is not there.



Report •

#11
April 23, 2012 at 19:49:00
Sorry for taking so long to get back to you. Your suggestions were very useful and I've implemented them both. The suggestion regarding awk resulted in a 45% reduction in running time!

With all the different changes made since my last major version, quoted in Comment #4 here, I have a 70-80% reduction in running time.

Thanks again for all your help. You have been central to this program's development!

guest


Report •

#12
April 24, 2012 at 14:08:31
Thank you for the kind words! It's much appreciated.

Report •

#13
April 26, 2012 at 02:18:56
You're welcome!

Incidentally, I ended up removing the rm section of the code, as I found the same function could be achieved by replacing the two sets of >> with > in the 'else' part of the if statement. This meant if the file already existed it would be overwritten with the header line, then the data appended line by line as usual in the first part of the if statement.

guest


Report •

Ask Question