Computing.Net > Forums > Unix > compare lines within file

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

compare lines within file

Reply to Message Icon

Name: scjdp
Date: January 22, 2004 at 13:43:19 Pacific
OS: unix
CPU/Ram: 5gb
Comment:

ex data
aaa20040110
aaa20040110
aaa20040110
aaa20040111
aaa20040111

File may contain data for 2 dates or single day. I need to find if it has 2 day data or one day data



Sponsored Link
Ads by Google

Response Number 1
Name: thepubba
Date: January 22, 2004 at 19:37:32 Pacific
Reply:

#!/bin/ksh
exec 3< ./junk.data
while read -u3 line
do
if [[ $(print $line | wc -w) -eq 2 ]]
then
print $line
fi


0

Response Number 2
Name: James Boothe
Date: January 23, 2004 at 06:28:01 Pacific
Reply:

Jerry's solution prints each line containing 2 words.

To determine how many unique dates your file contains (keying on word1 in each line), you can isolate that word (with awk or cut, for example), and then do a unique sort which will eliminate duplicates. And this is korn shell syntax, but can be converted for other shells.

k=$(awk '{print $1}' myfile.txt | sort -u | wc -l)
echo "number of unique dates:" $k

The following script will summarize your file on word1:

awk '\
    {date[$1]++}
END {for (i in date)
         print i, date[i]}
' myfile.txt


0

Response Number 3
Name: scjdp
Date: January 23, 2004 at 06:34:06 Pacific
Reply:

Thanks a lot. Next hurdle is I want to know if there are more than 2 dates then how many records for each date.

in above example.
20040110 - 3
20040111 - 2

Once again thanks a lot for replying fast.



0

Response Number 4
Name: James Boothe
Date: January 23, 2004 at 07:49:31 Pacific
Reply:

I'm ahead of you - my first reply provides those answers.

The first script gives you a count ($k) of how many unique dates you have.

The second script provides the summary you are asking for.


0

Response Number 5
Name: scjdp
Date: January 23, 2004 at 10:40:26 Pacific
Reply:

Thanks,

it's working great. it's dumb q. but how can I get summary data into unix shell variable ( or array) and manipulate it.


0

Related Posts

See More



Response Number 6
Name: scjdp
Date: January 23, 2004 at 10:41:30 Pacific
Reply:

and also skip first and last line of file.


0

Response Number 7
Name: James Boothe
Date: January 23, 2004 at 11:51:06 Pacific
Reply:

It is very easy to waste the first line.  awk could do an initial getline to waste it, or tail +1 would do it.  But the last line is much more of a pain.  awk could do that too, but only awkwardly (sorry about that) by holding each line and processing it on a delayed basis.  I took the easy way below and just let sed strip those.

There are several ways to get output from awk (or from any command) back to the shell.  But when the output is any number of lines, you need a construct that can process multiple lines.  Below, the awk output is piped into a while-loop, where each iteration will process one line.

By the way, on some linux, any environment variables established within that while-loop will go away after the while-loop.

sed '1d;$d' myfile.txt |
awk '\
{date[$1]++}
END {for (i in date)
        print i, date[i]}' |
while read date k
do
echo "date=$date k=$k"
done


0

Response Number 8
Name: scjdp
Date: January 23, 2004 at 12:10:13 Pacific
Reply:

I used
set -A record_count `awk 'BEGIN{FS = "|"}{ if(NF!=1) date[$1]++}END {for (i in date) print date[i]}' $fposted`

and then array record_count will have data as i require. also if(NF!=1) got rid of head and tail as they will not have dilimiter(which i did not mention in my post, sorry)

Thanks for all your help
I'd appritiate if you can give me some useful links on net for code sample/ useful articles for unix shell scripting etc


0

Response Number 9
Name: James Boothe
Date: January 23, 2004 at 12:35:42 Pacific
Reply:

Excellent! Looks like before long, you will be replying to the questions instead of asking them.

I do not have a good set of links (sorry) except for sed. But I think the other regulars on this board have some good recommendations.

But I would suggest that you post your question in a new thread because it might not be seen at the bottom of this thread.


0

Sponsored Link
Ads by Google
Reply to Message Icon






Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Unix Forum Home


Sponsored links

Ads by Google


Results for: compare lines within file

Compare Lines within a file www.computing.net/answers/unix/compare-lines-within-a-file/7893.html

comparing line by line www.computing.net/answers/unix/comparing-line-by-line/8214.html

remove top line in file www.computing.net/answers/unix/remove-top-line-in-file/4769.html