How to remove duplicate entries with awk.

input file:
1
1
2
3
3
4
4

expected output:
2

basically I want to completely remove duplicate entries and print only non duplicate one.

Thanks for the help!

Regards,

it works with
sort <inputfile> | uniq -u

but how to do it with awk?

Regards,

If the data file is already sorted, this works:

cat data.file|uniq -u

I'm assuming that data.file is already sorted:

awk ' {
if(NR == 1)
{
fr=$1
cnt=1
continue
}

if(fr == $1 )
cnt++
else
{
if(cnt == 1)
{
print fr
cnt=1
}
else # skip cause there's more than 1
cnt=1
fr=$1
}
}
END { if(cnt == 1)
print fr
} ' data.file

it works! thanks nails!

Regards,

In a shell scripting class, you would have to explain this:

sort <inputfile> | uniq -u

since

sort -n -u < inputfile

does the same thing. 2 commands should never be used when one will do.

Nails:

UUOC award for this suggestion:

cat data.file|uniq -u

uniq < data.file

Not sure why uniq has a -u flag, since it only returns unique lines if invoked without arguements.

Jerry:

Yup, that's a UUOC alright. I should have picked it up.

You might take a closer look at the original requirement. The command:

sort -n -u data.file

eliminates an adjacent duplicate. uniq's -u option surpresses all repeated lines.