I am having some serious issues with my program I think for the most part it works, but it isn't deleting duplicate lines... ANy assistance would be greatly appreciated, or if I am going about everything wrong, some other guidance would be helpful as well. Here is my program #!/bin/sh #Created by Michael Scarpa #Project 2 #this following section allows for the various commands to happen, it takes in $ #and it removes the html, strips out back to and chapter, puts each word on its$ BEGIN {RS=" "} { #makes it all lowercase $0 = tolower($0) #removes punctuation gsub (/[,;:!\?\.\(\)]/,"", $0) #removes back gsub (/back/,"",$0) #removes page gsub (/page/,"",$0) #removes numbers gsub (/[0-9]/,"",$0) #removes blank space sub (/^[ \t]+/, "") } #this section should only print non duplicates { if (data[$0]++ ==0) lines[++count]=$0 } END { for (i=1;i<=count;i++) print lines[i] }