Remove Duplicate Lines of text

November 16, 2009 at 02:23:53
Specs: Windows XP
Hi,

Im trying to create a simple batch script which i can leave running as a scheduled task and will build an output file containing a list host names !

Iv already got a script dumping the result of netstat into a text file, and iv already managed to get a filter part working to trim just the hosts connected on a selected port.

But im planning to leave this script running for some time, and want it to just accrew the unique hostnames, not keep appending the file and adding loads more entries.

Anyone have any ideas how i can get a batch script to run through text file output from my trimming script and trim out all the non-unique hosts.


See More: Remove Duplicate Lines of text

Report •


#1
November 16, 2009 at 02:54:30
provide sample of input files, and show your output.

GNU win32 packages | Gawk


Report •

#2
November 16, 2009 at 03:05:12
The outbut as it currently stands is this ;

TCP WSKAPS1:4021 WSK606S14J:2838 ESTABLISHED 1604
TCP WSKAPS1:4021 WSK606S14J:3099 ESTABLISHED 1604
TCP WSKAPS1:4021 WSK106S14J:1064 ESTABLISHED 1604
TCP WSKAPS1:4021 WSK106S14J:1105 ESTABLISHED 1604
TCP WSKAPS1:4021 WSK406S14J:1069 ESTABLISHED 1604
TCP WSKAPS1:4021 WSK406S14J:1127 ESTABLISHED 1604

I basically just need the part saying "WSK406S14J"


Report •

#3
November 16, 2009 at 04:38:01
Perhaps prevention is better that cure?

If your about to append to the file maybe you could use "find" on the text file with either the whole line your about to add or just the host name. Then you could check errorlevels and only append if the entry doesn't exist.


Batch Variable how to


Report •

Related Solutions

#4
November 16, 2009 at 07:03:31
@echo off > newfile & setLocal enableDELAYedexpansion

for /f "tokens=3 delims= " %%a in (myfile) do (
set S=%%a
set S=!S:~0,-5!
find "!S!" < newfile > nul
if errorlevel 1 (
>> newfile echo !S!
)
)


=====================================
Helping others achieve escape felicity

M2


Report •

#5
November 18, 2009 at 01:57:46
Works, mostly... any chance you can explain out your code a little, my batch is (rusty would be an understatement)...

Report •

#6
November 18, 2009 at 05:56:48
tokens= tells it which column

set S=!S:~0,-5! drops the last 5 chars


=====================================
Helping others achieve escape felicity

M2


Report •


Ask Question