Delete duplicate lines

Lenovo / 6073ae8
May 4, 2010 at 06:32:54
Specs: Microsoft Windows XP Professional, 2.328 GHz / 2005 MB
I would like to have a batch file to delete duplicate lines after this batch file generates it's output:
[code]del dates_found.txt
findstr /c:" 1996 " _99-dbug.txt >>dates_found.txt
findstr /c:" 1997 " _99-dbug.txt >>dates_found.txt
findstr /c:" 1998 " _99-dbug.txt >>dates_found.txt
findstr /c:" 1999 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2000 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2001 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2002 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2003 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2004 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2005 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2006 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2007 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2008 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2009 " _99-dbug.txt >>dates_found.txt
findstr /c:" 2010 " _99-dbug.txt >>dates_found.txt
[/code]

The output of the above batch is this (it's pulling driver names and dates from a Windows Debugging session):
[code]8b5f2000 8b5f2680 giveio giveio.sys Wed Apr 03 22:33:25 1996 (316334F5)
8b5f2001 8b5f2682 giveio giveio.sys Wed Apr 03 22:33:25 1996 (316334F5)
8b5d8000 8b5d9480 speedfan speedfan.sys Sun Sep 24 09:28:47 2006 (4516880F)
90f4f000 90f57000 FwLnk FwLnk.sys Sun Nov 19 09:11:12 2006 (45606600)
9dab8000 9dac2000 secdrv secdrv.SYS Wed Sep 13 09:18:32 2006 (45080528)
90f4f000 90f57000 FwLnk FwLnk.sys Sun Nov 19 09:11:12 2006 (45606600)
9dab8000 9dac2000 secdrv secdrv.SYS Wed Sep 13 09:18:32 2006 (45080528)
8b5d8000 8b5d9480 speedfan speedfan.sys Sun Sep 24 09:28:47 2006 (4516880F)
90f6d000 90fbe000 rixdptsk rixdptsk.sys Wed Mar 21 09:02:02 2007 (46012CCA)
91dc1000 91dd0000 rimmptsk rimmptsk.sys Sat Feb 24 00:42:21 2007 (45DFD03D)
91dd0000 91de4000 rimsptsk rimsptsk.sys Tue Jan 23 02:40:19 2007 (45B5BBE3)
91dc1000 91dd0000 rimmptsk rimmptsk.sys Sat Feb 24 00:42:21 2007 (45DFD03D)
91dd0000 91de4000 rimsptsk rimsptsk.sys Tue Jan 23 02:40:19 2007 (45B5BBE3)
90f6d000 90fbe000 rixdptsk rixdptsk.sys Wed Mar 21 09:02:02 2007 (46012CCA)
91600000 91be8000 atikmdag atikmdag.sys Mon Dec 01 16:01:46 2008 (493450BA)
91d4c000 91d7c000 SynTP SynTP.sys Thu Aug 14 22:02:22 2008 (48A4E3AE)
98c01000 98d07000 AGRSM AGRSM.sys Mon Nov 10 09:56:37 2008 (49184BA5)
98c01000 98d07000 AGRSM AGRSM.sys Mon Nov 10 09:56:37 2008 (49184BA5)
91600000 91be8000 atikmdag atikmdag.sys Mon Dec 01 16:01:46 2008 (493450BA)
91d4c000 91d7c000 SynTP SynTP.sys Thu Aug 14 22:02:22 2008 (48A4E3AE)
80b9e000 80ba6000 kdcom kdcom.dll Mon Jul 13 21:08:58 2009 (4A5BDAAA)
82a10000 82a47000 hal halmacpi.dll Mon Jul 13 19:11:03 2009 (4A5BBF07)[/code]

We do not care if the first 3 columns are identical or not (the first 2 are memory addresses that may change) - but we need to remove any otherwise identical lines


See More: Delete duplicate lines

Report •


#1
May 4, 2010 at 09:01:16
not tested

================================
@echo off > newfile & setLocal enableDELAYedeXpansion

for /f "tokens=1-2* delims= " %%a in (myoutput) do (
find "%%c" < newfile > nul
if errorlevel 1 >> newfile echo.%%a %%b %%c
)


=====================================
Helping others achieve escape felicity

M2


Report •

#2
May 4, 2010 at 12:47:16
It works great! Thanks for your help!

Report •

Related Solutions


Ask Question