Kill Dup lines

Bootp WINDOWS XP
June 29, 2010 at 09:51:37
Specs: Windows XP, 2g
Hey guys got another one.

I have a txt file that has single word lines
example:

file1:
--------
car
toy
boat
toy
dog
cat
hat
dog
apple
hat

-----

I would like to simply remove the duplicate entires in the file. After reading the IF info /? pages, I attempted to write a script but its not working for reason. below is what I have:

set compare=""

for /f in %%a (c:\stuff.txt) do (
if NOT %%a==%Compare% echo %%a >> C:\new.txt
if NOT %%a==%Compare% set Compare=%%a
)
----------------------

I went to look at the new file it was supposed to create but nothing is there so im sure my code is messed up somewhere.

thanks in advance for any advice!

Adam


See More: Kill Dup lines

Report •


#1
June 29, 2010 at 10:32:42
Here's an example that uses FIND, and the resulting
errorlevel (0 found, 1 not found), to populate a temp file.
The MOVE command replaces the original file at the end
with the temp file:

@ECHO OFF

TYPE NUL>"\Your\Path\To\stuff.tmp"
FOR /f "usebackq tokens=* delims=" %%a IN ("\Your\Path\To\stuff.txt") DO (
	FIND "%%a" < "\Your\Path\To\stuff.tmp" > NUL
	IF ERRORLEVEL 1 ECHO %%a >> "\Your\Path\To\stuff.tmp"
)

MOVE /Y "\Your\Path\To\stuff.tmp" "\Your\Path\To\stuff.txt"
EXIT /B

See if that will do it for you!

Insomniac at large


Report •

#2
June 29, 2010 at 18:06:37
hi O!

this may or may not work, depending on a whole crap load of things (ie: lots of weakensses)

@echo off >new.txt & setlocal enabledelayedexpansion
for /f "tokens=*" %%a in (stuff.txt) do (
set xx=%%a
if not defined !xx! >>new.txt echo %%a & set !xx!=1
)
more new.txt
::--------
each line of the file becomes its own variable-identity, so memory and batch constraints might shut it down, also "special" char.s in the file will inevitably blow it up.


Report •
Related Solutions


Ask Question