Batch file for removing dupes

Microsoft Microsoft windows 2000 scripti...
September 20, 2009 at 10:39:24
Specs: Windows 7, Q6600 4Gb RAM
Hi,

I'm trying to workout how to code a batch file
which can sit in a folder and when run, scan
every text file in the same folder and remove
duplicate lines of text. I've seen several
versions on computing.net but unfortunately
none work in my instance, as I would like to
run it against all files in the same directory,
not just a single file.

I found this posted on computing.net already but can't work it... I also want to be able to just update the existing file, not output to a new file - if at all possible;

===== BATCH SCRIPT BEGIN =====
:: Leonardo Pignataro (secret_doom@hotmail.com)
@echo off
if not "%1"=="" goto chkext
echo Missing parameter
echo Sintax: %0 filename
goto eof
:chkext
if exist %1 goto start
echo File not found - %1
goto eof
:start
type nul> %TEMP%.\TEMP.DAT
FOR /F "delims=" %%a in (%1) do (
FIND "%%a" < %TEMP%.\TEMP.DAT > nul
if errorlevel=1 echo %%a>> %TEMP%.\TEMP.DAT
)
copy/y %TEMP%.\TEMP.DAT %1 > nul
if exist %TEMP%.\TEMP.DAT del %TEMP%.\TEMP.DAT
:eof
===== BATCH SCRIPT END =====

Can anyone help? I'm not a good coder and
I've searched high and low for a script already
available but to no avail.

Regards

Alex


See More: Batch file for removing dupes

Report •


#1
September 20, 2009 at 13:38:33
I am honoured to modify secret_doom's script, the ultimate Master of batch scripting, who guided me to that art.

:: NODUP.BAT  Usage: nodup
@echo off
echo.
echo.  Processing .txt files in "%CD%"
echo.  Please wait...
for %%j in (*.txt) do (
  type nul > %TEMP%.\TEMP.DAT
  for /F "delims=" %%a in ('type "%%j"') do (
    find "%%a" < %TEMP%.\TEMP.DAT > nul
    if ErrorLevel 1 echo %%a>> %TEMP%.\TEMP.DAT
  )
  copy %TEMP%.\TEMP.DAT "%%j" > nul
  if exist %TEMP%.\TEMP.DAT del %TEMP%.\TEMP.DAT
)
echo.  DONE!
:: End_Of_Batch


Report •

#2
September 21, 2009 at 05:44:49
Wow, thats a great response and the batch as you already
knows works flawlessly. I must learn to batch code myself.
Excellent job IVO I really appreciate it.

Report •

Related Solutions


Ask Question