Computing.Net > Forums > Windows XP > Batch file for removing dupes

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Over 90% answered within 24 hours! Click here to start participating now! Also, be sure to check out the New User Guide.

Batch file for removing dupes

Reply to Message Icon

Name: hawkinsa21
Date: September 20, 2009 at 10:39:24 Pacific
OS: Windows 7
CPU/Ram: Q6600 4Gb RAM
Product: Microsoft Microsoft windows 2000 scripting guide
Subcategory: General
Comment:

Hi,

I'm trying to workout how to code a batch file
which can sit in a folder and when run, scan
every text file in the same folder and remove
duplicate lines of text. I've seen several
versions on computing.net but unfortunately
none work in my instance, as I would like to
run it against all files in the same directory,
not just a single file.

I found this posted on computing.net already but can't work it... I also want to be able to just update the existing file, not output to a new file - if at all possible;

===== BATCH SCRIPT BEGIN =====
:: Leonardo Pignataro (secret_doom@hotmail.com)
@echo off
if not "%1"=="" goto chkext
echo Missing parameter
echo Sintax: %0 filename
goto eof
:chkext
if exist %1 goto start
echo File not found - %1
goto eof
:start
type nul> %TEMP%.\TEMP.DAT
FOR /F "delims=" %%a in (%1) do (
FIND "%%a" < %TEMP%.\TEMP.DAT > nul
if errorlevel=1 echo %%a>> %TEMP%.\TEMP.DAT
)
copy/y %TEMP%.\TEMP.DAT %1 > nul
if exist %TEMP%.\TEMP.DAT del %TEMP%.\TEMP.DAT
:eof
===== BATCH SCRIPT END =====

Can anyone help? I'm not a good coder and
I've searched high and low for a script already
available but to no avail.

Regards

Alex



Sponsored Link
Ads by Google

Response Number 1
Name: IVO
Date: September 20, 2009 at 13:38:33 Pacific
Reply:

I am honoured to modify secret_doom's script, the ultimate Master of batch scripting, who guided me to that art.

:: NODUP.BAT  Usage: nodup
@echo off
echo.
echo.  Processing .txt files in "%CD%"
echo.  Please wait...
for %%j in (*.txt) do (
  type nul > %TEMP%.\TEMP.DAT
  for /F "delims=" %%a in ('type "%%j"') do (
    find "%%a" < %TEMP%.\TEMP.DAT > nul
    if ErrorLevel 1 echo %%a>> %TEMP%.\TEMP.DAT
  )
  copy %TEMP%.\TEMP.DAT "%%j" > nul
  if exist %TEMP%.\TEMP.DAT del %TEMP%.\TEMP.DAT
)
echo.  DONE!
:: End_Of_Batch


0

Response Number 2
Name: hawkinsa21
Date: September 21, 2009 at 05:44:49 Pacific
Reply:

Wow, thats a great response and the batch as you already
knows works flawlessly. I must learn to batch code myself.
Excellent job IVO I really appreciate it.


0

Sponsored Link
Ads by Google
Reply to Message Icon

Related Posts

See More






Use following form to reply to current message:

Login or Register to Reply
LoginRegister


Sponsored links

Ads by Google


Results for: Batch file for removing dupes

batch file for time delay in xp www.computing.net/answers/windows-xp/batch-file-for-time-delay-in-xp/78216.html

batch file for permissions www.computing.net/answers/windows-xp/batch-file-for-permissions/170216.html

Batch file for launching Mozilla www.computing.net/answers/windows-xp/batch-file-for-launching-mozilla/137694.html