remove duplicate lines in txt file

July 20, 2009 at 03:58:26
Specs: Windows XP / Vista
I have a txt file called elements.txt that contains content similar to this:
---------- C:\TEMP\1000604\SAMPLEALBUM\1000604\LAST\LAST.PCK
PIC_FRAME_NAME = fr_3d - beveled 01 - large landscape

---------- C:\TEMP\1000604\SAMPLEALBUM\1000604\PAGE_1\PAGE_1.PCK
PIC_FRAME_NAME = fr_3d - beveled 01 - small portrait
PIC_FRAME_NAME = fr_3d - beveled 01 - small portrait

I need to delete all lines starting with "-----" and all duplicate lines... so that I end up with a file looking something like this:
PIC_FRAME_NAME = fr_3d - beveled 01 - large landscape
PIC_FRAME_NAME = fr_3d - beveled 01 - small portrait

If I could do it in the batch file that runs all my other stuff it would be helpful. & I want to output the result to a different txt file (outfile.txt)


See More: remove duplicate lines in txt file

Report •


#1
July 20, 2009 at 04:40:15
vbscript
Set objFS = CreateObject("Scripting.FileSystemObject")
strFile = "c:\test\file.txt"
Set objFile = objFS.OpenTextFile(strFile)
Set d = CreateObject("Scripting.Dictionary")
Do Until objFile.AtEndOfStream
	strLine = objFile.ReadLine
	If Not InStr(strLine,"--------") >0 Then
		If Not d.Exists(strLine) Then 
			d.Add strLine , 0
		End If 
	End If 
Loop
x=d.Items
For Each strKey In d.keys
	WScript.Echo strKey
Next

GNU win32 packages | Gawk


Report •

#2
July 20, 2009 at 04:51:24
Thanx for the quick response. It only echos the two lines in a pop-up though... How do I get it into a txt file? I'm not very clued up with vbs.

Report •

#3
July 20, 2009 at 05:22:49
save the script as myscript.vbs and on command line
c:\test> cscript /nologo myscript.vbs > newfile.txt

simple as that.

GNU win32 packages | Gawk


Report •

Related Solutions

#4
July 20, 2009 at 05:44:43
It's simple when you know what you're doing :) lol
Thanx again!!

Report •


Ask Question