Computing.Net > Forums > Programming > remove duplicate lines in txt file

Computer Problems? Computing.Net has over 1,000,000 posts about all things technology related! Click here to start participating now! Also, check out the New User Guide.

remove duplicate lines in txt file

Reply to Message Icon

Name: fcarstens
Date: July 20, 2009 at 03:58:26 Pacific
OS: Windows XP / Vista
Subcategory: Batch
Comment:

I have a txt file called elements.txt that contains content similar to this:
---------- C:\TEMP\1000604\SAMPLEALBUM\1000604\LAST\LAST.PCK
PIC_FRAME_NAME = fr_3d - beveled 01 - large landscape

---------- C:\TEMP\1000604\SAMPLEALBUM\1000604\PAGE_1\PAGE_1.PCK
PIC_FRAME_NAME = fr_3d - beveled 01 - small portrait
PIC_FRAME_NAME = fr_3d - beveled 01 - small portrait

I need to delete all lines starting with "-----" and all duplicate lines... so that I end up with a file looking something like this:
PIC_FRAME_NAME = fr_3d - beveled 01 - large landscape
PIC_FRAME_NAME = fr_3d - beveled 01 - small portrait

If I could do it in the batch file that runs all my other stuff it would be helpful. & I want to output the result to a different txt file (outfile.txt)



Sponsored Link
Ads by Google

Response Number 1
Name: ghostdog
Date: July 20, 2009 at 04:40:15 Pacific
Reply:

vbscript

Set objFS = CreateObject("Scripting.FileSystemObject")
strFile = "c:\test\file.txt"
Set objFile = objFS.OpenTextFile(strFile)
Set d = CreateObject("Scripting.Dictionary")
Do Until objFile.AtEndOfStream
	strLine = objFile.ReadLine
	If Not InStr(strLine,"--------") >0 Then
		If Not d.Exists(strLine) Then 
			d.Add strLine , 0
		End If 
	End If 
Loop
x=d.Items
For Each strKey In d.keys
	WScript.Echo strKey
Next

GNU win32 packages | Gawk


1

Response Number 2
Name: fcarstens
Date: July 20, 2009 at 04:51:24 Pacific
Reply:

Thanx for the quick response. It only echos the two lines in a pop-up though... How do I get it into a txt file? I'm not very clued up with vbs.


0

Response Number 3
Name: ghostdog
Date: July 20, 2009 at 05:22:49 Pacific
Reply:

save the script as myscript.vbs and on command line

c:\test> cscript /nologo myscript.vbs > newfile.txt

simple as that.

GNU win32 packages | Gawk


1

Response Number 4
Name: fcarstens
Date: July 20, 2009 at 05:44:43 Pacific
Reply:

It's simple when you know what you're doing :) lol
Thanx again!!


0

Sponsored Link
Ads by Google
Reply to Message Icon

Related Posts

See More


batch to rename txt file ... Generate a report



Post Locked

This post is quite old and has been locked from receiving new replies. Please create a new posting instead.


Go to Programming Forum Home


Sponsored links

Ads by Google


Results for: remove duplicate lines in txt file

Remove duplicate lines in pure .bat www.computing.net/answers/programming/remove-duplicate-lines-in-pure-bat/15777.html

Delete 1st 4 lines & last 8 Lines in TXT File www.computing.net/answers/programming/delete-1st-4-lines-last-8-lines-in-txt-file/19676.html

Select First Line in text.txt www.computing.net/answers/programming/select-first-line-in-texttxt/10514.html