Solved Vbscript to delete dulicate data

July 11, 2012 at 18:49:35
Specs: Windows 7
Hi Community

I am new to vbscripting and was having problems with a piece of code. When i run this script on the command prompt, it sometimes gives me dulpicate data. I have added the code which deletes the duplicate data but somehow its nor working. I need ur help ASAP. TIA

' Constants used by OpenTextFile()
Const ForReading = 1
Const ForWriting = 2
Const ForAppending = 8


Sub Main

Dim out_fso, out_file
Dim in_fso, in_file
Dim out_file_open

'String for matching

OpenTag = vbTab + "<cm:CustomComicsMaster>"
CloseTag = vbTab + "</cm:CustomComicsMaster>"
UuidTag = vbTab + vbTab + "<cm:uuid>"
EndTag = "</feed>"

SeriesOpenTag = vbTab + vbTab + "<cm:CustomComicsSeries>"
SeriesCloseTag = vbTab + vbTab + "</cm:CustomComicsSeries>"
TargetExtIdTag = vbTab + vbTab + vbTab + "<cm:externalID>"

filecount = 0
linecount = 0
i = 1

'Open Input file for Read

in_filename = "ee771858-5ce9-4821-bb23-69f100b94d59.xml"

in_filepath = "C:\Projects\Korea Telco\feedxml\comics\raw\" & in_filename
Set in_fso = CreateObject("Scripting.FileSystemObject")
Set in_file = in_fso.OpenTextFile(in_filepath, ForReading)

out_filepath = "C:\Projects\Korea Telco\feedxml\comics\raw\Series-extIDlist.csv"
Set out_fso = CreateObject("Scripting.FileSystemObject")
Set out_file = out_fso.OpenTextFile(out_filepath, ForAppending)
wscript.echo "Processing... "

'Loop input xml
Do Until in_file.AtEndofStream
linecount = linecount + 1
in_xml = in_file.ReadLine

if in_xml = OpenTag Then
'New Master
InMaster_Flag = true
InSeries = false
uuid = ""
End if

'Store Uuid and use it as output filename
if mid(in_xml,1,11) = UuidTag Then
uuid = mid(in_xml,12,36)
wscript.echo uuid
End if

if mid(in_xml,1,25) = SeriesOpenTag Then
InSeries = True
End if

If InSeries and mid (in_xml,1,18) = TargetExtIdTag Then
ExtId = mid(in_xml, 19, len(in_xml) - 34)
wscript.echo ExtId
End if

'Detect CloseTag and close output file
if mid(in_xml, 1, 25) = CloseTag Then
wscript.echo "uuid: " + uuid
wscript.echo "ExtId: " + ExtId
out_rec = uuid + "," + ExtId
wscript.echo out_rec
end if

Dim fso, tsIn, tsOut, dic, TheLine, Repeat

Set fso = CreateObject("Scripting.FileSystemObject")
Set dic = CreateObject("Scripting.Dictionary")
dic.CompareMode = vbTextCompare 'NOT case sensitive. omit for case sensitive
Set tsIn = fso.OpenTextFile("c:\Input.csv")
Set tsOut = fso.OpenTextFile("c:\Output.csv", 2, True)

Do Until tsIn.AtEndOfStream
TheLine = tsIn.ReadLine
If TheLine <> "" Then
If dic.Exists(TheLine) Then
Repeat = True
Repeat = False
dic.Add TheLine, TheLine
End If
Repeat = False
End If
If Not Repeat Then tsOut.WriteLine TheLine

Set tsIn = Nothing
Set tsOut = Nothing
Set fso = Nothing
Set dic = Nothing

' wscript.echo "Lines Processed: " & linecount & " lines"
' wscript.echo "Files Splitted : " & filecount & " xml files"
SET out_file = NOTHING
SET out_fso = NOTHING

End Sub

Thanks once again

See More: Vbscript to delete dulicate data

Report •

July 13, 2012 at 21:19:14
✔ Best Answer
Since no-one has replied yet, I'll try to help... BUT FIRST! It would be really good if you could supply a sample data-file (using substituted values to preserve confidentiality), AND a better description of how you want to manipulate the data. I gathered a foggy idea from your code, but I'm sure I'm wrong. Are the series "nested" inside of masters, or independant, that sort of thing.
ps: create-filesystemobject only needs to be done ONCE in the script:
set fso=createobject(scripting.filesystemobject)
you don't need separate instances for "read"/"write" or for multiple files, those are handled by the opentextfile methods.

Report •
Related Solutions

Ask Question