I have a text file that has every field start with =

February 6, 2019 at 07:59:23
Specs: Windows 10
I have a text file that has every field start with "="
How do I remove it? The file is too large for notepad to open and for wordpad to replace all the "=" with nothing.
Can i accomplish it using DOS commands, if yes, can u provide the dos command?

See More: I have a text file that has every field start with =

Reply ↓  Report •

#1
February 6, 2019 at 08:34:47
Which DOS are we talking about here? MS-DOS, which was replaced by Windows 95 over 20 years ago; DOSBox, which is a 1990's era DOS emulator; FreeDOS, which is around; or do you mean a console window in Windows 10, which has nothing to do with DOS?

How To Ask Questions The Smart Way


Reply ↓  Report •

#2
February 6, 2019 at 08:37:34
Razor2.3 we need you for a powershell script, get in here!
if he doesnt answer in the next 18 hours or so, ill try with Batch, it just takes a bit more effort, especially if there are empty lines or special characters in the file like: !@#$%^&*()"[]~'

Soo eeh maby like next monday :D

i5-6600K[delid]@4.7GHz/4.4GHz cache/@1.395v LLC=6 | 2x4GB Crucial-DDR4-2400CL17@14-15-15-28 1T 3000MHz@1.4v | MSI Armor RX 570 4GB@1410Mhz core@1.2v/1920MHz


Reply ↓  Report •

#3
February 6, 2019 at 08:44:18
But PowerShell's not available to any version of DOS. :(

Unless he means Windows. In which case this file has to be at least a few hundred MB.

How To Ask Questions The Smart Way


Reply ↓  Report •

Related Solutions

#4
February 6, 2019 at 09:00:34
I am on windows 10, and I meant cmd. sorry abt it.

Reply ↓  Report •

#5
February 6, 2019 at 09:01:37
The file size is 1.6 GB . its a simple text file.

Reply ↓  Report •

#6
February 6, 2019 at 09:03:23
The text qualifiers in the file are present as " ". I dont want to remove them. I just want the ' = ' (equal sign) removed from the file.

Reply ↓  Report •

#7
February 6, 2019 at 09:41:06
Is this a CSV, or something? What separates the fields?

How To Ask Questions The Smart Way


Reply ↓  Report •

#8
February 6, 2019 at 18:35:42
Yes, "a sample is worth a thousand words" - how about a few sample "lines" (assuming the text-file is in lines, ie crlf-breaks, and not just a big gob.) Then we know if every single equals needs replaced with null, or just ones qualified by spaces or etc. I know that powershell can do this, so Razor will have the solution. Vbscript can also handle this with minimal coding:
set fso=createobject("scripting.filesystemobject")
f=wscript.arguments(0)
target=wscript.arguments(1)
repl=wscript.arguments(2)
a=fso.opentextfile(f,1).readall
a=replace(a,target,repl,1,-1,text)
fso.opentextfile(f,2,true).write a

Reply ↓  Report •

#9
February 7, 2019 at 01:35:02
1.6GB? how about HxD hex editor, and look for:
CR LF =
replace with nothing

you might have to add one LF for the firts line after the replacement.

i5-6600K[delid]@4.7GHz/4.4GHz cache/@1.395v LLC=6 | 2x4GB Crucial-DDR4-2400CL17@14-15-15-28 1T 3000MHz@1.4v | MSI Armor RX 570 4GB@1410Mhz core@1.2v/1920MHz


Reply ↓  Report •

#10
February 8, 2019 at 09:57:52
No response from the OP. I'm just going to assume he found his answer elsewhere. We can't assume anything else about the file, including encoding, line endings, or if the targeted equals sign starts a new line. We still will, obviously, such as how I'm assuming there might be equals signs in double quotes that shouldn't be deleted.

I wanted to do something different, so I went with a single, needlessly very large buffer. My test data was an almost 400MB log file with some equals signs in it, which is why processing is split across two for loops.

$sr = New-Object IO.StreamReader "some.log"
$sr.Peek() | Out-Null
$sw = New-Object IO.StreamWriter "out.txt", $false, $sr.CurrentEncoding 
[char[]]$workArea = , 0 * 8mb
[bool]$inQuotes = $false

while (!$sr.EndOfStream) {
  ($sr.BaseStream.Position / $sr.BaseStream.Length).ToString('P2')
  $end = $sr.Read($workArea, 0, $workArea.Length)
  :findStart for ($i = 0; $i -lt $end; ++$i) {
    switch ($workArea[$i]) {
      '"' { $inQuotes = !$inQuotes }
      '=' { if (!$inQuotes) { break findStart }}
    }
  }
  $writeTo = $i++
  :changeWorkArea for (; $i -lt $end; ++$i) {
    switch ($workArea[$i]) {
      '"' { $inQuotes = !$inQuotes }
      '=' { if (!$inQuotes) { continue changeWorkArea }}
    }
    $workArea[$writeTo++] = $workArea[$i]
  }
  $sw.Write($workArea, 0, $writeTo)
}
$sr.Close(); $sw.Close()
This ended up being ridiculously slow in PowerShell. My estimate for processing a similar log file 1.6GB long would take over 5 hours. The split for loops actually shaved off roughly an additional 90 minutes from that estimate. I'm not sure how long it'd take as a pure PowerShell pipeline script, but it'd probably be slower. Could I do better? Probably, but it was getting late.

So I did a straight port over to C#, and asked PowerShell to run that. Estimated completion? 30 seconds.

(Add-Type @'
using System;
using System.Text;
using System.IO;

public class ProcFile {
  public static void ProccessFile(string inPath, string outPath) {
    var sr = new StreamReader(inPath);
    sr.Peek();
    var sw = new StreamWriter(outPath, false, sr.CurrentEncoding);
    char[] workArea = new char[8 * 1024 * 1024];
    var inQuotes = false;

    while (!sr.EndOfStream) {
      Console.WriteLine("{0:P2}", (double)sr.BaseStream.Position / sr.BaseStream.Length);
      var end = sr.Read(workArea, 0, workArea.Length);
      int i;
      for (i = 0; i < end; ++i) {
        switch (workArea[i]) {
          case '"':
            inQuotes = !inQuotes;
            break;
          case '=':
            if (!inQuotes)
              goto modWorkArea;
            break;
        }
      }
modWorkArea:
      var writeTo = i++;
      for (; i < end; ++i) {
        switch (workArea[i]) {
          case '"':
            inQuotes = !inQuotes;
            break;
          case '=':
            if (!inQuotes)
              continue;
            break;
        }
        workArea[writeTo++] = workArea[i];
      }
      sw.Write(workArea, 0, writeTo);
    }
    sr.Close(); sw.Close();
  }
}
'@ -PassThru)::ProccessFile("some.log", "out.txt")

How To Ask Questions The Smart Way

message edited by Razor2.3


Reply ↓  Report •

Ask Question