issue with sed.exe on Windows 10

November 17, 2019 at 05:20:06
Specs: Windows 10, 8GB
I'm using a GNU sed.exe (64-bit) on a Windows 10 and I have this issue:

When I run this command

echo testme| sed "s/me$//g

it is answering:

testme

- while it should remove the "me" string. I have a 32-bit version of the same vendor (GNU) on a Windows 64 bit, and on that one it replies with:

test

( and thus, removing the "me" string )

All the other GNU tools I use seem to work correctly (AWK, PRINTF, GREP), but not the above one.

Any idea why this is happening ?


See More: issue with sed.exe on Windows 10

Reply ↓  Report •

#1
November 17, 2019 at 13:01:54
I believe it's your use of $ to designate an end of line.
There is no end of line character in your test string.

This works for me:

echo testme| sed "s/me//g"

Also, in your example you left off the trailing quote.

MIKE

http://www.skeptic.com/

message edited by mmcconaghy


Reply ↓  Report •

#2
November 17, 2019 at 13:13:40
> I believe it's your use of $ to designate an end of line.
> There is no end of line character in your test string.

There is: on Windows, the ECHO command always end with an end of line

If I omit the $ all is OK because the issue is with the $ sign

For example:
This one works on both Windows 7 and 10 (for me):
echo testme| sed "s/me//g"

This one only works correct on Windows 7 (for me):
echo testme| sed "s/me$//g"

For example, on my Windows 7 machine:

C:\> echo testme| sed "s/me$//g"
test

C:\>

message edited by Looge


Reply ↓  Report •

#3
November 17, 2019 at 13:14:41
> Also, in your example you left off the trailing quote.

Yes I did, there should be a double quote on the end, I tested it with that, but in the thread here is was removed somehow


Reply ↓  Report •

Related Solutions

#4
November 17, 2019 at 13:15:22
> This works for me:
>
> echo testme| sed "s/me//g"

Yes, it does for me, the issue is with the command using the $ sign


Reply ↓  Report •

#5
November 17, 2019 at 14:07:00
It would appear that the issue is with MS

This one works on both Windows 7 and 10:
echo testme| sed "s/me//g"

This one only works correct on Windows 7:
echo testme| sed "s/me$//g

So what did MS do to change the function?

Because: echo testme| sed "s/me//g"
works on Windows XP, but echo testme| sed "s/me$//g"
does not.

Perhaps, as we used to say, it's an undocumented feature??

Might also be how they implemented SED between 32 & 64 bit versions.

MIKE

http://www.skeptic.com/

message edited by mmcconaghy


Reply ↓  Report •

#6
November 17, 2019 at 14:35:34
It'll not be an undocumented feature (by GNU) since the $ sign is an important regular expression - But I need to have a check on something you mentioned ..

Reply ↓  Report •

#7
November 23, 2019 at 09:56:25
I checked using the 32 and 64-bit GNU toolset, but they have the same issue, on Windows 10.

If I run an older version of the same GNU set, it works correctly, but each call gets this warning message. It's a quite technical one, and I can't recall it, but Google says this is because there is some issue when Windows works with old versions of GNU.


Reply ↓  Report •

#8
November 23, 2019 at 10:58:18
Windows 10 comes with ability to run Linux.
What happens if you run Linux on Windows and try it?
Not sure which distro they use or how they have implemented it.
Do not think it's a dual boot situation.

I'm on Win 7 so can't give it a try.

MIKE

http://www.skeptic.com/


Reply ↓  Report •

#9
November 29, 2019 at 12:57:19
I can try to run the Linux thing on Windows 10 - I'm sure that will work just fine.


Reply ↓  Report •

#10
November 30, 2019 at 12:50:35
The Bash on Windows asks me to install the Distribution by visting the Microsoft Store

But, my Bash from Cygwin has an interesting behaviour;
- the ^ character in SED works just fine
- the $ works .. if I use it with PRINTF instead of ECHO

So, that explains a bit more where the issue is


C:\cygwin32\bin>printf "testabc"| sed "s/abc$//g"
test

C:\cygwin32\bin>echo testabc| sed "s/abc$//g"
testabc

C:\cygwin32\bin>echo testabc| sed "s/^test//g"
abc

C:\cygwin32\bin>


Reply ↓  Report •

#11
November 30, 2019 at 14:14:26
Got it - it turns out the ECHO command, the one that is the internal command from the CMD interpreter, some way or another does not respect the end-of-line character. I don't understand why, but this is the only failure I found in the system.

The CygWin module also brings its own echo command, but as a file : echo.exe
That one works better, in this context

So I'm guessing that I'll be using this one insetad. But then I needed to find out how to get the system's ECHO command not being used. I found that on StackOverflow - not sure if I can share such an URL here - but it works just great. You know, you think you know all ... and then you find another great thing, which is around from at least Windows 7.

Anyway, here it is:


He's using the internal Windows ECHO command

C:\cygwin64\bin>echo testabc| sed "s/abc$//g"
testabc

C:\cygwin64\bin>

But if you use the echo.exe from inside the same directory,
it works:

C:\cygwin64\bin>dir echo.exe /b
echo.exe

C:\cygwin64\bin>echo.exe testabc| sed "s/abc$//g"
test

C:\cygwin64\bin>

So, I found this interesting command on StackOverflow,
that explains how to override internal commands from the CMD

This is how:

C:\cygwin64\bin>@doskey echo=c:\cygwin64\bin\echo.exe $*

and if you then run the ECHO command:

C:\cygwin64\bin>echo testabc| sed "s/abc$//g"
test

C:\cygwin64\bin>


Reply ↓  Report •

#12
December 2, 2019 at 08:51:56
Although the above works, it's just fixing the exact usage as shown there.

The real issue is that the $ no longer accepts DOS style EOL's (end of line, line feed, carriage return, or however it is named).

Apparently, Cygwin has changed the behaviour a couple of years ago, so that now you have what is considered the correct behaviour. Note that that behaviour is in the context of files : text files with DOS type EOLs versus Linux/Unix style EOLs.

I'm just using the EOL in context of a one line command, where the EOL is just the end of the line, or the end of the command (which is the same in this kind of usage).

And since ECHO, the command on Windows that is, uses a Windows EOL ... the subsequent SED command doesn't understand it, and doesn't work on it. The same issue can be see with GREP, or whatever other command allows usage of $ signs.

It also is NOT a Windows 10 issue, the reason is just that since Cygwin changed the behaviour, Windows 7 is already around, and Cygwin is already installed on most Windows 7 machines, unlike new installs on Windows 10.

When I run the old Cygwin on Windows 10, I get errors left and right, it just doesn't work. 64 or 32 bit has nothing to do with it

message edited by Looge


Reply ↓  Report •

Ask Question