Read a file, output each line to a new file, preserve all characters?

[Rekrul]

I'm trying to write a script that will read a given file line by line and output each one to a new file. I want to do this, so that the script can make some conditional changes to some of the lines. It should be a simple process, but of course it's not because Batch is the most ass-backward, convoluted mess ever invented.

The main problem is all the symbols that you absolutely, positively cannot under any circumstances, have in a string, unless the whole thing is in quotes. Stuff like the ampersand, percent sign, greater than, less than, etc. Trying to echo such a line will cause all sorts of problems. OK, most of those can be escaped, but you can't do that when reading in lines of a file. And the ampersand can't be escaped at all. If there's a way to do it, I can't figure it out.

So you enclose it in quotes, problem solved, right? Not even close! You may have gotten those symbols into a string, but now every line is going to have quotes on it when you output it to the new file. So just echo the string without the first and last characters, right? Nope! Then you're back to having a problem with the above characters.

I know! use substring replace operations to escape those characters! Nope, that doesn't work. Well, it does if you use DelayedExpansion, and then use two variables as placeholders for the characters, since you STILL can't include them directly in the replace operation, even by escaping them.

So problem solved? Yeah, except for one tiny, insignificant little problem: DelayedExpansion makes it impossible to use exclamation points in strings. .They can contain them if it's defined BEFORE Delayed Expansion is enabled, but the second you do ANY string operations with it enabled, it immediately strips every exclamation point.

ARGH!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Is there any way that I can read each line of a file, which may contain ! % & < >, put it into a variable/string, then output it to a new file, WITHOUT having the line enclosed in quotes in the new file?

I've tried Googling this problem and come up empty. All the solutions assume you just want to remove the quotes, but don't include the problem of how to handle all the problem characters without the quotes.

[Simon Sheppard]

An approach I have used is to replace all the problem characters with something else, either a character like ~ or ` use a unique phrase "Gr33nTreez"

Then do all your processing and then rename the characters back to what they started as.

https://ss64.com/vb/syntax-replace.html

[Rekrul]

Hmm, I thought I saved my reply...

Anyway, thanks for replying.

An approach I have used is to replace all the problem characters with something else, either a character like ~ or ` use a unique phrase "Gr33nTreez"

Then do all your processing and then rename the characters back to what they started as.

I guess that's what I'll have to do. I was hoping to do it all in Batch to make it as portable as possible, but I guess there's no avoiding the use of additional programs to get the job done.

[bluesxman]

Have you tried something like this approach?

Code: Select all

setlocal disabledelayedexpansion
set /a count=0
for /f "useback tokens=* delims=" %%L in ("your_file.txt") do (
  set "line=%%L"
  call :write_file
)

goto :EOF

:write_file
set /a "count+=1"
REM your conditional changes would go here
REM example
set "line=%line:foo=bar%" 
(
  <nul set /p "out=%line%"
  echo:
) > your_file_line_%count%.txt
goto :EOF

But I'll caveat this by saying that CMD is really not well suited to this sort of thing, there are various pitfalls. The majority can probably be handled, but you'd have to take them on a case by case basis.

[GOD]

Have you tried something like this approach?

Code: Select all

setlocal disabledelayeexpansion
set /a count=0
for /f "useback tokens=* delims=" %%L in ("your_file.txt") do (
  set "line=%%L"
  call :write_file
)

goto :EOF

:write_file
set /a "count+=1"
REM your conditional changes would go here
REM example
set "line=%line:foo=bar%" 
(
  <nul set /p "out=%line%"
  echo:
) > your_file_line_%count%.txt
goto :EOF

But I'll caveat this by saying that CMD is really not well suited to this sort of thing, there are various pitfalls. The majority can probably be handled, but you'd have to take them on a case by case basis.

You really do write some adorable cute code.

[Shadow Thief]

Have you tried something like this approach?

Code: Select all

setlocal disabledelayeexpansion
set /a count=0
for /f "useback tokens=* delims=" %%L in ("your_file.txt") do (
  set "line=%%L"
  call :write_file
)

goto :EOF

:write_file
set /a "count+=1"
REM your conditional changes would go here
REM example
set "line=%line:foo=bar%" 
(
  <nul set /p "out=%line%"
  echo:
) > your_file_line_%count%.txt
goto :EOF

But I'll caveat this by saying that CMD is really not well suited to this sort of thing, there are various pitfalls. The majority can probably be handled, but you'd have to take them on a case by case basis.

You really do write some adorable cute code.

What a weird thing to say to someone

[Simon_Weel]

As you discovered, the Windows SMD Shell isn't exactly suited for this kind of jobs. You may want to look into something like Autohotkey https://www.autohotkey.com/?

[bluesxman]

You really do write some adorable cute code.

Thanks, I was going for "functional and concise, with a hint of 'inoffensive'" but these days I'll take whatever compliments I can get.

[jeb]

To be bullet proof, you need the delayed toggling technique.

Code: Select all

@echo off
set "filename=%~1"

SETLOCAL DisableDelayedExpansion
FOR /F "delims=" %%a in ('findstr /n "^" "%filename%"') do (
    set "var=%%a"
    SETLOCAL EnableDelayedExpansion
    set "var=!var:*:=!"
    echo(!var!
    ENDLOCAL
)

The findstr is used to prefix each line with a line number, this is necessary to avoid the removal of empty lines or lines beginning with the EOL character (default is semi colon).

The delayedexpansion has to be disabled in:

Code: Select all

set "var=%%a"

Because with enabled mode, exclamation marks and carets would be removed.

But to remove the line number in front, the delayedexpansion mode has to be enabled
The line number is removed by the

Code: Select all

set "var=!var:*:=!"

You can't use the delims=: here, because this would also remove all colons at the beginning of lines.

Code: Select all

echo(!var! 

Is essential, to avoid problems with lines like "OFF", "/?" or "..\..\windows\system32\calc.exe".
echo(!var! is the only echo that can output any content even empty content.