certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Microsoft Windows
Post Reply
User avatar
MigrationUser
Posts: 336
Joined: 2021-Jul-12, 1:37 pm
Contact:

certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Post by MigrationUser »

06 Nov 2012 15:47
npocmaka


Certutil is available on my WIN 7 and Vista machines by default.I think it should be also available for XP but I'm not 100% sure.

1. The thing I used this for was to decode and encode BASE64 strings. (-decode and -encode command switches) .It has two annoying features here - for decode and encode it needs
-----END CERTIFICATE----- and -----BEGIN CERTIFICATE----- at begining and at the of base64 file.And it prints decoded file in lines with max length of 64 symbols.
So here are two very very very simple scripts that use certutil to decode and encode base64 string (and dealing with begin and end tags) (there are no checks for file existence and if the parameters are correct - I rely on certutil error messages):

decode:

Code: Select all

@echo off
setlocal
for /f "delims== tokens=1" %%A  in ('type %1') do (
	set encoded=%%A
)
rem two additional lines are needed by certutil
echo -----BEGIN CERTIFICATE-----> %2.tmp
echo %encoded%>>%2.tmp
echo -----END CERTIFICATE----->> %2.tmp


certutil /decode %2.tmp %2
del /S /Q %2.tmp
endlocal
encode:

Code: Select all

@echo off
setlocal
certutil /encode %1 %1.encoded
rem create an empty file
break > %2

setlocal  ENABLEDELAYEDEXPANSION
(
	for  /f  "eol=-" %%A in (' type %1.encoded ') do (
		rem this not works - left an empty spaxes after each line from typed file
		rem null< (set /p #=%%A) >>%1.encoded
		
		set result=!result!%%A
	)
)
endlocal & set result=%result%
echo %result%>> %2%
del /S /Q
endlocal


The bad thing is that the base64 strings are stored in a variable and there's a limitations for it's size.But for a small strings it works.





2. Much more interesting.In the help it shows that there's an -decodehex switch.And I was surprised to find that there's also an undocumented switch -encodehex (strange - decodehex looks more dangerous , because it can be used to produce binaries). Here's an example structure of encoded file:

Code: Select all

0000	73 65 74 20 78 3d 15 0d  0a 73 65 74 20 79 3d 12   set x=...set y=.
0010	0d 0a 65 63 68 6f 20 2a  2a 25 78 25 25 79 25 2a   ..echo **%x%%y%*
0020	2a                                                 *


(just as every hex editor)

This was my attempt to create file that set LF and CR to variables :) - and I still don't know why it was unsuccessful it prints "§↕" (any help here will be welcomed - I don't know what goes wrong).
EDIT: I've set DEC codes instead HEX .Anyway still not works.
In fact the data that is behind the hexes is not necessary - it's only for visualisation .
To decode the sample just use this (if this above is saved in sample.hex) :

certutil -decodehex sample.hex not.working.bat


Any way it works for creation of the famous beep.bat :

Code: Select all

0000	65 63 68 6f 20 07
this pattern can be used for creation of bat that echoes a random symbols by hex.Just edit the last character.

and this is a pattern for setting a symbol by hex to %#% variable (just edit the last character):

Code: Select all

0000	73 65 74 20 78 3d 15 0d  
Anyway there are few more things that I want to try with this :)

here's a how forfiles can be used for hex symbols

Last edited by npocmaka (12 Dec 2012 00:05)

----------------------------

#2 07 Nov 2012 06:57
dbenham


Very very cool - I've never seen this command before. I love the encocdehex / decodehex options.

Unfortunately, CERTUTIL is not standard on XP, though I believe there is a download that can be installed.

Your attempt to create a variable with LF and CR is doing exactly what I would expect it to do.

Here is a working HEX2STR.BAT script that can convert any string encoded as hex into a valid text string with 2 exceptions:

1) if the result is returned in a variable then the hex string cannot contain 00 (the nul character)

2) if the result is printed to the screen and not stored in a variable then the result will be truncated at the first occurence of 1A. Also, any 00 will show up on the screen as a space.

The script will work properly regardless whether delayed expansion is enabled or not.

Code: Select all

@echo off
:hex2str  HexStr  [RtnVar]
::
:: Convert a hexadecimal encoded string HexStr into a text string.
::
:: Return the result in RtnVar. RtnVar will be undefined if HexStr
:: contains 00 (the encoding for the nul byte).
::
:: Print the result to to screen (stdout) if RtnVar is not specified.
:: The screen output will be truncated at the first occurance of 0x1A.
::
:: HexStr - The hex encoded string to be converted. Each character must be
::          encoded as a pair of hexadecimal digits. Hex pairs may be
::          delimited by one or more space, tab, or new line characters.
::
:: RtnVar - The optional name of a variable used to store the result.
::
setlocal
set "NotDelayed=!"
setlocal enableDelayedExpansion
set "tempFile=!temp!\hex2str%random%"
if "%~2" equ "" (
  >"!tempFile!.hex" echo %~1
  >nul 2>&1 certutil -f -decodehex "!tempFile!.hex" "!tempFile!.txt"
  type "!tempFile!.txt"
  2>nul del "!tempFile!.hex" "!tempFile!.txt"
  exit /b
)
set "hex=%~1"
set "hex=!hex:25=25 35!"
set "hex=!hex:22=25 7e 36!"
set "hex=!hex:0A=25 7e 33!"
set "hex=!hex:0D=25 34!"
if not defined NotDelayed (
  set "hex=!hex:5e=5e 5e!"
  set "hex=!hex:21=5e 21!"
)
>"!tempFile!.hex" echo !hex!
>nul 2>&1 certutil -f -decodehex "!tempFile!.hex" "!tempFile!.txt"
setlocal disableDelayedExpansion
for /f usebackq^ delims^=^ eol^= %%a in ("%tempFile%.txt") do set "rtn=%%a"
setlocal enableDelayedExpansion
set LF=^


set "replace=%% """"
for %%3 in ("!LF!") do for /f %%4 in ('copy /Z "%~dpf0" nul') do (
  for /f "tokens=1,2" %%5 in ("!replace!") do (
    endlocal
    endlocal
    endlocal
    endlocal
    set "%~2=%rtn%" !
    2>nul del "%tempFile%.hex" "%tempFile%.txt"
  )
)
exit /b
To get a LF variable, you can use CALL HEX2STR 0A LF

I have also written a script to go in reverse - STR2HEX.BAT.

The output contains just the encoded hex data, without the leading address info and without the trailing ASCII representation. update - added a /C (compress) option to strip spaces from the output

Code: Select all

@echo off
:str2hex  StrVar  [RtnVar]  [/C]
::
:: Encode the string within variable StrVar as hexadecimal.
::
:: Return the result in RtnVar
:: or
:: Print the results to the screen (stdout) if RtnVar is not specified.
::
:: StrVar - The name of the variable containing the string to be encoded.
::
:: RtnVar - The optional name of a variable used to store the result.
::
::     /C - Compress option: All spaces are stripped from the output.
::          The RtnVar must be specified to use this option. Compressed
::          output may be sent to the screen by specifying an empty quoted
::          string for the RtnVar.
::
setlocal enableDelayedExpansion
set "tempFile=%temp%\hex2str%random%"
>"!tempFile!.txt" echo(!%~1!
>nul 2>&1 certutil -f -encodehex "!tempFile!.txt" "!tempFile!.hex"
set "hex="
set "delim= "
if /i "%~3" equ "/C" set "delim="
for /f "usebackq tokens=1*" %%A in ("!tempFile!.hex") do (
  set "ln=%%B"
  set "ln=!ln:~0,48!"
  if /i "%~3" equ "/C" (set "ln=!ln: =!") else set "ln=!ln:  = !"
if "%~2" neq "" (set "hex=!hex!!ln!%delim%") else echo !ln!
)
2>nul del "!tempFile!.hex" "!tempFile!.txt"
if "%~2" neq "" (
  endlocal
  set "%~2=%hex%"
)
exit /b
Dave Benham

Last edited by dbenham (08 Nov 2012 02:13)

----------------------------

#3 07 Nov 2012 16:38
npocmaka


Hello Dave,

Thanks for the help.

I thought that the line identificators and spaces between HEXes in encoded file are mandatory.
But it can process the file even without them ! Never came to my mind to try that - this makes its use far more easier.
/*
off-topic
I don't completely understand everything in your code ,
but for a first time I pay enough attention to
overlapped setlocal/endlocal and parentheses - it's realy
amazing . You still have access to FOR tokens ,and you can use
enableddelayedexpansion features?!?!!

and what is this lonely ! here(hex2str):

Code: Select all

 set "%~2=%rtn%" !
?

this looks very twisted :)

*/

You're right about XP - it's not native , but can be installed - the first hit from google => http://www.microsoft.com/en-us/download ... x?id=16770 (Administration Tools Pack).

One of the next things on my list that I want to try (in fact it's almost done ) is to convert to string to hex without temp file -> with certutil -dump file you can receive hex codes right in the console.

One more off-hex topic:

And one more feature of certutil that I want to emphasize on. -URLCache -v - this displays the whole internet history and locations of cache files.The most valuable of this is that it searches
trough ..\AppData\Local\Microsoft\Windows\Temporary Internet Files\Content.IE5\XXXXX\ files. As I often use internet explorer as a command line http client via its com-objects I really want to have access to cached files.But the folders under Content.IE5 are über-hidden and über-protected (if don't believe me try to access them with windows explorer or search there with DIR).Although the files can be copied if you know their exact location.The only one other tool that I know that can do this is IECacheView by NirSoft .

Looks like the most valuable features of certutil for me have nothing to do with certificates :)

Last edited by npocmaka (08 Nov 2012 08:50)

----------------------------

#4 08 Nov 2012 04:16
dbenham


I edited my prior post: I fixed the docs a bit for hex2str, and added a /C (compress) option to the str2hex.

The hex2str uses a safe return technique that allows the return of any string value across the ENDLOCAL barrier. The rogue ! is a critical piece of code that helps ensure the safe return technique works regardless whether or not delayed expansion is enabled within the calling routine. jeb developed the technique in support of routines I was writing to inter-convert between ASCII codes and string values: Re: new functions: :chr, :asc, :asciiMap. aGerman asks about the ! in the next post in the thread, and jeb replies with an explanation.

I see how -dump can eliminate the need to write the result of str2hex to a temporary file. But I don't see how you can avoid writing the source string to a temporary file.

The -encode and -decode functions are generally reciprocal in nature, but -decode is a bit too clever for its own good. It allows a high degree of flexibility as to the format of the hex source. But it can get confused as to what is a hex pair and what is ASCII text. If you encode a text file containing nothing but hex digits and spaces, and then decode the result, you do not get the original text file. My str2hex script preserves only the hex pairs in the output, so it is a true reciprocal of hex2str.

Dave Benham

Last edited by dbenham (08 Nov 2012 04:18)

----------------------------

#5 09 Jan 2013 21:30
Simon Sheppard
npocmaka wrote:

2. Much more interesting.In the help it shows that there's an -decodehex switch.And I was surprised to find that there's also an undocumented switch -encodehex (strange - decodehex looks more dangerous , because it can be used to produce binaries).
Great find npocmaka,
I have now added -encodehex to the new CERTUTIL page along with a link back to this thread.

I wouldn't be surprised if they left it out of the documentation by accident.

----------------------------

#6 09 Jan 2013 22:51
npocmaka


Yey.I have a credit :-)

----------------------------

#7 10 Jan 2013 03:08
carlos


Is not to be disagreeable, but it is documented here:

https://docs.microsoft.com/en-us/previo ... 6(v=ws.10)

but the valids formats of the hex file for -decodehex is undocumented.
Anyways, the -encodehex not appear in the help of certutil of windows 2000 server, windows 2003 server, windows 7).

Last edited by carlos (10 Jan 2013 03:21)

----------------------------

#8 12 Jan 2013 13:42
Simon Sheppard
carlos wrote:

Is not to be disagreeable, but it is documented here:
Yes good catch, I've updated the SS64 page to note that there are a few small differences between the command line help (CERTUTIL -?) and the various MSDN online pages.
In many cases the -f force and -v verbose options make no difference, but I'm not sure if that means they are superfluous options or they just had nothing to do in the examples I tried.

It is a beast of a command and I'm tempted to split it across several pages, but I think on balance it's better being able to find things on one page with CTRL-F

----------------------------

#9 26 Apr 2013 15:47
npocmaka


Or -dump option is buggy or does not dump file in hex and is intended only for certification files (but with bigger files it usually works as hex dumper)?

Code: Select all

C:\Users\>echo aa > aa.txt

C:\Users\>type aa.txt
aa

C:\Users\>certutil -dump aa.txt
    69                                                 i
CertUtil: -dump command completed successfully.

C:\Users\>certutil -encodehex aa.txt aa.hex.txt
Input Length = 5
Output Length = 63
CertUtil: -encodehex command completed successfully.

C:\Users\>type aa.hex.txt
0000    61 61 20 0d 0a                                     aa ..

C:\Users\>
dump never works with utf-16 files (i don't know for other patterns...)
-encodehex on the other hand has no problems.
The help only says

Code: Select all

  -dump             -- Dump configuration information or files
Last edited by npocmaka (26 Apr 2013 16:20)

----------------------------

#10 27 Apr 2013 14:47
Simon Sheppard


Good point, I've changed the SS64 description.

It does look like CertUtil is very much built for handling certificates, it's probably never been tested as a general purpose utility - such is the Microsoft way!

----------------------------

#11 20 May 2013 09:31
npocmaka


eventually it will be fixed in win server 2012 :

https://connect.microsoft.com/WindowsSe ... n-is-buggy

:)
Post Reply