#1 06 Nov 2012 15:47

npocmaka
Member
From: Bulgaria
Registered: 03 Dec 2009
Posts: 328

certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Certutil is available on my WIN 7 and Vista machines by default.I think it should be also available for XP but I'm not 100% sure.


1.  The thing I used this for wad to decode and encode BASE64 strings. (-decode and -encode command switches) .It has two annoying  features here - for decode and encode it needs
-----END CERTIFICATE----- and -----BEGIN CERTIFICATE----- at begining and at the of base64 file.And it prints decoded file in lines with max length of 64 symbols.
So here are two very very very simple scripts that use certutil to decode and encode base64 string (and dealing with begin and end tags) (there are no checks for file existence and if the parameters are correct - I rely on certutil error messages):

decode:

@echo off
setlocal
for /f "delims== tokens=1" %%A  in ('type %1') do (
	set encoded=%%A
)
rem two additional lines are needed by certutil
echo -----BEGIN CERTIFICATE-----> %2.tmp
echo %encoded%>>%2.tmp
echo -----END CERTIFICATE----->> %2.tmp


certutil /decode %2.tmp %2
del /S /Q %2.tmp
endlocal

encode:

@echo off
setlocal
certutil /encode %1 %1.encoded
rem create an empty file
break > %2

setlocal  ENABLEDELAYEDEXPANSION
(
	for  /f  "eol=-" %%A in (' type %1.encoded ') do (
		rem this not works - left an empty spaxes after each line from typed file
		rem null< (set /p #=%%A) >>%1.encoded
		
		set result=!result!%%A
	)
)
endlocal & set result=%result%
echo %result%>> %2%
del /S /Q
endlocal


The bad thing is that the base64 strings are stored in a variable and there's a limitations for it's size.But for a small strings it wokrs.






2.   Much more interesting.In the help it shows that there's an -decodehex switch.And I was surprised to find that there's also an undocumentes switch -encodehex (strange - decodehex looks more dangerous , because it can be used to produce binaries). Here's an example structure of encoded file:

0000	73 65 74 20 78 3d 15 0d  0a 73 65 74 20 79 3d 12   set x=...set y=.
0010	0d 0a 65 63 68 6f 20 2a  2a 25 78 25 25 79 25 2a   ..echo **%x%%y%*
0020	2a                                                 *

 

(just as every hex editor)

This was my attempt to create file that set LF and CR to variables smile - and I still don't know why it was unsuccessful it prints  "§↕"   (any help here will be welcomed - I don't know what goes wrong).
EDIT: I've set DEC codes instead HEX .Anyway still not works.
In fact the data that is behind the hexes is not necessary - it's only for visualisation .
To decode the sample just use this (if this above is saved in sample.hex) :

certutil -decodehex sample.hex not.working.bat

Any way it works for creation of the famous beep.bat :


0000	65 63 68 6f 20 07

this pattern can be used for creation of bat that echoes a random symbols by hex.Just edit the last character.

and this is a pattern for setting a symbol by hex to %#% variable (just edit the last character):

0000	73 65 74 20 78 3d 15 0d  

Anyway there are few more things that I want to try with this smile


here's a how forfiles can be used for hex symbols

Last edited by npocmaka (12 Dec 2012 00:05)


http://www.facebook.com/npocmaka

Offline

#2 07 Nov 2012 06:57

dbenham
Member
From: U.S. east coast
Registered: 15 Apr 2012
Posts: 100

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Very very cool - I've never seen this command before. I love the encocdehex / decodehex options.

Unfortunately, CERTUTIL is not standard on XP, though I believe there is a download that can be installed.

Your attempt to create a variable with LF and CR is doing exactly what I would expect it to do.

Here is a working HEX2STR.BAT script that can convert any string encoded as hex into a valid text string with 2 exceptions:

1) if the result is returned in a variable then the hex string cannot contain 00 (the nul character)

2) if the result is printed to the screen and not stored in a variable then the result will be truncated at the first occurence of 1A. Also, any 00 will show up on the screen as a space.

The script will work properly regardless whether delayed expansion is enabled or not.

@echo off
:hex2str  HexStr  [RtnVar]
::
:: Convert a hexadecimal encoded string HexStr into a text string.
::
:: Return the result in RtnVar. RtnVar will be undefined if HexStr
:: contains 00 (the encoding for the nul byte).
::
:: Print the result to to screen (stdout) if RtnVar is not specified.
:: The screen output will be truncated at the first occurance of 0x1A.
::
:: HexStr - The hex encoded string to be converted. Each character must be
::          encoded as a pair of hexadecimal digits. Hex pairs may be
::          delimited by one or more space, tab, or new line characters.
::
:: RtnVar - The optional name of a variable used to store the result.
::
setlocal
set "NotDelayed=!"
setlocal enableDelayedExpansion
set "tempFile=!temp!\hex2str%random%"
if "%~2" equ "" (
  >"!tempFile!.hex" echo %~1
  >nul 2>&1 certutil -f -decodehex "!tempFile!.hex" "!tempFile!.txt"
  type "!tempFile!.txt"
  2>nul del "!tempFile!.hex" "!tempFile!.txt"
  exit /b
)
set "hex=%~1"
set "hex=!hex:25=25 35!"
set "hex=!hex:22=25 7e 36!"
set "hex=!hex:0A=25 7e 33!"
set "hex=!hex:0D=25 34!"
if not defined NotDelayed (
  set "hex=!hex:5e=5e 5e!"
  set "hex=!hex:21=5e 21!"
)
>"!tempFile!.hex" echo !hex!
>nul 2>&1 certutil -f -decodehex "!tempFile!.hex" "!tempFile!.txt"
setlocal disableDelayedExpansion
for /f usebackq^ delims^=^ eol^= %%a in ("%tempFile%.txt") do set "rtn=%%a"
setlocal enableDelayedExpansion
set LF=^


set "replace=%% """"
for %%3 in ("!LF!") do for /f %%4 in ('copy /Z "%~dpf0" nul') do (
  for /f "tokens=1,2" %%5 in ("!replace!") do (
    endlocal
    endlocal
    endlocal
    endlocal
    set "%~2=%rtn%" !
    2>nul del "%tempFile%.hex" "%tempFile%.txt"
  )
)
exit /b

To get a LF variable, you can use CALL HEX2STR 0A LF

I have also written a script to go in reverse - STR2HEX.BAT.

The output contains just the encoded hex data, without the leading address info and without the trailing ASCII representation. update - added a /C (compress) option to strip spaces from the output

@echo off
:str2hex  StrVar  [RtnVar]  [/C]
::
:: Encode the string within variable StrVar as hexadecimal.
::
:: Return the result in RtnVar
:: or
:: Print the results to the screen (stdout) if RtnVar is not specified.
::
:: StrVar - The name of the variable containing the string to be encoded.
::
:: RtnVar - The optional name of a variable used to store the result.
::
::     /C - Compress option: All spaces are stripped from the output.
::          The RtnVar must be specified to use this option. Compressed
::          output may be sent to the screen by specifying an empty quoted
::          string for the RtnVar.
::
setlocal enableDelayedExpansion
set "tempFile=%temp%\hex2str%random%"
>"!tempFile!.txt" echo(!%~1!
>nul 2>&1 certutil -f -encodehex "!tempFile!.txt" "!tempFile!.hex"
set "hex="
set "delim= "
if /i "%~3" equ "/C" set "delim="
for /f "usebackq tokens=1*" %%A in ("!tempFile!.hex") do (
  set "ln=%%B"
  set "ln=!ln:~0,48!"
  if /i "%~3" equ "/C" (set "ln=!ln: =!") else set "ln=!ln:  = !"
if "%~2" neq "" (set "hex=!hex!!ln!%delim%") else echo !ln!
)
2>nul del "!tempFile!.hex" "!tempFile!.txt"
if "%~2" neq "" (
  endlocal
  set "%~2=%hex%"
)
exit /b

Dave Benham

Last edited by dbenham (08 Nov 2012 02:13)

Offline

#3 07 Nov 2012 16:38

npocmaka
Member
From: Bulgaria
Registered: 03 Dec 2009
Posts: 328

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Hello Dave,

Thanks for the help.

I thought that the line identificators and spaces between HEXes in encoded file are mandatory.
But it can process the file even without them ! Never came to my mind to try that - this makes its use far more easier.
/*
   off-topic
   I don't completely understand everything in your code ,
   but for a first time I pay enough attention to
   overlapped setlocal/endlocal and parentheses - it's realy
   amazing . You still have access to FOR tokens ,and you can use
   enableddelayedexpansion features?!?!!
   
   and what is this lonely ! here(hex2str):
   

 set "%~2=%rtn%" !

  ?
 
  this looks very twisted smile

*/

You're right about XP - it's not native , but can be installed - the first hit from google => http://www.microsoft.com/en-us/download … x?id=16770  (Administration Tools Pack).

One of the next things on my list that I want to try (in fact it's almost done ) is to convert to string to hex without temp file -> with certutil -dump file you can receive hex codes right in the console.

One more off-hex topic:

And one more feature of certutil that I want to emphasize on.  -URLCache -v   - this displays the whole internet history and locations of cache files.The most valuable of this is that it searches
trough ..\AppData\Local\Microsoft\Windows\Temporary Internet Files\Content.IE5\XXXXX\ files. As I often use internet explorer as a command line http client via its com-objects I really want to have access to cached files.But the folders under Content.IE5 are über-hidden and über-protected (if don't believe me try to access them with windows explorer or search there with DIR).Although the files can be copied if you know their exact location.The only one other tool that I know that can do this is IECacheView by NirSoft .

Looks like the most valuable features of certutil for me have nothing to do with certificates smile

Last edited by npocmaka (08 Nov 2012 08:50)


http://www.facebook.com/npocmaka

Offline

#4 08 Nov 2012 04:16

dbenham
Member
From: U.S. east coast
Registered: 15 Apr 2012
Posts: 100

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

I edited my prior post: I fixed the docs a bit for hex2str, and added a /C (compress) option to the str2hex.

The hex2str uses a safe return technique that allows the return of any string value across the ENDLOCAL barrier. The rogue ! is a critical piece of code that helps ensure the safe return technique works regardless whether or not delayed expansion is enabled within the calling routine. jeb developed the technique in support of routines I was writing to inter-convert between ASCII codes and string values: Re: new functions: :chr, :asc, :asciiMap. aGerman asks about the ! in the next post in the thread, and jeb replies with an explanation.

I see how -dump can eliminate the need to write the result of str2hex to a temporary file. But I don't see how you can avoid writing the source string to a temporary file.

The -encode and -decode functions are generally reciprocal in nature, but -decode is a bit too clever for its own good. It allows a high degree of flexibility as to the format of the hex source. But it can get confused as to what is a hex pair and what is ASCII text. If you encode a text file containing nothing but hex digits and spaces, and then decode the result, you do not get the original text file. My str2hex script preserves only the hex pairs in the output, so it is a true reciprocal of hex2str.


Dave Benham

Last edited by dbenham (08 Nov 2012 04:18)

Offline

#5 09 Jan 2013 21:30

Simon Sheppard
Super Administrator
Registered: 27 Aug 2005
Posts: 764
Website

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

npocmaka wrote:

2.   Much more interesting.In the help it shows that there's an -decodehex switch.And I was surprised to find that there's also an undocumented switch -encodehex (strange - decodehex looks more dangerous , because it can be used to produce binaries).

Great find npocmaka,
I have now added -encodehex to the new CERTUTIL page along with a link back to this thread.

I wouldn't be surprised if they left it out of the documentation by accident.

Offline

#6 09 Jan 2013 22:51

npocmaka
Member
From: Bulgaria
Registered: 03 Dec 2009
Posts: 328

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Yey.I have a credit :-)


http://www.facebook.com/npocmaka

Offline

#7 10 Jan 2013 03:08

carlos
Member
From: Chile
Registered: 04 Nov 2008
Posts: 232
Website

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Is not to be disagreeable, but it is documented here:

http://msdn.microsoft.com/en-us/subscriptions/cc772656%28v=ws.10%29.aspx#BKMK_EncodeHex

but the valids formats of the hex file for -decodehex is undocumented.
Anyways, the -encodehex not appear in the help of certutil of windows 2000 server,  windows 2003 server, windows 7).

Last edited by carlos (10 Jan 2013 03:21)

Offline

#8 12 Jan 2013 13:42

Simon Sheppard
Super Administrator
Registered: 27 Aug 2005
Posts: 764
Website

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

carlos wrote:

Is not to be disagreeable, but it is documented here:

Yes good catch, I've updated the SS64 page to note that there are a few small differences between the command line help (CERTUTIL -?) and the various MSDN online pages.
In many cases the -f force and -v verbose options make no difference, but I'm not sure if that means they are superfluous options or they just had nothing to do in the examples I tried.

It is a beast of a command and I'm tempted to split it across several pages, but I think on balance it's better being able to find things on one page with CTRL-F

Offline

#9 26 Apr 2013 15:47

npocmaka
Member
From: Bulgaria
Registered: 03 Dec 2009
Posts: 328

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Or -dump option is buggy or does not dump file in hex and  is intended only for certification files (but with bigger files it usually works as hex dumper)?

C:\Users\>echo aa > aa.txt

C:\Users\>type aa.txt
aa

C:\Users\>certutil -dump aa.txt
    69                                                 i
CertUtil: -dump command completed successfully.

C:\Users\>certutil -encodehex aa.txt aa.hex.txt
Input Length = 5
Output Length = 63
CertUtil: -encodehex command completed successfully.

C:\Users\>type aa.hex.txt
0000    61 61 20 0d 0a                                     aa ..

C:\Users\>

dump never works with utf-16 files (i don't know for other patterns...)
-encodehex on the other hand has no problems.
The help only says

  -dump             -- Dump configuration information or files

Last edited by npocmaka (26 Apr 2013 16:20)


http://www.facebook.com/npocmaka

Offline

#10 27 Apr 2013 14:47

Simon Sheppard
Super Administrator
Registered: 27 Aug 2005
Posts: 764
Website

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

Good point, I've changed the SS64 description.

It does look like CertUtil is very much built for handling certificates, it's probably never been tested as a general purpose utility - such is the Microsoft way!

Offline

#11 20 May 2013 09:31

npocmaka
Member
From: Bulgaria
Registered: 03 Dec 2009
Posts: 328

Re: certutil - decode/encode BASE64/HEX strings.Print symbols by HEX code.

eventually it will be fixed in win server 2012 :

https://connect.microsoft.com/WindowsSe … n-is-buggy

smile


http://www.facebook.com/npocmaka

Offline

Board footer

Powered by FluxBB