MD5sum DLL for Windows

[marinaz28]

Hi!

I have a script, which works like rsync, but it's very slow, because count MD5 using pure rexx. I have DLL for OS/2, very old, srxfunc.dll by Daniel Hellerstein, but can't find anything for Windows-32 bit.

 

[mikrom]

There is COM objekt:
System.Security.Cryptography.MD5CryptoServiceProvider
see

http://stackoverflow.com/questions/...ulates-a-md5-sha1-hash-in-windows-2003-server

http://stackoverflow.com/questions/10198690/how-to-generate-md5-using-vb-in-classic-asp

The VBscript example from the second link worked for me.
So you could try to use Open Object REXX to access the COM objects and call the methods you need.

 

[mikrom]

IMO the simplest for a REXX program would be to use a command.

On Unix like systems there is a nice command md5sum which does everything we need.
I have MinGW + MSYS installed on my windows and therefore I have this command available.
But you can get it without installing of MinGW too:
For example I downloaded here

http://blog.nfllab.com/archives/152-Win32-native-md5sum,-sha1sum,-sha256sum-etc..html

these binaries compiled from coreutils-6.11.

So with this version

c:\_mikrom\Work\MD5_example>md5sum --version
md5sum (GNU coreutils) 6.11
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <[URL unfurl="true"]http://gnu.org/licenses/gpl.html>[/URL]
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Ulrich Drepper, Scott Miller, and David Madore.

I tried this code

[COLOR=#0000ff]-- define temporary I/O files --[/color]
input_file [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"tmp_md5_inp.txt"[/color]
output_file [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"tmp_md5_out.txt"[/color]
[COLOR=#0000ff]--------------------------------[/color]

str [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"Hello World"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_string([/color]str[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

str [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"The quick brown fox jumps over the lazy dog"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_string([/color]str[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

fpath[COLOR=#804040][b]=[/b][/color][COLOR=#ff00ff]"winmd5free.zip"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_file([/color]fpath[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]fpath[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

fpath[COLOR=#804040][b]=[/b][/color][COLOR=#ff00ff]"winmd5.exe"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_file([/color]fpath[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]fpath[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

[COLOR=#0000ff]----[/color]
[COLOR=#804040][b]exit[/b][/color]

[COLOR=#0000ff]/*      Functions        */[/color]
[COLOR=#008080]md5_from_string[/color][COLOR=#804040][b]:[/b][/color] [COLOR=#804040][b]procedure expose[/b][/color] input_file output_file
  [COLOR=#804040][b]parse arg[/b][/color] str
  [COLOR=#0000ff]/* write string into input_file */[/color]
  [COLOR=#0000ff]-- open temporary input file for writing[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]lineout[/color] input_file[COLOR=#804040][b],[/b][/color] [COLOR=#804040][b],[/b][/color] 1   
  [COLOR=#0000ff]-- write string into temporary input_file[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]charout[/color] input_file[COLOR=#804040][b],[/b][/color] str
  [COLOR=#0000ff]-- close temporary input_file[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]lineout[/color] input_file

  [COLOR=#0000ff]-- create and execute md5sum command with output into[/color]
  [COLOR=#0000ff]-- temporary output_file [/color]
  cmd [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"md5sum <"[/color] input_file [COLOR=#ff00ff]">"[/color] output_file
  cmd
  [COLOR=#0000ff]-- read result from temporary output_file[/color]
  [COLOR=#804040][b]do [/b][/color][COLOR=#804040][b]while[/b][/color][COLOR=#804040][b] [/b][/color][COLOR=#008080]lines([/color]output_file[COLOR=#008080])[/color] [COLOR=#804040][b]\=[/b][/color] 0
    rslt [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]linein([/color]output_file[COLOR=#008080])[/color]
  [COLOR=#804040][b]end[/b][/color]
  [COLOR=#0000ff]-- close temporary output_file[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]lineout[/color] output_file
  [COLOR=#0000ff]-- delete temporary input_file[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]delete_temp_file[/color] input_file
  [COLOR=#0000ff]-- delete temporary output_file[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]delete_temp_file[/color] output_file
  [COLOR=#0000ff]-- get 1.word[/color]
  rslt [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]word([/color]rslt[COLOR=#804040][b],[/b][/color] 1[COLOR=#008080])[/color]
  [COLOR=#0000ff]-- return result[/color]
  [COLOR=#804040][b]return[/b][/color] rslt

[COLOR=#008080]md5_from_file[/color][COLOR=#804040][b]:[/b][/color] [COLOR=#804040][b]procedure expose[/b][/color] output_file
  [COLOR=#804040][b]parse arg[/b][/color] file_path
  [COLOR=#0000ff]-- create and execute md5sum command with output into[/color]
  [COLOR=#0000ff]-- temporary output_file [/color]
  cmd [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"md5sum "[/color] file_path [COLOR=#ff00ff]">"[/color] output_file
  cmd
  [COLOR=#0000ff]-- read result from temporary output_file[/color]
  [COLOR=#804040][b]do [/b][/color][COLOR=#804040][b]while[/b][/color][COLOR=#804040][b] [/b][/color][COLOR=#008080]lines([/color]output_file[COLOR=#008080])[/color] [COLOR=#804040][b]\=[/b][/color] 0
    rslt [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]linein([/color]output_file[COLOR=#008080])[/color]
  [COLOR=#804040][b]end[/b][/color]
  [COLOR=#0000ff]-- close temporary output_file[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]lineout[/color] output_file
  [COLOR=#0000ff]-- delete temporary output_file[/color]
  [COLOR=#804040][b]call [/b][/color][COLOR=#008080]delete_temp_file[/color] output_file
  [COLOR=#0000ff]-- get 1.word[/color]
  rslt [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]word([/color]rslt[COLOR=#804040][b],[/b][/color] 1[COLOR=#008080])[/color]
  [COLOR=#0000ff]-- return result[/color]
  [COLOR=#804040][b]return[/b][/color] rslt

[COLOR=#008080]delete_temp_file[/color][COLOR=#804040][b]:[/b][/color] [COLOR=#804040][b]procedure[/b][/color]
  [COLOR=#804040][b]parse arg[/b][/color] tmp_md5_file
  [COLOR=#0000ff]-- delete temporary file[/color]
  cmd [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"del "[/color] tmp_md5_file
  cmd
  [COLOR=#804040][b]return[/b][/color]

Here is the output

c:\_mikrom\Work\MD5_example>rexx md5_example.rex
"Hello World"
"b10a8db164e0754105b7a99be72e3fe5"

"The quick brown fox jumps over the lazy dog"
"9e107d9d372bb6826bd81d3542a419d6"

"winmd5free.zip"
"73f48840b60ab6da68b03acd322445ee"

"winmd5.exe"
"944a1e869969dd8a4b64ca5e6ebc209a"

I downloaded the files winmd5free.zip and winmd5.exe from here

http://winmd5.com/

to compare the MD5 hash values:
WinMD5Free.zip MD5: 73f48840b60ab6da68b03acd322445ee
WinMD5Free.exe MD5: 944a1e869969dd8a4b64ca5e6ebc209a

 

[marinaz28]

Thank you, mikrom, but I already do this

A bit revised fragment of my the script below:

sst2 = CHARIN(src_file,,1024*1024*4)
string_size = LENGTH(sst2)
IF string_size>5000 THEN DO
tmp_file='tmp_'||MD5(src_file)||'.bin'
RC3=SysFileDelete(tmp_file)
RC4=CHAROUT(tmp_file,SST2)
RC5=STREAM(tmp_file,"C","CLOSE")
ss2='@md5sums.exe -u '||tmp_file||' | rxqueue '
ss2
PARSE PULL MD5_r1
DO WHILE QUEUED()>0
PARSE PULL MD5_r2
END
MD5_result = TRANSLATE(SUBSTR(MD5_r1,1,32))
RC3=SysFileDelete(tmp_file)
END
ELSE DO
MD5_result = TRANSLATE(MD5(sst2))
END


MD5() is pure rexx-based MD5 count function. Nice, but VERY slow .

So, i will think to connect to Windows COM object, but it's completely new for me, I like classic rexx.

Why this? tmp_file='tmp_'||MD5(src_file)||'.bin'

Yes, i can generate temporal name by REXX , but this looks better

 

[mikrom]

So, i will think to connect to Windows COM object, but it's completely new for me, I like classic rexx.

I had some experiences with COM objects before - for example with ADO - and so I tried to rewrite the VBscript code posted here

http://stackoverflow.com/questions/10198690/how-to-generate-md5-using-vb-in-classic-asp

into REXX.
..But I didn't succeed I had a strange problem: the output of the function stringToUTFBytes was not datatype-compatible with the input of the function md5hashBytes.

On the other hand it did not make much sense. If you have a Unix-utility such as md5sum available on Windows, it is much easier to take advantage of this and also a multiplatform solution.

 

[mikrom]

Lately I really use REXX very rarely, so I have almost forgotten the RXQUEUE.
But the code of marinaz28 reminded me again

I have looked at the other command line utility:
md5 from here

https://www.fourmilab.ch/md5/

because it has this option -d which makes possible to calculate MD5 hash from a string without a need of redirecting it from temporary input file. Then I don't need temporary files and could use RXQUEUE.

Now everything is simpler:

str [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"Hello World"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_string([/color]str[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

str [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"The quick brown fox jumps over the lazy dog"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_string([/color]str[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

fpath[COLOR=#804040][b]=[/b][/color][COLOR=#ff00ff]"winmd5free.zip"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_file([/color]fpath[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]fpath[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

fpath[COLOR=#804040][b]=[/b][/color][COLOR=#ff00ff]"winmd5.exe"[/color]
md5_str [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]md5_from_file([/color]fpath[COLOR=#008080])[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]fpath[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color] [COLOR=#ff00ff]""""[/color]md5_str[COLOR=#ff00ff]""""[/color]
[COLOR=#804040][b]say[/b][/color]

[COLOR=#0000ff]----[/color]
[COLOR=#804040][b]exit[/b][/color]

[COLOR=#0000ff]/*      Functions        */[/color]
[COLOR=#008080]md5_from_string[/color][COLOR=#804040][b]:[/b][/color] [COLOR=#804040][b]procedure expose[/b][/color] input_file output_file
  [COLOR=#804040][b]parse arg[/b][/color] str
  [COLOR=#0000ff]-- create and execute md5sum command with output into RXQUEUE[/color]
  cmd [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"md5 -d"""[/color][COLOR=#804040][b]||[/b][/color]str[COLOR=#804040][b]||[/b][/color][COLOR=#ff00ff]""" | rxqueue"[/color]
  cmd
  [COLOR=#804040][b]parse pull[/b][/color] rslt
  [COLOR=#804040][b]return[/b][/color] rslt

[COLOR=#008080]md5_from_file[/color][COLOR=#804040][b]:[/b][/color] [COLOR=#804040][b]procedure expose[/b][/color] output_file
  [COLOR=#804040][b]parse arg[/b][/color] file_path
  [COLOR=#0000ff]-- create and execute md5sum command with output into RXQUEUE[/color]
  cmd [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]"md5sum "[/color][COLOR=#804040][b]||[/b][/color]file_path[COLOR=#804040][b]||[/b][/color][COLOR=#ff00ff]" | rxqueue"[/color]
  cmd
  [COLOR=#804040][b]parse pull[/b][/color] rslt
  [COLOR=#0000ff]-- get 1.word[/color]
  rslt [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]word([/color]rslt[COLOR=#804040][b],[/b][/color] 1[COLOR=#008080])[/color]
  [COLOR=#804040][b]return[/b][/color] rslt

The output is almost same as in the example before:

C:\Work\MD5_example>rexx md5_example.rex
"Hello World"
"B10A8DB164E0754105B7A99BE72E3FE5"

"The quick brown fox jumps over the lazy dog"
"9E107D9D372BB6826BD81D3542A419D6"

"winmd5free.zip"
"73f48840b60ab6da68b03acd322445ee"

"winmd5.exe"
"944a1e869969dd8a4b64ca5e6ebc209a"

 

[marinaz28]

I had some experiences with COM objects before - for example with ADO - and so I tried to rewrite the VBscript code posted here

http://stackoverflow.com/questions/10198690/how-to...

into REXX.
..But I didn't succeed smile I had a strange problem: the output of the function stringToUTFBytes was not datatype-compatible with the input of the function md5hashBytes.

Can You please, post this rewritten REXX sample? As I say before, I am 'zero' in OO/COM

 

[mikrom]

Here is the working VBscript example from the link given above:

str = "Hello World"
WScript.echo BytesToBase64(md5hashBytes(stringToUTFBytes(str)))
WScript.echo bytesToHex(md5hashBytes(stringToUTFBytes(str)))

str ="The quick brown fox jumps over the lazy dog"
WScript.echo BytesToBase64(md5hashBytes(stringToUTFBytes(str)))
WScript.echo bytesToHex(md5hashBytes(stringToUTFBytes(str)))

path="winmd5free.zip"
WScript.echo BytesToBase64(md5hashBytes(GetBytes(path)))
WScript.echo BytesToHex(md5hashBytes(GetBytes(path)))

path="winmd5.exe"
WScript.echo BytesToBase64(md5hashBytes(GetBytes(path)))
WScript.echo BytesToHex(md5hashBytes(GetBytes(path)))

'#*****************************************************************************
'# Functions
'#*****************************************************************************
function md5hashBytes(aBytes)
    Dim MD5
    set MD5 = CreateObject("System.Security.Cryptography.MD5CryptoServiceProvider")

    MD5.Initialize()
    'Note you MUST use computehash_2 to get the correct version of this method, and the bytes MUST be double wrapped in brackets to ensure they get passed in correctly.
    md5hashBytes = MD5.ComputeHash_2( (aBytes) )
end function

function stringToUTFBytes(aString)
    Dim UTF8
    Set UTF8 = CreateObject("System.Text.UTF8Encoding")
    stringToUTFBytes = UTF8.GetBytes_4(aString)
end function

function bytesToHex(aBytes)
    dim hexStr, x
    for x=1 to lenb(aBytes)
        hexStr= hex(ascb(midb( (aBytes),x,1)))
        if len(hexStr)=1 then hexStr="0" & hexStr
        bytesToHex=bytesToHex & hexStr
    next
end function

Function BytesToBase64(varBytes)
    With CreateObject("MSXML2.DomDocument").CreateElement("b64")
        .dataType = "bin.base64"
        .nodeTypedValue = varBytes
        BytesToBase64 = .Text
    End With
End Function

Function GetBytes(sPath)
    With CreateObject("Adodb.Stream")
        .Type = 1 ' adTypeBinary
        .Open
        .LoadFromFile sPath
        .Position = 0
        GetBytes = .Read
        .Close
    End With
End Function

You can try it with

c:\_mikrom\Work\MD5_example>[highlight]cscript /NoLogo md5_example.vbs[/highlight]
sQqNsWTgdUEFt6mb5y4/5Q==
B10A8DB164E0754105B7A99BE72E3FE5
nhB9nTcrtoJr2B01QqQZ1g==
9E107D9D372BB6826BD81D3542A419D6
c/SIQLYKttposDrNMiRF7g==
73F48840B60AB6DA68B03ACD322445EE
lEoehplp3YpLZMpebrwgmg==
944A1E869969DD8A4B64CA5E6EBC209A

and here is, what I tried to rewrite into ooREXX, which doesn't work

str = "Hello World"
say BytesToBase64(md5hashBytes(stringToUTFBytes(str)))
--say bytesToHex(md5hashBytes(stringToUTFBytes(str)))

str ="The quick brown fox jumps over the lazy dog"
say BytesToBase64(md5hashBytes(stringToUTFBytes(str)))
--say bytesToHex(md5hashBytes(stringToUTFBytes(str)))

path="winmd5free.zip"
say BytesToBase64(md5hashBytes(GetBytes(path)))
--say BytesToHex(md5hashBytes(GetBytes(path)))

path="winmd5.exe"
say BytesToBase64(md5hashBytes(GetBytes(path)))
--say BytesToHex(md5hashBytes(GetBytes(path)))

exit

/*
#******************************************************************************
# Functions
#******************************************************************************
*/
md5hashBytes: procedure
    parse arg aBytes
    MD5=.OLEObject~New("System.Security.Cryptography.MD5CryptoServiceProvider")
    MD5~Initialize()
    /* 
    Note you MUST use computehash_2 to get the correct version of this method, 
    and the bytes MUST be double wrapped in brackets to ensure they get passed 
    in correctly 
    */
    md5hashBytes=MD5~ComputeHash_2( (aBytes) )
    return md5hashBytes

stringToUTFBytes : procedure
    parse arg aString
    UTF8 = .OLEObject~New("System.Text.UTF8Encoding")
    stringToUTFBytes = UTF8~GetBytes_4(aString)
    return stringToUTFBytes

/*
-- TODO
bytesToHex: procedure
    use arg aBytes
    do x=1 to lenb(aBytes)
        hexStr= hex(ascb(midb( (aBytes),x,1)))
        if len(hexStr)=1 then hexStr="0" & hexStr
        bytesToHex=bytesToHex & hexStr
    end
    return bytesToHex
*/

BytesToBase64 : procedure 
    use arg varBytes
    obj= .OLEObject~New("MSXML2.DomDocument").CreateElement("b64")
    obj~dataType = "bin.base64"
    obj~.nodeTypedValue = varBytes
    BytesToBase64 = .Text
    return BytesToBase64

GetBytes : procedure
    use arg sPath
    say sPath
    obj= .OLEObject~New("Adodb.Stream")
    obj~Type = 1 -- adTypeBinary
    obj~Open
    obj~LoadFromFile(sPath)
    obj~Position = 0
    GetBytes = obj~Read
    obj~Close
    return GetBytes

If you will be able to find an error and to correct it, please post the solution.