hashdeep with sha1 and sha1sum create different results

Question

I computed a sha1 hash of the file '/etc/mtab' with hashdeep on Ubuntu 16.04. Then I computed a sha1 hash of the same file with sha1sum. And I got a different hash!

When computing hashes for other files, both programs seem to create the same hash. Finally, I copied the file to another computer and ran the two programs there. On the second computer they both computed the same hash that was computed on the first computer by sha1sum. At last, I checked the file '/etc/mtab' on the second computer, not the copied one, with hashdeep and sha1sum and I also got different results again.

The same goes for sha256 when using hashdeep and sha256sum.

I am calling hashdeep -c sha256 /etc/mtab and sha256sum /etc/mtab

Does anyone know how this can happen? Is there a caching implementation for hashdeep, thus older hashes are stored?

Answer 1

I just ran that on my system and it went well these were my results:

Ran: hashdeep -c sha256 Xwrapper.config

%%%% HASHDEEP-1.0
%%%% size,sha256,filename
## Invoked from: /home/george/Documents/askubuntu
## $ hashdeep -c sha256 Xwrapper.config
## 
630,c53eb030a8f9a2f7c08dd04e3611c9470f840d90d1261ef3e6643b1d1aff8608,/home/george/Documents/askubuntu/Xwrapper.config

Ran: sha256sum Xwrapper.config

c53eb030a8f9a2f7c08dd04e3611c9470f840d90d1261ef3e6643b1d1aff8608  Xwrapper.config

Ok let explain as best as I can, after checking several things I see some thing about how hashdeep and sha256sum see symbolic links. When I do stat /etc/mtab I see the size is 19, but when I run ( with the l, link option):

 hashdeep -o l /etc/mtab

It returns nothing but if I do:

 hashdeep -o f /atc/mtab

It returns the hash below but note that the size is 0 different from what stat /ec/mtab gave earlier:

 %%%% HASHDEEP-1.0
 %%%% size,md5,sha256,filename
 ## Invoked from: /etc
 ## $ hashdeep -o lf /etc/mtab
 ## 0,d41d8cd98f00b204e9800998ecf8427e,e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855,/etc/mtab

It sees the file as regular file since I used the f option and hence reports or calculates the sha256 from an empty file, whereas the sha256sum sees it as a symbolic link and of size 19. So simply put when the file is a regular file it gives similar results but not so when a link.

Note that that file is a link to proc/self/mounts which hashdeep still sees as 0 in size but is seen by stat as 19 size, so hashdeep does follow that link but seems not to be able to read the contents or see the contents of that file. It does however see the /etc/vtrgb as a non-empty file, so I guess /etc/mtab link to /proc/self/mounts matters.

It even gets more interesting and confirms my suspicion about /etc/mtab link to /proc/self/mounts as I ran this on another link hashdeep /etc/vtrgb:

%%%% HASHDEEP-1.0
%%%% size,md5,sha256,filename
## Invoked from: /etc
## $ hashdeep /etc/vtrgb
## 158,1fb3c13c4fcfa8cc4131aba905df559e,684cd905549f78e025870dd5c8a3835e49f79f2bb08952eb7424537f6df5fa13,/etc/vtrgb

As you will note the size is not zero as was the case with /etc/mtab, and running:

 sha256sum /etc/vtrgb

Returns the same checksum:

 684cd905549f78e025870dd5c8a3835e49f79f2bb08952eb7424537f6df5fa13  /etc/vtrgb

Further investigation shows that hashdeep returns the same hash

0,d41d8cd98f00b204e9800998ecf8427e,e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

for all files and directory in /proc/ which is actually a pseudo filesystem hence their information is stored in memory not on disk. Whereas sha256sum does not but returns different checksum for these same files.