Computer Forensics and Data Analysis
Software Training Services  
      Search:
NSRL Hash Set

Latest update (as of December 2017) of a subset of the NSRL data available here.

I have taken two data sets from the NIST NSRL website and massaged the data files to produce two output files.

The original data set is from the link from "Current RDS Hash Set". Modern RDS (link is external) (microcomputer applications from 2000 to present).
Found on this page. This I assume (I have not been able to get anyone from NIST to confirm what I am about to say), is a complete set of hashes which they have processed. My processing of the data indicated that the original volume contained approximately 112,884,111, (14.5Gig) items. Whether this set includes IOS and/or Android, NIST explanation is not clear (at least not to me).

These 112 million items were reduced and de-duped on SHA which resulted in approximately 38,312,509 unique SHA values. These SHA along with their MD5 values were output to a single fixed length record file. The format of the output file is a comma delimieted fixed length file containing three fields: SHA, MD5, CRC

"0000002D9D62AEBE1E0E9DB6C4C4C7C16A163D2C","1D6EBB5A789ABD108FF578263E1F40F3","FFFFFFFF"
"00000142988AFA836117B1B572FAE4713F200567","9B3702B0E788C6D62996392FE3C9786A","05E566DF"
This should be easily massaged for input to most any program capable of handling the values. (Although i must say, I have complained for some time, that import to X-Ways is difficult if the first record doesn't contain the word SHA or MD5. ) I Leave it to the user to figure out how to get the first record adjusted accordingly.

In an effort to make life easy for others, I have then taken only the MD5 values and placed them in a data file which looks like

00000040F69913A27FF7401B8BF3CFD1
000001EEB4F15B705A97758C91F2C73A
Notice again, NO MD5 header. You are a forensic expert, figure how to add the header. :-).

There is also a subset of approximately 23,598,567 items which is a unique set of SHA values from the "Modern RDS (link is external) (unique) " found on this page.

"0000002D9D62AEBE1E0E9DB6C4C4C7C16A163D2C","1D6EBB5A789ABD108FF578263E1F40F3","FFFFFFFF"
"000001E4975FA18878DF5C0989024327FBE1F4DF","AA1F522B2AA3EFDCD8D813DE3D1B2451","01BAC904"

This sub-set is currently not on the website. But can be made available.

The two files mentioned above, the complete 33million SHA's, or the 33 Million MD5's can be downloaded here.

SHA/MD5   file. MD5   file.

They are .7z compression because of the large data sizes. But in order for these to be downloade, I had to add the zip extension. So, if you download don't forget to rename (remove the .zip) and work with the .7z file. Remember to drop the .zip when expandind.

A reference page for algorithms and other documemts may be found at: NIST.  Research the documents link.

Top

Files A-C  |  Files D-F  |  Files G-K  |  Files L-O  |  Files P-S  |  Files T-Z  |
 
copyright 1998-2016 by Mares and Company, LLC