hashlib

My current approach is this: def get_hash(path=PATH, hash_type='md5'): func = getattr(hashlib, hash_type)() with open(path, 'rb') as f: for block in iter(lambda: f.read(1024*func.block_size, b''): func.update(block) return func.hexdigest() It takes about 3.5 seconds to calculate the md5sum of a 842MB iso file on an i5 @ 1.7 GHz. I have tried different methods of reading the file, but all of them yield slower results. Is there, perhaps, a faster solution? EDIT: I replaced 2**16 (inside the f.read() ) with 1024*func.block_size , since the default block_size for most hashing functions supported

How do I convert an unsigned integer (representing a user ID) to a random looking but actually a deterministically repeatable choice? The choice must be selected with equal probability (irrespective of the distribution of the the input integers). For example, if I have 3 choices, i.e. [0, 1, 2] , the user ID 123 may always be randomly assigned choice 2, whereas the user ID 234 may always be assigned choice 1. Cross-language and cross-platform algorithmic reproducibility is desirable. I'm inclined to use a hash function and modulo unless there is a better way. Here is what I have: >>> num

Well, today I was checking the hashlib module in python, but then I found something that I still can't figure out. Inside this python module, there is an import that I can't follow. I goes like this: def __get_builtin_constructor(name): if name in ('SHA1', 'sha1'): import _sha return _sha.new I tried to import the _sha module from a python shell, but is seems that it cannot be reached that way.My first guess is that it's a C module, but I'm not sure. So tell me guys, do you know where is that module? How do they import it? Actually, the _sha module is provided by shamodule.c and _md5 is

How do I convert a string, e.g. a user ID plus salt, to a random looking but actually a deterministically repeatable uniform probability in the semi-open range [0.0, 1.0)? This means that the output is ≥ 0.0 and < 1.0. The output distribution must be uniform irrespective of the input distribution. For example, if the input string is 'a3b2Foobar', the output probability could repeatably be 0.40341504. Cross-language and cross-platform algorithmic reproducibility is desirable. I'm inclined to use a hash function unless there is a better way. Here is what I have: >>> in_str = 'a3b2Foobar' >>>

I'm trying to create an HMAC-SHA512 signed request for an API call in Python 3.4 using the requests library. I'm trying to follow docs, but am hitting this error: AttributeError: '_hashlib.HASH' object has no attribute 'new' Here's some code. It's failing with the error on the hmac constructor. It's fine if I try and pass hashlib.md5() or omit the digest parameter entirely. I'm not sure if I'm signing the request properly afterwards as I haven't got that far yet. The docs for the service I'm trying to use say to sign the URL with my secret. I need this to be a byte string for this to work.

I'm trying to replicate the behavior of a Python 2.7 function in Java , but I'm getting different results when running a (seemingly) identical sequence of bytes through a SHA-256 hash. The bytes are generated by manipulating a very large integer (exactly 2048 bits long) in a specific way (2nd line of my Python code example). For my examples, the original 2048-bit integer is stored as big_int and bigInt in Python and Java respectively, and both variables contain the same number. Python2 code I'm trying to replicate: raw_big_int = ("%x" % big_int).decode("hex") buff = struct.pack(">i", len(raw