I am making a inverted index using hadoop and python. I want to know how can I include the byte offset of a line/word in python. I need something like this
hello hello.txt@1124
I need the locations for making a full inverted index. Please help.
Like this?
file.tell()
Return the file’s current position, like stdio's ftell().
http://docs.python.org/library/stdtypes.html#file-objects
Unfortunately tell() does not function since OP is using stdin instead of a file. But it is not hard to build a wrapper around it to give what you need.
class file_with_pos(object):
def __init__(self, fp):
self.fp = fp
self.pos = 0
def read(self, *args):
data = self.fp.read(*args)
self.pos += len(data)
return data
def tell(self):
return self.pos
Then you can use this instead:
fp = file_with_pos(sys.stdin)
来源:https://stackoverflow.com/questions/3642088/how-to-get-byte-offset-in-a-file-in-python