问题
I imaging this is going to be a simple task but I can't find what I am looking for exactly in previous StackOverflow questions to here goes...
I have large text files in a proprietry format that look comething like this:
:Entry
- Name
John Doe
- Date
20/12/1979
:Entry
-Name
Jane Doe
- Date
21/12/1979
And so forth.
The text files range in size from 10kb to 100mb. I need to split this file by the :Entry
delimiter. How could I process each file based on :Entry
blocks?
回答1:
You could use itertools.groupby to group lines that occur after :Entry
into lists:
import itertools as it
filename='test.dat'
with open(filename,'r') as f:
for key,group in it.groupby(f,lambda line: line.startswith(':Entry')):
if not key:
group = list(group)
print(group)
yields
['- Name\n', 'John Doe\n', '\n', '- Date\n', '20/12/1979\n']
['\n', '-Name\n', 'Jane Doe\n', '- Date\n', '21/12/1979\n']
Or, to process the groups, you don't really need to convert group
to a list:
with open(filename,'r') as f:
for key,group in it.groupby(f,lambda line: line.startswith(':Entry')):
if not key:
for line in group:
...
回答2:
If every entry block starts with a colon, you can just split by that:
with open('entries.txt') as fp:
contents = fp.read()
for entry in contents.split(':'):
# do something with entry
来源:https://stackoverflow.com/questions/7980288/splitting-large-text-file-by-a-delimiter-in-python