Python help to debug an error when modify a file in a stream mode

问题

I have the following problem. I am reading a file x,y,z as:

481492.93 6244326.24 26.56
481493.03 6244325.60 25.06
481493.17 6244324.68 22.89
481493.50 6244322.52 17.80
481492.84 6244327.05 27.84
481492.90 6244326.66 26.90
481492.86 6244327.16 27.45
481493.48 6244323.08 17.79
481492.80 6244327.80 28.30
481492.94 6244326.84 26.04
..........................

i wish to read, modify, and write on the same file (without create a back-up file because the originals file is more than 10GB)

481492.93 6244326.24 26.56 (375, 2902)
481493.03 6244325.60 25.06 (376, 2902)
481493.17 6244324.68 22.89 (377, 2902)
481493.50 6244322.52 17.80 (379, 2903)
481492.84 6244327.05 27.84 (375, 2902)
481492.90 6244326.66 26.90 (375, 2902)
481492.86 6244327.16 27.45 (374, 2902)
481493.48 6244323.08 17.79 (379, 2903)
481492.80 6244327.80 28.30 (374, 2902)
481492.94 6244326.84 26.04 (375, 2902)
..........................

i wrote the following approach

def get_point_grid_id(x,y,x_min,y_max,x_dist,y_dist):
        col = int((x - x_min)/x_dist)
        row = int((y_max - y)/y_dist)
        return (row, col)

with open(file_temp, "r+") as f:
    for line in open(file_temp):
        x,y,z = line.split()
        id = get_point_grid_id(float(x),float(y),origin[0],origin[1],1,1)
        element = [x,y,z,id]
        newelement = " ".join([str(e) for e in element])+ "\n"
        f.write(newelement)

when i run the function i got the following error

Traceback (most recent call last):
  File "<editor selection>", line 3, in <module>
ValueError: too many values to unpack

i suppose it's a connection problem with the original file

the error appears

>>> x,y,z = line.split()
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
ValueError: too many values to unpack

where line is strange

'481499.82 6244470.31 29.23 (231, 2909)\n' 

instead of '481499.82 6244470.31 29.23\n'

using print line after for line in open(file_temp): i got this print after run from a new file

481499.98 6244494.02 34.14
481499.98 6244494.02 34.14 (208, 2909)
481499.96 6244471.05 33.39
481499.96 6244471.05 33.39 (231, 2909)
481499.95 6244471.27 33.46
481499.95 6244471.27 33.46 (230, 2909)
481499.98 6244473.84 32.72
481499.98 6244473.84 32.72 (228, 2909)
481499.98 6244474.07 32.70
481499.98 6244474.07 32.70 (228, 2909)
481499.97 6244474.28 32.93
481499.97 6244474.28 32.93 (227, 2909)
481499.88 6244474.40 34.35
481499.88 6244474.40 34.35 (227, 2909)

回答1:

opening the line in r+ mode means that you read a line, ie 38 characters read. Then you modify those 38 characters Then, at the current file position (character 39) you over write the existing data

I would guess this is not what you want

Hope this helps

回答2:

This just isn't going to work. As Martijn said,

I

file objects have a buffer position. Every time you read a character the buffer position advances by 1. Suppose you read a line that's 10 characters long:

>>> myfile = open('some_file.txt')
>>> myfile.tell() #gets the buffer position
0
>>> myfile.readline()
'012345678\n'

Now the buffer position is advanced by len(line) characters:

>>> myfile.tell()
10

This means that when you call myfile.write(), it starts writing at position 10.

II

You simply can't "insert" characters into a file without overwriting something, or appending characters to the end (assuming that the buffer position is at the end of the file).

So what do you do?

You can create a temporary file, and simultaneously read from your input file, and write to your temp file. Afterwards (if you should wish), you can replace your original file with the temporary one:

with open(input_file) as infile, open(output_temp_file, "w") as outfile:
    for line in infile:
        x, y, z = line.split()
        new_line = ' '.join([x, y, z] + [function_of_xyz(x, y, z)]) + '\n'
        outfile.write(new_line)

You should also check out the csv module.

来源：https://stackoverflow.com/questions/15033309/python-help-to-debug-an-error-when-modify-a-file-in-a-stream-mode