MPI-IO: MPI_File_Set_View vs. MPI_File_Seek

问题

I encounter a problem when trying to write a file with MPI-IO, in Fortran 90. If I do the following, using MPI_File_Set_View

program test
  implicit none

  include "mpif.h"

  integer :: myrank, nproc, fhandle, ierr
  integer :: xpos, ypos
  integer, parameter :: loc_x=10, loc_y=10
  integer :: loc_dim
  integer :: nx=2, ny=2
  real(8), dimension(loc_x, loc_y) :: data, data_read
  integer :: written_arr
  integer, dimension(2) :: wa_size, wa_subsize, wa_start
  integer :: int_size, double_size
  integer(kind=MPI_OFFSET_KIND) :: offset

  call MPI_Init(ierr)
  call MPI_Comm_Rank(MPI_COMM_WORLD, myrank, ierr)
  call MPI_Comm_Size(MPI_COMM_WORLD, nproc, ierr)

  xpos = mod(myrank, nx)
  ypos = mod(myrank/nx, ny)

  data = myrank

  loc_dim    = loc_x*loc_y

  ! Write using MPI_File_Set_View
  wa_size    = (/ nx*loc_x, ny*loc_y /)
  wa_subsize = (/ loc_x, loc_y /)
  wa_start   = (/ xpos, ypos /)*wa_subsize
  call MPI_Type_Create_Subarray(2, wa_size, wa_subsize, wa_start &
       , MPI_ORDER_FORTRAN, MPI_DOUBLE_PRECISION, written_arr, ierr)
  call MPI_Type_Commit(written_arr, ierr)

  call MPI_Type_Size(MPI_INTEGER, int_size, ierr)
  call MPI_Type_Size(MPI_DOUBLE_PRECISION, double_size, ierr)

  call MPI_File_Open(MPI_COMM_WORLD, "file_set_view.dat" &
       , MPI_MODE_WRONLY + MPI_MODE_CREATE, MPI_INFO_NULL, fhandle, ierr)
  call MPI_File_Set_View(fhandle, 0, MPI_DOUBLE_PRECISION, written_arr &
       , "native", MPI_INFO_NULL, ierr)
  call MPI_File_Write_All(fhandle, data, loc_dim, MPI_DOUBLE_PRECISION &
       , MPI_STATUS_IGNORE, ierr)
  call MPI_File_Close(fhandle, ierr)

  call MPI_Finalize(ierr)

end program test

I get a 69Go file, which is way too big considering what I am writing in it. By the way, the size of the file does not change if I increase loc_x and loc_y.

However, if I use MPI_File_Seek, it works much better; a file of a reasonable size is created, containing the data I want to write

program test
  implicit none

  include "mpif.h"

  integer :: myrank, nproc, fhandle, ierr
  integer :: xpos, ypos
  integer, parameter :: loc_x=10, loc_y=10
  integer :: loc_dim
  integer :: nx=2, ny=2
  real(8), dimension(loc_x, loc_y) :: data, data_read
  integer :: written_arr
  integer, dimension(2) :: wa_size, wa_subsize, wa_start
  integer :: int_size, double_size
  integer(kind=MPI_OFFSET_KIND) :: offset

  call MPI_Init(ierr)
  call MPI_Comm_Rank(MPI_COMM_WORLD, myrank, ierr)
  call MPI_Comm_Size(MPI_COMM_WORLD, nproc, ierr)

  xpos = mod(myrank, nx)
  ypos = mod(myrank/nx, ny)

  data = myrank

  loc_dim    = loc_x*loc_y

  ! Write using MPI_File_Seek
  call MPI_File_Open(MPI_COMM_WORLD, "file_seek.dat" &
       , MPI_MODE_WRONLY + MPI_MODE_CREATE, MPI_INFO_NULL, fhandle, ierr)
  offset = loc_x*loc_y*myrank
  print*, 'myrank, offset, data: ', myrank, offset, data(1,:2)
  call MPI_File_Seek(fhandle, offset, MPI_SEEK_SET)
  call MPI_File_Write_All(fhandle, data, loc_dim, MPI_DOUBLE_PRECISION &
       , MPI_STATUS_IGNORE, ierr)
  call MPI_File_Close(fhandle, ierr)

  call MPI_Finalize(ierr)

end program test

It seems to me that these two methods should produce the same thing, and, in particular, that the first method should create a so large file.

I compile my code with gfortran 4.6.3 and OpenMPI 1.6.2.

Any help would be appreciated!

回答1:

The answer was actually given in Hristo Iliev's answer of this question:

Replace the 0 in the MPI_FILE_SET_VIEW call with 0_MPI_OFFSET_KIND or declare a constant of type INTEGER(KIND=MPI_OFFSET_KIND) and a value of zero and then pass it.
call MPI_File_Set_View(fhandle, 0_MPI_OFFSET_KIND, MPI_DOUBLE_PRECISION, ...
or
integer(kind=MPI_OFFSET_KIND), parameter :: zero_off = 0
...
call MPI_File_Set_View(fhandle, zero_off, MPI_DOUBLE_PRECISION, ...
Both methods lead to an output file of size 3200 bytes (as expected).

来源：https://stackoverflow.com/questions/18462088/mpi-io-mpi-file-set-view-vs-mpi-file-seek

标签

fortran

mpi

mpi-io