问题
I am trying to write a bunch of large matrices in fortran to disk.
Matrices are V
, C
and d
. All have size (2 , n1, n2, n3, n4, n5).
These are large matrices. And fortran is taking around 3 hours to write them.
do ind1=1,n1
do ind2=1,n2
do ind3=1,n3
do ind4=1,n4
write(filename,'(a,i0,a,i0,a,i0,a,i0,a,i0,a)')'PF',t,'_',ind1,'_',ind2,'_',ind3,'_',ind4,'.txt'
OPEN(UNIT=25,FILE=filename,STATUS='replace',ACTION='write')
do ind5=1,n5
WRITE(25,*) c(2,ind1,ind2,ind3, ind4,ind5)
end do
do ind5=1,n5
WRITE(25,*) v(2,ind1,ind2,ind3, ind4,ind5)
end do
do ind5=1,n5
WRITE(25,*) d(1,ind1,ind2,ind3, ind4,ind5)
end do
CLOSE(UNIT=25)
end do
end do
end do
end do
Any smart way of doing this faster?
回答1:
Let me start by saying that I agree with the comments, the best way would be to move to something like HDF5 or NetCDF which should give you both the portability of the data and also high performance.
That said if you want to stick with simple Fortran I/O it's usually a good idea to keep the number of I/O transactions to a minimum. I suspect all your opens are not a good idea, I would stick it all in one file. If that's OK you then have 3 routes as shown below and run on my laptop (which admittedly has a SSD). Anyway it should give you some ideas.
ian-admin@agon ~/test $ cat io.f90
Program test_io
Implicit None
Integer, Parameter :: wp = Selected_real_kind( 12, 70 )
Integer, Parameter :: n1 = 201
Integer, Parameter :: n2 = 50
Integer, Parameter :: n3 = 2
Integer, Parameter :: n4 = 2
Integer, Parameter :: n5 = 21
Real( wp ), Dimension( 1:2, 1:n1, 1:n2, 1:n3, 1:n4, 1:n5 ) :: stuff
Integer :: start, finish, rate
Integer :: i1, i2, i3, i4, i5
Call Random_Number( stuff )
Call System_Clock( start, rate )
Do i1 = 1, n1
Do i2 = 1, n2
Do i3 = 1, n3
Do i4 = 1, n4
Do i5 = 1, n5
Write( 10, * ) stuff( 1, i1, i2, i3, i4, i5 )
End Do
End Do
End Do
End Do
End Do
Call System_Clock( finish, rate )
Write( *, * ) 'Loops : ', Real( finish - start ) / Real( rate )
Call System_Clock( start, rate )
Write( 11, * ) stuff( 1, :, :, :, :, : )
Call System_Clock( finish, rate )
Write( *, * ) 'Array : ', Real( finish - start ) / Real( rate )
Call System_Clock( start, rate )
Write( 12 ) stuff( 1, :, :, :, :, : )
Call System_Clock( finish, rate )
Write( *, * ) 'Unform: ', Real( finish - start ) / Real( rate )
End Program test_io
ian-admin@agon ~/test $ gfortran -O -Wall -Wextra -std=f2003 io.f90
ian-admin@agon ~/test $ ./a.out
Loops : 2.28500009
Array : 1.80200005
Unform: 5.79999983E-02
ian-admin@agon ~/test $
来源:https://stackoverflow.com/questions/43637321/efficiency-in-writing-to-disk-in-fortran