Speed up read.dbf in R (problems with importing large dbf file)

问题

I have a dataset given in .dbf format and need to import it into R. I haven't worked with such extension previously, so have no idea how to export dbf file with multiple tables into different format.

Simple read.dbf has been running hours and still no results. Tried to look for speeding up R performance, but not sure whether it's the case, think the problem is behind reading the large dbf file itself (weights ~ 1.5Gb), i.e. the command itself must be not efficient at all. However, I don't know any other option how to deal with such dataset format. Is there any other option to import the dbf file?

P.S. (NOT R ISSUE) The source of the dbf file uses visual foxpro, but can't export it to other format. I've installed foxpro, but given that I've never used it before, I don't know how to export it in the right way. Tried simple "Export to type=XLS" command, but here comes a problem with encoding as most of variables are in Russian Cyrillic and can't be decrypted by excel. In addition, the dbf file contains multiple tables that should be merged in 1 big table, but I don't know how to export those tables separately to xls, same as I don't know how to export multiple tables as a whole into xls or csv, or how to merge them together as I'm absolutely new to dbf files theme (though looked through base descriptions already)

Any helps will be highly appreciated. Not sure whether I can provide with sample dataset, as there are many columns when I look the dbf in foxpro, plus those columns must be merged with other tables from the same dbf file, and have no idea how to do that. (sorry for the mess)

回答1:

Your can export from Visual FoxPro in many formats using the COPY TO command via the Command Window, as per the VFP help file.

For example:

use mydbf in 0
select mydbf
copy to myfile.xls type xl5
copy to myfile.csv type delimited

If you're having language-related issues, you can add an 'as codepage' clause to the end of those. For example:

copy to myfile.csv type delimited as codepage 1251

If you are not familiar with VFP I would try to get the raw data out like that, and into a platform that you are familiar with, before attempting merges etc.

To export them in a loop you could use the following in a .PRG file (amending the two path variables at the top to reflect your own setup).

    Close All
Clear All
Clear 

lcDBFDir = "c:\temp\"           && -- Where the DBF files are.
lcOutDir = "c:\temp\export\"        && -- Where you want your exported files to go.

lcDBFDir = Addbs(lcDBFDir)      && -- In case you forgot the backslash.
lcOutDir = Addbs(lcOutDir)  

* -- Get the filenames into an array.
lnFiles = ADir(laFiles, Addbs(lcDBFDir) + "*.DBF")

* -- Process them. 
For x = 1 to lnFiles

    lcThisDBF = lcDBFDir + laFiles[x, 1]
    Use (lcThisDBF) In 0 Alias currentfile
    Select currentfile
    Copy To (lcOutDir + Juststem(lcThisDBF) + ".csv") type csv
    Use in Select("Currentfile")        && -- Close it.

EndFor

Close All

... and run it from the Command Window - Do myprg.prg or whatever.

来源：https://stackoverflow.com/questions/17443524/speed-up-read-dbf-in-r-problems-with-importing-large-dbf-file

标签

foxpro

dbf