Parallel processing of big rasters in R (windows)

安稳与你 提交于 2019-12-05 03:32:42

I had this exact problem while running the rasterize fucntion inside a cluster in R.

All tests worked perfectly but when I upscaled to very large and fine resolution rasters, I repeatedly got errors regarding temp files that I couldn't even find on my computer. The list object, which I needed to merge and write as 1 raster, was in R but I could do nothing with it.

After watching the temp file directory whilst the cluster was running I noticed that closing the cluster will auto-delete all temp files created, so I had to perform the merge and writeRaster functions inside the cluster, otherwise it would fail on a very similar error to yours.

You could pass specific filenames to calc (or, e.g., reclassify), and have your function return those filenames as a vector to be read into a stack:

ff <- parSapply(cl, list.x, function(x) { 
  calc(x, function(x) x*10, filename=f <- tempfile(fileext='.tif'))
  f
})

s <- stack(ff)

But also look at ?clusterR- I suspect it will work with reclassify. From the docs:

This function only works with functions that have a Raster* object as first argument and that operate on a cell by cell basis (i.e., there is no effect of neigboring cells) and return an object with the same number of cells as the input raster object. The first argument of the function called must be a Raster* object. There can only be one Raster* object argument. For example, it works with calc and it also works with overlay as long as you provide a single RasterStack or RasterBrick as the first argument.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!