R memory leak using C external pointers

问题

I'm trying to use external pointers in a package, but I ran into an issue where it seemed like the finalizer was not being called and memory leaked.

Below is an extremely contrived example of the issue:

#include <Rcpp.h>
using namespace Rcpp;

void finalize(SEXP xp){
  delete static_cast< std::vector<double> *>(R_ExternalPtrAddr(xp));
}

// [[Rcpp::export]]
SEXP ext_ref_ex() {
  std::vector<double> * x  = new std::vector<double>(1000000);
  SEXP xp = PROTECT(R_MakeExternalPtr(x, R_NilValue, R_NilValue));
  R_RegisterCFinalizer(xp, finalize);
  UNPROTECT(1);
  return xp;
}

R side:

library(Rcpp)
sourceCpp("tests.cpp")

# breaks and/or crashes
for(i in 1:10000) {
  z <- ext_ref_ex()
}

# no issue
for(i in 1:10000) {
  z <- ext_ref_ex()
  rm(z)
  gc()
}

Running the first loop, R eventually segfaults after enough iterations (issue #1), whereas the expected behavior is that the data should be cleaned up and there should be no segfault.

Issue #2 is that if you interrupt the process and call gc(), sometimes the memory will be cleared but usually it won't. Based on the htop report, R uses 60-70% of the memory, even after rm(list=ls()) and gc().

The second loop experiences no apparent memory issues.

Am I doing something wrong on the C side? Am I running into a bug?

(R version 3.5.2 ubuntu 18.04 on Windows.)

回答1:

I can reproduce the issue even when using Rcpp instead of the C API for creating the external pointer and registering the finalizer:

#include <Rcpp.h>

// [[Rcpp::export]]
Rcpp::XPtr< std::vector<double> > ext_ref_ex() {
  std::vector<double> * x  = new std::vector<double>(1000000);
  Rcpp::XPtr< std::vector<double> > xp(x, true) ;
  return xp;
}

For me, just including gc() in the loop is enough to fix the issue:

for (i in 1:10000) {
  z <- ext_ref_ex()
  gc() # crash without this line
}

So it seems not an issue with "finalizer not running" but with "garbage collection not running". My interpretation: You are allocating a lot of memory for the vector and a little memory for the external pointer. R knows only about the external pointer. So if that goes out of scope, R does not see a reason to run the garbage collection.

来源：https://stackoverflow.com/questions/55235959/r-memory-leak-using-c-external-pointers

标签

rcpp