CARET. Relationship between data splitting and trainControl

Question

I have carefully read the CARET documentation at: http://caret.r-forge.r-project.org/training.html, the vignettes, and everything is quite clear (the examples on the website hel

Answer 1

If you are not sure what role method plays if you use index, why not to apply all the methods and compare results. It is a blind method of comparaison, but it can give you some intuitions.

  methods <- c('boot', 'boot632', 'cv', 
               'repeatedcv', 'LOOCV', 'LGOCV')

I create my index:

  n <- 100
  tmp <- createDataPartition(logBBB,p = .8, times = n)

I apply trainControl for my list of method, and I remove index from result since it is common to all my methods.

ll <- lapply(methods,function(x)
         trControl = trainControl(method = x, index = tmp))
ll <- sapply(ll,'[<-','index', NULL)

Hence my ll is :

                 [,1]      [,2]      [,3]      [,4]         [,5]      [,6]     
method            "boot"    "boot632" "cv"      "repeatedcv" "LOOCV"   "LGOCV"  
number            25        25        10        10           25        25       
repeats           25        25        1         1            25        25       
verboseIter       FALSE     FALSE     FALSE     FALSE        FALSE     FALSE    
returnData        TRUE      TRUE      TRUE      TRUE         TRUE      TRUE     
returnResamp      "final"   "final"   "final"   "final"      "final"   "final"  
savePredictions   FALSE     FALSE     FALSE     FALSE        FALSE     FALSE    
p                 0.75      0.75      0.75      0.75         0.75      0.75     
classProbs        FALSE     FALSE     FALSE     FALSE        FALSE     FALSE    
summaryFunction   ?         ?         ?         ?            ?         ?        
selectionFunction "best"    "best"    "best"    "best"       "best"    "best"   
preProcOptions    List,3    List,3    List,3    List,3       List,3    List,3   
custom            NULL      NULL      NULL      NULL         NULL      NULL     
timingSamps       0         0         0         0            0         0        
predictionBounds  Logical,2 Logical,2 Logical,2 Logical,2    Logical,2 Logical,2