Error with h2o.predict in R

Question

I am getting an error when trying to create deep learning predictions with h2o in R. The error occurs for about one third of predictions with the command h2o.predict. Here is the model setup:

localH2O = h2o.init(ip = "localhost", port = 54321, startH2O = TRUE,max_mem_size='20g',nthreads=6)
model <- h2o.deeplearning(x = 2:100, y = 1, training_frame = x, l1 = 1e-5, l2 = 1e-5, epochs=500, hidden = c(800,800,100))
prediction <- h2o.predict(model, x[,2:100])

Here is the error that occurs on and off:

ERROR: Unexpected HTTP Status code: 500 Server Error (url = http://localhost:54321/99/Rapids)

java.lang.RuntimeException
[1] "water.MRTask.getResult(MRTask.java:505)"                                              
[2] "water.MRTask.doAll(MRTask.java:379)"                                                  
[3] "water.MRTask.doAll(MRTask.java:375)"                                                  
[4] "water.rapids.ASTRowSlice.apply(ASTColSlice.java:123)"                                 
[5] "water.rapids.ASTExec.exec(ASTExec.java:46)"                                           
[6] "water.rapids.ASTTmpAssign.apply(ASTAssign.java:255)"                                  
[7] "water.rapids.ASTTmpAssign.apply(ASTAssign.java:248)"                                  
[8] "water.rapids.ASTExec.exec(ASTExec.java:46)"                                           
[9] "water.rapids.Session.exec(Session.java:56)"                                           
[10] "water.rapids.Exec.exec(Exec.java:63)"                                                 
[11] "water.api.RapidsHandler.exec(RapidsHandler.java:23)"                                  
[12] "sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)"                          
[13] "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)"
[14] "java.lang.reflect.Method.invoke(Method.java:497)"                                     
[15] "water.api.Handler.handle(Handler.java:64)"                                            
[16] "water.api.RequestServer.handle(RequestServer.java:644)"                               
[17] "water.api.RequestServer.serve(RequestServer.java:585)"                                
[18] "water.JettyHTTPD$H2oDefaultServlet.doGeneric(JettyHTTPD.java:617)"                    
[19] "water.JettyHTTPD$H2oDefaultServlet.doPost(JettyHTTPD.java:565)"                       
[20] "javax.servlet.http.HttpServlet.service(HttpServlet.java:755)"                         
[21] "javax.servlet.http.HttpServlet.service(HttpServlet.java:848)"                         
[22] "org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)"               

Show Traceback

Rerun with Debug
Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = page,  : 
water.DException$DistributedException: from /127.0.0.1:54321; by class water.rapids.ASTRowSlice$1; class java.lang.NegativeArraySizeException: null Error in class(obj) <- "rs.scalar" : attempt to set an attribute on NULL

Here is something about my system architecture. Running system("java -version") gives:

java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)

Here is the output of sessionInfo():

R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server 2008 R2 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=Danish_Denmark.1252  LC_CTYPE=Danish_Denmark.1252    LC_MONETARY=Danish_Denmark.1252
[4] LC_NUMERIC=C                    LC_TIME=Danish_Denmark.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] h2o_3.6.0.8              statmod_1.4.22           TTR_0.23-0               selectiveInference_1.1.1
[5] intervals_0.15.1         lars_1.2                 glmnet_2.0-2             foreach_1.4.3           
[9] Matrix_1.2-2             wq_0.4.4                 zoo_1.7-12               skm_1.0.2               
[13] stringi_1.0-1            devtools_1.9.1          

loaded via a namespace (and not attached):
[1] Rcpp_0.12.2      magrittr_1.5     MASS_7.3-43      munsell_0.4.2    colorspace_1.2-6
[6] lattice_0.20-33  stringr_1.0.0    plyr_1.8.3       xts_0.9-7        tools_3.2.2     
[11] grid_3.2.2       gtable_0.1.2     iterators_1.0.8  digest_0.6.8     reshape2_1.4.1  
[16] ggplot2_1.0.1    bitops_1.0-6     codetools_0.2-14 RCurl_1.95-4.7   memoise_0.2.1   
[21] scales_0.3.0     jsonlite_0.9.19  proto_0.3-10    

Any help much appreciated.

Answer 1

The error you've hit upon is originating from the internal H2O language processing framework (called Rapids).

The likely culprit is not predict, but rather the snippet

   x[,2:100]

which should be doing a column slice (ASTColSlice). But it looks like it's executing code in ASTRowSlice... Could get a better handle on what's going on if you're able to provide any logs (the stdout/err help here, you can get them with the method

        h2o.downloadAllLogs

One thing that stands out is:

  Error in class(obj) <- "rs.scalar" : attempt to set an attribute on NULL

"rs.scalar" doesn't have any significance in our R code, is it something that you recognize? At any rate, the logs should shed more light on how the NegativeArraySizeException is occurring.

Thanks!