How to reduce the memory usage of Appstats on Google App Engine Java

问题

This is related to question Java Appengine APPSTATS causing java out of memory error.

Appstats seems to cause a java.lang.OutOfMemoryError on 128MB instances and I wonder whether there are ways to reduce the amount of logging. Is there a way to filter some package names from the stack trace?

GAE API:

 @14ms memcache.Get real=7ms api=0ms
Stack:
  com.google.appengine.tools.appstats.Recorder:290 makeAsyncCall()
  com.google.apphosting.api.ApiProxy:184 makeAsyncCall()
  com.google.apphosting.api.ApiProxy:123 makeAsyncCall()
  com.google.appengine.api.memcache.MemcacheServiceApiHelper:104 makeAsyncCall()
  com.google.appengine.api.memcache.AsyncMemcacheServiceImpl:372 doGetAll()
  com.google.appengine.api.memcache.AsyncMemcacheServiceImpl:333 getIdentifiables()
  com.google.appengine.api.memcache.MemcacheServiceImpl:61 getIdentifiables()

my appplication API:

  com.googlecode.objectify.cache.EntityMemcache:215 getAll()
  com.googlecode.objectify.cache.CachingAsyncDatastoreService:253 get()
  com.googlecode.objectify.cache.CachingDatastoreService:161 get()
  com.googlecode.objectify.cache.CachingDatastoreService:147 get()
  com.googlecode.objectify.cache.CachingDatastoreService:128 get()
  siena.gae.GaePersistenceManager:231 getByKey()
  siena.Model:106 getByKey()
  com.sirtrack.iridium.model.GroupEntity:147 getByKey()
  com.sirtrack.iridium.task.ProjectUpdateTask:49 doPost()

irrelevant:

  javax.servlet.http.HttpServlet:637 service()
  javax.servlet.http.HttpServlet:717 service()
  org.mortbay.jetty.servlet.ServletHolder:511 handle()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1166 doFilter()
  com.google.appengine.tools.appstats.AppstatsFilter:141 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.SiteFilter:106 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.RewriteFilter:79 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.AuthenticationFilter:83 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.PluginCronFilter:78 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.LanguageFilter:66 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.UpdateFilter:78 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.InitFilter:80 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.vosao.filter.ContextFilter:74 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  com.google.apphosting.utils.servlet.ParseBlobUploadFilter:102 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  com.google.apphosting.runtime.jetty.SaveSessionFilter:35 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  com.google.apphosting.utils.servlet.TransactionCleanupFilter:43 doFilter()
  org.mortbay.jetty.servlet.ServletHandler$CachedChain:1157 doFilter()
  org.mortbay.jetty.servlet.ServletHandler:388 handle()
  org.mortbay.jetty.security.SecurityHandler:216 handle()
  org.mortbay.jetty.servlet.SessionHandler:182 handle()
  org.mortbay.jetty.handler.ContextHandler:765 handle()
  org.mortbay.jetty.webapp.WebAppContext:418 handle()
  com.google.apphosting.runtime.jetty.AppVersionHandlerMap:249 handle()
  org.mortbay.jetty.handler.HandlerWrapper:152 handle()
  org.mortbay.jetty.Server:326 handle()
  org.mortbay.jetty.HttpConnection:542 handleRequest()
  org.mortbay.jetty.HttpConnection$RequestHandler:923 headerComplete()
  com.google.apphosting.runtime.jetty.RpcRequestParser:76 parseAvailable()
  org.mortbay.jetty.HttpConnection:404 handle()
  com.google.apphosting.runtime.jetty.JettyServletEngineAdapter:135 serviceRequest()
  com.google.apphosting.runtime.JavaRuntime$RequestRunnable:446 run()
  com.google.tracing.TraceContext$TraceContextRunnable:449 runInContext()
  com.google.tracing.TraceContext$TraceContextRunnable$1:455 run()
  com.google.tracing.TraceContext:695 runInContext()
  com.google.tracing.TraceContext$AbstractTraceContextCallback:333 runInInheritedContextNoUnref()
  com.google.tracing.TraceContext$AbstractTraceContextCallback:325 runInInheritedContext()
  com.google.tracing.TraceContext$TraceContextRunnable:453 run()
  com.google.apphosting.runtime.ThreadGroupPool$PoolEntry:251 run()
  java.lang.Thread:679 run()

Here's a stack trace of the maybe-caused-by-stacktrace OutOfMemoryError:

Error for /_ah/queue/projectupdate
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3057)
    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:407)
    at java.lang.StringBuffer.append(StringBuffer.java:241)
    at java.io.StringWriter.write(StringWriter.java:112)
    at java.io.PrintWriter.write(PrintWriter.java:429)
    at java.io.PrintWriter.write(PrintWriter.java:446)
    at java.io.PrintWriter.print(PrintWriter.java:576)
    at java.io.PrintWriter.println(PrintWriter.java:712)
    at java.lang.Throwable.printStackTrace(Throwable.java:529)
    at com.google.appengine.tools.appstats.Recorder.createStackTrace(Recorder.java:160)
    at com.google.appengine.tools.appstats.Recorder.initializeIntermediary(Recorder.java:271)
    at com.google.appengine.tools.appstats.Recorder.makeAsyncCall(Recorder.java:290)
    at com.googlecode.objectify.cache.TriggerFutureHook.makeAsyncCall(TriggerFutureHook.java:144)
    at com.google.apphosting.api.ApiProxy.makeAsyncCall(ApiProxy.java:184)

For Google Engineers, here's a link to those errors

回答1:

I've found an undocumented (AFAIK) parameter called "maxLinesOfStackTrace" in the class com.google.appengine.tools.appstats.Recorder.java.

In web.xml:

<filter>
  <filter-name>appstats</filter-name>
    <filter-class>com.google.appengine.tools.appstats.AppstatsFilter</filter-class>
    <init-param>
      <param-name>maxLinesOfStackTrace</param-name>
      <param-value>16</param-value>
    </init-param>
</filter>

It seems to work locally, I'll let you know whether it also helps avoiding the OutOfMemoryError.

来源：https://stackoverflow.com/questions/10592598/how-to-reduce-the-memory-usage-of-appstats-on-google-app-engine-java