1. 技术文章
  2. SchedulerStateManagerAdaptor failed to fetch data from zookeeper path in Heron Cluster

SchedulerStateManagerAdaptor failed to fetch data from zookeeper path in Heron Cluster

由 泪湿孤枕 提交于 2019-12-13 03:35:57

问题


In Heron, I instantiated a SchedulerStateManagerAdaptor named adaptor using IStateManager and created a new Runtime config to saved this adaptor. Then, I created a ISchedulerClient object for invoking its schedulerClient.updateTopology function. Code as follows:

String stateMgrClass = Context.stateManagerClass(this.config); // get state manager instance
IStateManager stateMgr = null;
try {
     stateMgr = ReflectionUtils.newInstance(stateMgrClass);
     FileUtils.writeToFile(filename, "Create IStateManager object success...");
} catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) {
     e.printStackTrace();
}
stateMgr.initialize(this.config);
stateManagerAdaptor = new SchedulerStateManagerAdaptor(stateMgr, 5000);

Config runtime = Config.newBuilder()
                .putAll(primaryRuntime)
                .put(Key.TOPOLOGY_NAME, Context.topologyName(config))
                .put(Key.SCHEDULER_STATE_MANAGER_ADAPTOR, stateManagerAdaptor)
                .build();

if (!schedulerClient.updateTopology(updateTopologyRequest)) {
     throw new TopologyRuntimeManagementException(String.format(
             "Failed to update " + topology.getName() +" with Scheduler, updateTopologyRequest=" + updateTopologyRequest));
}

However, when I tried to adaptor.getPhysicalPlan in scheduler.onUpdate function that was invoked by instantiated scheduler. This Exception happened: What's more, I find the Exception in zookeeper log:

2018-07-07 21:01:09,855 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41348 which had sessionid 0x16469a49d550f23
2018-07-07 21:01:12,001 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f29, timeout of 4000ms exceeded
2018-07-07 21:01:12,002 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f28, timeout of 4000ms exceeded
2018-07-07 21:01:12,002 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f29
2018-07-07 21:01:12,003 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f28
2018-07-07 21:01:20,001 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f2a, timeout of 10000ms exceeded
2018-07-07 21:01:20,003 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f24, timeout of 10000ms exceeded
2018-07-07 21:01:20,003 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f23, timeout of 10000ms exceeded
2018-07-07 21:01:20,003 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16474d4e4bb0003, timeout of 10000ms exceeded
2018-07-07 21:01:20,004 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16474d4e4bb0004, timeout of 10000ms exceeded
2018-07-07 21:01:20,005 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f2a
2018-07-07 21:01:20,005 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f24
2018-07-07 21:01:20,006 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f23
2018-07-07 21:01:20,006 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0003
2018-07-07 21:01:20,006 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0004
2018-07-07 21:01:23,779 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41378
2018-07-07 21:01:23,779 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41378; will be dropped if server is in r-o mode
2018-07-07 21:01:23,779 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41378
2018-07-07 21:01:23,780 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41380
2018-07-07 21:01:23,780 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41380; will be dropped if server is in r-o mode
2018-07-07 21:01:23,780 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41380
2018-07-07 21:01:23,783 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41382
2018-07-07 21:01:23,783 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41382; will be dropped if server is in r-o mode
2018-07-07 21:01:23,783 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41382
2018-07-07 21:01:23,784 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0005 with negotiated timeout 10000 for client /218.195.228.24:41378
2018-07-07 21:01:23,785 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41384
2018-07-07 21:01:23,785 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.24:41384; will be dropped if server is in r-o mode
2018-07-07 21:01:23,785 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41384
2018-07-07 21:01:23,786 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0006 with negotiated timeout 10000 for client /218.195.228.24:41380
2018-07-07 21:01:23,787 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0007 with negotiated timeout 10000 for client /218.195.228.24:41382
2018-07-07 21:01:23,789 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0008 with negotiated timeout 10000 for client /218.195.228.24:41384
2018-07-07 21:01:38,005 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f70, timeout of 30000ms exceeded
2018-07-07 21:01:38,005 [myid:] - INFO  [SessionTracker:ZooKeeperServer@358] - Expiring session 0x16469a49d550f71, timeout of 30000ms exceeded
2018-07-07 21:01:38,005 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f70
2018-07-07 21:01:38,005 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16469a49d550f71
2018-07-07 21:02:19,897 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41428
2018-07-07 21:02:19,899 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41428
2018-07-07 21:02:19,900 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0009 with negotiated timeout 30000 for client /218.195.228.24:41428
2018-07-07 21:02:26,397 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41436
2018-07-07 21:02:26,398 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41436
2018-07-07 21:02:26,399 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000a with negotiated timeout 10000 for client /218.195.228.24:41436
2018-07-07 21:02:26,403 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb000a
2018-07-07 21:02:26,404 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41436 which had sessionid 0x16474d4e4bb000a
2018-07-07 21:02:29,672 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40088
2018-07-07 21:02:29,675 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40088
2018-07-07 21:02:29,679 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000b with negotiated timeout 10000 for client /218.195.228.43:40088
2018-07-07 21:02:30,480 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.28:57174
2018-07-07 21:02:30,493 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.28:57174
2018-07-07 21:02:30,498 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000c with negotiated timeout 10000 for client /218.195.228.28:57174
2018-07-07 21:02:31,233 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52872
2018-07-07 21:02:31,235 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.19:52872
2018-07-07 21:02:31,239 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000d with negotiated timeout 10000 for client /218.195.228.19:52872
2018-07-07 21:02:34,002 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0009
2018-07-07 21:02:34,009 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41428 which had sessionid 0x16474d4e4bb0009
2018-07-07 21:02:57,272 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40100
2018-07-07 21:02:57,274 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
    at java.lang.Thread.run(Thread.java:748)
2018-07-07 21:02:57,275 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.43:40100 (no session established for client)
2018-07-07 21:02:57,276 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40102
2018-07-07 21:02:57,277 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40102
2018-07-07 21:02:57,280 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000e with negotiated timeout 10000 for client /218.195.228.43:40102
2018-07-07 21:02:57,357 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40106
2018-07-07 21:02:57,358 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.43:40106; will be dropped if server is in r-o mode
2018-07-07 21:02:57,358 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40106
2018-07-07 21:02:57,362 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb000f with negotiated timeout 30000 for client /218.195.228.43:40106
2018-07-07 21:02:58,936 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.43:40110
2018-07-07 21:02:58,938 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.43:40110
2018-07-07 21:02:58,943 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0010 with negotiated timeout 30000 for client /218.195.228.43:40110
2018-07-07 21:03:05,562 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52884
2018-07-07 21:03:05,564 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
    at java.lang.Thread.run(Thread.java:748)
2018-07-07 21:03:05,565 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.19:52884 (no session established for client)
2018-07-07 21:03:05,568 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52886
2018-07-07 21:03:05,569 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.19:52886
2018-07-07 21:03:05,575 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0011 with negotiated timeout 10000 for client /218.195.228.19:52886
2018-07-07 21:03:05,794 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.19:52888
2018-07-07 21:03:05,795 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /218.195.228.19:52888; will be dropped if server is in r-o mode
2018-07-07 21:03:05,796 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.19:52888
2018-07-07 21:03:05,798 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0012 with negotiated timeout 30000 for client /218.195.228.19:52888
2018-07-07 21:03:43,958 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.24:41544
2018-07-07 21:03:43,960 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /218.195.228.24:41544
2018-07-07 21:03:43,984 [myid:] - INFO  [SyncThread:0:ZooKeeperServer@687] - Established session 0x16474d4e4bb0013 with negotiated timeout 30000 for client /218.195.228.24:41544
2018-07-07 21:03:44,041 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x16474d4e4bb0013
2018-07-07 21:03:44,043 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /218.195.228.24:41544 which had sessionid 0x16474d4e4bb0013
2018-07-07 21:03:53,413 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /218.195.228.28:57186
2018-07-07 21:03:53,415 [myid:] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
    at java.lang.Thread.run(Thread.java:748)

I don't know what's wrong with it! But when I invoke the adaptor.getPhysicalPlan in the file that created adaptor object, it worked. What reason caused this problem? Thanks for your help.

来源：https://stackoverflow.com/questions/51188138/schedulerstatemanageradaptor-failed-to-fetch-data-from-zookeeper-path-in-heron-c

标签
apache-zookeeper
scheduler
heron
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!