apache-storm

问题 After i wrote mvn -f pom.xml compile exec:java -Dexec.classpathScope=Compile-Dexec.main Class=storm.starter.WordCountTopology and found this !! [INFO] One or more required plugin parameters are invalid/missing for 'exec:java' [0] Inside the definition for plugin 'exec-maven-plugin' specify the following: ... VALUE -OR- on the command line, specify: '-Dstorm.topology=VALUE 回答1: If you link your pom.xml then this would be easier. I'm guessing you're using Storm. Have you written your own

问题 After i wrote mvn -f pom.xml compile exec:java -Dexec.classpathScope=Compile-Dexec.main Class=storm.starter.WordCountTopology and found this !! [INFO] One or more required plugin parameters are invalid/missing for 'exec:java' [0] Inside the definition for plugin 'exec-maven-plugin' specify the following: ... VALUE -OR- on the command line, specify: '-Dstorm.topology=VALUE 回答1: If you link your pom.xml then this would be easier. I'm guessing you're using Storm. Have you written your own

问题 While running the command: storm jar target/crawlIndexer-1.0-SNAPSHOT.jar org.apache.storm.flux.Flux --local es-injector.flux --sleep 86400000 I get an error saying: 8710 [Thread-26-status-executor[4 4]] ERROR c.d.s.e.p.StatusUpdaterBolt - Can't connect to ElasticSearch When running http://localhost:9200/ in browser ES successfully loads up. Kibana also connects to ES. So it must just be the connection from StromCrawler to ElasticSearch. What could be the issue? Snippet of full error: 8710

问题 I tried to deploy apache storm for a streaming program development. 4 servers for 1 nimbus/ui and 3 supervisors. After all setup, nimbus/ui/supervisors are launched without errors/exceptions reported in logs. But when access to ui via browser, the page is stuck in a spinning prompted "loading summary" like for ever. All processes are killed and zookeeper node "/storm" and storm data folder are cleared. Still stuck on ui after restart the cluster. Also switched to lower rev 1.1.3, still same

问题 I'm trying to setup Apache Storm (1.0.2) on my Macbook Pro but apparently running into timeout issues if I try to deploy the topology. Also the UI hangs up spitting the same exception. 3491 [main] INFO o.a.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -8915636774701640550:-6510752657961785886 3580 [main] INFO o.a.s.s.a.AuthUtils - Got AutoCreds [] Exception in thread "main" java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException: java.net

问题 Structure to the problem : 4 node cluster Bolt 2 - parallelism factor is 8, so 2 executers per worker. I have notice lag overtime in the storm topology due to high latency in one of the executers. If you'll notice in the pic, only 1 executer is having slightly more load and significantly high latency. Spout is working great! Any pointer to the possible issues appreciated. Noting unusual observer in worker/supervisor logs. The worker JVM has high has huge memory so that is not a issue. PS: cpu

问题 To connect to a mongo database,in our non-storm applications we would create a singleton instance of the Mongo class and share it across the application Would like to know what is the right way to implement a Mongo connection pooling in the storm framework? Options we have tried : We have a spout which reads from the Mongo database. we just put the singleton Mongo class inside the spout and initialise it in the open method? - But this approach made it impossible to share the Mongo instance to

问题 Internal Server Error org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift7.transport.TSocket.open(TSocket.java:183) at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81) at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75) at backtype.storm.ui.core$all_topologies_summary.invoke(core.clj:515) at backtype.storm.ui.core$fn__8018.invoke(core.clj:851) at compojure.core$make_route$fn__6199

问题 We have been struggling with this issue for a long time now. In short, our storm topology stops emitting messages from spout after some time in a random fashion. We have an automated script which re-deploys the topology at 06:00 UTC everyday after the master data refresh activity is complete. In the last 2 weeks, our topology stopped emitting the messages for 3 times in late UTC hours (between 22:00 and 02:00). It only comes online when we restart it which is around 06:00 UTC. I've searched

问题 I am trying to setup a multi-cluster storm system. I have found several 3rd party step by step guides on this. They all have Java, Python, ZeroMQ 2.1.7 and JZMQ as the requirements for the Nimbus and Supervisor/Slave nodes. But on the official Apache Storm website, the only requirements for the Nimbus and Supervisor nodes is Java 6 and Python 2.6.6 (https://storm.apache.org/documentation/Setting-up-a-Storm-cluster.html) Does anyone know if ZeroMQ and JZMQ are required for Storm cluster