apache-storm

问题 How do you provide a custom configuration to a storm topology? For example, if I have a topology that I built that connects to a MySQL cluster and I want to be able to change which servers I need to to connect to without recompiling, how would I do that? My preference would be to use a config file, but my concern is that the file itself is not deployed to the cluster, therefore it won't be run (unless my understanding of how a cluster works is flawed). The only way I've seen so far to pass

问题 We have an use case where we do not want to run storm topology continuously. Instead, there are set of inputs( 10K+) that should be processed at the specified time, Spout continuously emits these inputs and get processed by rest of the bolts in the topology. Once all the inputs are processed, there is nothing to emit from nextTuple in my spout. At this time we wanted our topology to go to sleep and restart the process everyday night 12:00 am. Is there any property to set in the storm config

问题 My Nimbus host and supervisor run properly. When I submit a wordcount topology to the Nimbus host it successfully upload. After uploading topology supervisor giving error kill: No such process When I check the worker-6001.log and 6002.log I found a following error: 2014-02-09 17:20:05 b.s.m.TransportFactory [INFO] Storm peer transport plugin:backtype.storm.messaging.zmq 2014-02-09 17:20:05 b.s.d.worker [ERROR] Error on initialization of server mk-worker java.lang.UnsatisfiedLinkError: no jzmq

问题 I've have had no experience using Storm or Maven before, and I'm working on my starter project. When I compile the starter project uploaded on the git website using the command given there i.e. this: mvn compile exec:java -Dexec.classpathScope=compile -Dexec.mainClass=storm.starter.ExclamationTopology I can run the Exclamation topology class, but when I use this command: java -cp ./target/storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.ExclamationTopology I can't run it.

问题 I'm using storm with python. I used this command to run the topology locally mvn compile exec:java -Dexec.classpathScope=compile -Dexec.mainClass=my.Topology and got this error java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V at org.apache.log4j.Category.differentiatedLog(Category.java:186) at org.apache.log4j.Category.info(Category.java:229) I used this command mvn dependency

问题 I am new with storm, I was going through the word count example of storm. Here is the bolt which keeps track of the counts public static class WordCount extends BaseBasicBolt { Map<String, Integer> counts = new HashMap<String, Integer>(); @Override public void execute(Tuple tuple, BasicOutputCollector collector) { String word = tuple.getString(0); Integer count = counts.get(word); if (count == null) count = 0; count++; counts.put(word, count); collector.emit(new Values(word, count)); }

问题 I am always getting following error: [ERROR] connection attempt 86 to Netty-Client-/202.162.247.10:6731 failed: java.net.ConnectException: Connection refused: /202.162.247.10:6731 Why is this happening? I tried and googled multiple times but found no solution. 来源： https://stackoverflow.com/questions/34648103/storm-error-connection-attempt-86-to-netty-client

问题 I see the below error in worker logs, it happens almost every milliseconds, but the cluster is running fine, I wanted to know what does these error mean and any idea on why this would occur. This happens on all the worker nodes 2016-05-12T15:32:53.514-0500 b.s.m.n.Client [ERROR] connection attempt 3 to Netty-Client-xxxxx.hq.abc.com/xx.xx.xxx.xx:6700 failed: java.net.ConnectException: Connection refused: xxxxx.hq.abc.com/xx.xx.xxx.xxx:6700 And after some time i see this 2016-05-12T15:44:25.940

问题 It is unclear if you can do a fan-out (duplication) in Kafka like you can in Flume. I'd like to have Kafka save data to HDFS or S3 and send a duplicate of that data to Storm for real time processing. The output of Storm aggregations/analysis will be stored in Cassandra. I see some implementations flowing all data from Kafka into Storm and then two outputs from Storm. However, I'd like to eliminate the dependency of Storm for the raw data storage. Is this possible? Are you aware of any

问题 I have a problem.I understad trouble but I don't find solution.Cluster isn't shutdown,throw IOException.I/O methods not work because Windows doesn't allow temp folder.I mean this path; C:\Users\Mert\AppData\Local\Temp\8bbdc8e5-a753-40f3-afe0-0c2fcc1c25a0\version-2\log.1 Storm be able to delete log files for shutdown. I tried administrator mode for Eclipse.exe and javaw.exe but it doesn't work.What can I do ? IO Exception stack trace: exception.detailMessage -> java.io.IOException: Unable to