apache-storm

问题 numpy.array.tostring doesn't seem to preserve information about matrix dimensions (see this question), requiring the user to issue a call to numpy.array.reshape . Is there a way to serialize a numpy array to JSON format while preserving this information? Note: The arrays may contain ints, floats or bools. It's reasonable to expect a transposed array. Note 2: this is being done with the intent of passing the numpy array through a Storm topology using streamparse, in case such information ends

问题 Our Apache Storm topology listens messages from Kafka using KafkaSpout and after doing lot of mapping/reducing/enrichment/aggregation etc. etc finally inserts data into Cassandra. There is another kafka input where we receive user queries for data if topology finds a response then it sends that onto a third kafka topic. Now we want to write E2E test using Junit in which we can directly programmatically insert data into topology and then by inserting user query message, we can assert on third

问题 Here what i got when compile the project by this command mvn compile exec:java -Dexec.classPathScope=compile -Dexec.mainClass=trident.myproject got this java.lang.NoClassDefFoundError: storm/trident/state/StateFactory at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2427) at java.lang.Class.getMethod0(Class.java:2670) at java.lang.Class.getMethod(Class.java:1603) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:285) at

问题 I have recently migrated storm version from 0.8.2 to 0.9.4. In 0.8.2 I had implemented tick tuple functionality in a bolt in my code. When I used to run the topology it used to trigger periodic activity after the time specified in tickfrequency seconds (configured in getComponentConfiguration method). But the same piece of code is not running in apache storm version 0.9.4. The code inside if block in execute method is never getting executed because isTickTuple method is always returning false

问题 I have recently migrated storm version from 0.8.2 to 0.9.4. In 0.8.2 I had implemented tick tuple functionality in a bolt in my code. When I used to run the topology it used to trigger periodic activity after the time specified in tickfrequency seconds (configured in getComponentConfiguration method). But the same piece of code is not running in apache storm version 0.9.4. The code inside if block in execute method is never getting executed because isTickTuple method is always returning false

问题 I want to install Storm in my own machine(Ubuntu 12.04 LTS) and I follow the instructions in https://github.com/nathanmarz/storm/wiki/Installing-native-dependencies. When I installing jzmq, I firstly successfully perform the command sudo ./autogen.sh however, when I perform sudo ./configure The configuration error: 'configure: error: the JAVA_HOME environment variable must be set to your JDK location.' appears. I have set the JAVA_HOME using sudo gedit /etc/profile export JAVA_HOME=/usr/lib

问题 I am running storm jar in a cluster ,where I configured hadoop,kafka,storm cluster when I run the jar in local mode it works fine ,when I run it on storm cluster, I am finding respective error in Storm UI: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosTicket(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.<init>(UserGroupInformation.java:666) at org.apache.hadoop.security.UserGroupInformation

问题 I am able to run storm Kafka with local cluster but not able to run with storm Submitter below is my topology code can anyone please help me to solve this issue :) package com.org.kafka; import org.apache.storm.Config; import org.apache.storm.LocalCluster; import org.apache.storm.generated.AlreadyAliveException; import org.apache.storm.generated.AuthorizationException; import org.apache.storm.generated.InvalidTopologyException; import org.apache.storm.kafka.KafkaSpout; import org.apache.storm

问题 I have a bug that occurs in my Storm topology during initialization. I would like to set the log level to DEBUG from when the topology is started. I realize there is a mechanism to dynamically set the log level for a running topology using either the Storm UI or CLI, but I am not able to dynamically change this setting before the bug occurs in my topology during initialization. How can I statically set the log level to DEBUG so that I can see more detailed logs when my topology is initialized

问题 I build a topo to get messages from by kafka, and then grep some keyword, if fit, write to local file. I use OpaqueTridentKafkaSpout of storm-kafka to ensure the tuple will not miss or repeat, but consider one situation: when writing message to local file, and some error occur (for example, not enough space). At this moment, some messages have written to local file, and others not, if the spout resend the message, the message will be write twice. How to handle that? 回答1: It's simple. The code