How to run WordCountTopology from storm-starter in Intellij

后端 未结 3 693
忘掉有多难
忘掉有多难 2021-01-18 14:24

I work with Storm for a while already, but want to get started with development. As suggested, I am using IntelliJ (up to now, I was using Eclipse and did only write topolog

3条回答
  •  滥情空心
    2021-01-18 14:39

    I managed to run it on my virtualbox, storm version 1.2.2:

    just download https://github.com/apache/storm/blob/master/storm-multilang/python/src/main/resources/resources/storm.py and put it into any folder you want, for example: /apache-storm-1.2.2/examples/storm-starter/multilang/resources/ , and then change the main function:

    public static void main(String[] args) throws Exception {
    
        SplitSentence pythonSplit = new SplitSentence();
        Map env = new HashMap();
        env.put("PYTHONPATH", "/apache-storm-1.2.2/examples/storm-starter/multilang/resources/");
        pythonSplit.setEnv(env);
    
        TopologyBuilder builder = new TopologyBuilder();
    
        builder.setSpout("spout", new RandomSentenceSpout(), 5);
    
        builder.setBolt("split",pythonSplit, 8).shuffleGrouping("spout");
        builder.setBolt("count", new WordCount(), 12).fieldsGrouping("split", new Fields("word"));
    
        Config conf = new Config();
        conf.setDebug(true);
    
        if (args != null && args.length > 0) {
          conf.setNumWorkers(3);
    
          StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
        }
        else {
          conf.setMaxTaskParallelism(3);
    
          LocalCluster cluster = new LocalCluster();
          cluster.submitTopology("word-count", conf, builder.createTopology());
    
          Thread.sleep(600000);
    
          cluster.shutdown();
        }
      }
    

    the full instructions can be found on my blog which includes other issues encountered when running it in Local Mode and Local Cluster mode: https://lyhistory.com/storm/

提交回复
热议问题