reducers

问题 I'm pulling via REST a JSON with an array of objects with some fields and some nested objects. What I'm trying to create is a grouped summary object from the array of nested JSON objects with the following structure: var data = [ { "Id": 79, "Date": "2019-02-17T00:00:00-07:00", "StartTime": 1535385600, "EndTime": 1535416200, "Slots": [ { "blnEmptySlot": false, "strType": "B", "intStart": 3600, "intEnd": 5400, "intUnixStart": 1535389200, "intUnixEnd": 1535391000, } ], "OperationalUnit": 3,

问题 I'm having problems trying to get my reducer to work correctly in Redux. I'm new to Redux so I might be missing something simple, but I've played with it for a while and can't figure out what's going wrong. Here is my process: Define argument: First I define the index value that I need. When logged, this returns the correct number... const thisCommentIndex = parseInt(comments.indexOf(comment)) Function Call: <div onClick={this.props.removeComment.bind(null, thisCommentIndex)}></div> Action:

问题 I am getting JAVA Heap space error in my reducer phase .I have used 41 reducer in my application and also Custom Partitioner class . Below is my reducer code that throws below error . 17/02/12 05:26:45 INFO mapreduce.Job: map 98% reduce 0% 17/02/12 05:28:02 INFO mapreduce.Job: map 100% reduce 0% 17/02/12 05:28:09 INFO mapreduce.Job: map 100% reduce 17% 17/02/12 05:28:10 INFO mapreduce.Job: map 100% reduce 39% 17/02/12 05:28:11 INFO mapreduce.Job: map 100% reduce 46% 17/02/12 05:28:12 INFO

问题 I am implementing a left join functionality using map reduce. Left side is having around 600 million records and right side is having around 23 million records. In mapper I am making the keys using the columns used in left join condition and passing the key-value output from mapper to reducer. I am getting performance issue because of few mapper keys for which number of values in both the tables are high (eg. 456789 and 78960 respectively). Even though other reducers finish their job, these

问题 I've created some factory functions that give me simple (or more advanced) reducers. For example (simple one - base on action type set RequestState constant as a value): export const reduceRequestState = (requestTypes: RequestActionTypes) => (state: RequestState = RequestState.None, action: Action): RequestState => { switch (action.type) { case requestTypes.start: return RequestState.Waiting; case requestTypes.success: return RequestState.Success; case requestTypes.error: return RequestState

问题 I am little confused about the terms "Job scheduling" and "Task scheduling" in Hadoop when I was reading about delayed fair scheduling in this slide. Please correct me if I am wrong in my following assumptions: Default scheduler, Capacity scheduler and Fair schedulers are only valid at job level when multiple jobs are scheduled by the user. They don't play any role if there is only single job in the system. These scheduling algorithms form basis for "job scheduling" Each job can have multiple

问题 I'm wondering why the code below provides no speedup in the case of r/fold? Am I misunderstanding something about reducers? I'm running it on a pretty slow (although with 2 cores) Ubuntu 12.04 dev box, both through emacs and lein run, each with the same results. (require '[clojure.core.reducers :as r]) (.. Runtime getRuntime availableProcessors) ;; 2 (let [n 80000000 vs #(range n)] (time (reduce + (vs))) (time (r/fold + (vs))) "Elapsed time: 26076.434324 msecs" "Elapsed time: 25500.234034

问题 I'm trying to work out the ideal way to update several top level fields on my state tree while still maintaining split reducers. Here's a simple solution that I've come up with. var state = { fileOrder: [0], files: { 0:{ id: 0, name: 'asdf' } } }; function handleAddFile(state, action) { return {...state, ...{[action.id]:{id: action.id, name: action.name}}}; }; function addFileOrder(state, action) { return [...state, action.id]; } // Adding a file should create a new file, and add its id to

问题 I have a five node cluster, three nodes of which contain DataNodes and TaskTrackers. I've imported around 10million rows from Oracle via Sqoop and process it via MapReduce in an Oozie workflow. The MapReduce job takes about 30 minutes and is only using one reducer. Edit - If I run the MapReduce code on its own, separate from Oozie, the job.setNumReduceTasks(4) correctly establishes 4 reducers. I have tried the following methods to manually set the number of reducers to four, with no success:

问题 I have the following Reducer class public static class TokenCounterReducer extends Reducer<Text, Text, Text, Text> { public void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException { JSONObject jsn = new JSONObject(); for (Text value : values) { String[] vals = value.toString().split("\t"); String[] targetNodes = vals[0].toString().split(",",-1); jsn.put("source",vals[1] ); jsn.put("target",targetNodes); } // context.write(key, new Text(sum)); } }