How does one implement a Hadoop Mapper in Scala 2.9.0?

后端 未结 1 1309
后悔当初
后悔当初 2021-01-05 09:45

When I migrated to Scala 2.9.0 from 2.8.1, all of the code was functional except for the Hadoop mappers. Because I had some wrapper objects in the way, I distilled down to

相关标签:
1条回答
  • 2021-01-05 10:03

    Silly as it seems, you can work around the problem by defining the Mapper before the Job. The following compiles:

    import org.apache.hadoop._
    import org.apache.hadoop.io._
    import org.apache.hadoop.conf._
    import org.apache.hadoop.mapreduce._
    
    class MyMapper extends Mapper[LongWritable,Text,Text,Text] {
      override def map(key: LongWritable, value: Text, context: Mapper[LongWritable,Text,Text,Text]#Context) {
      }
    }
    
    object MyJob {
      def main(args:Array[String]) {
        val job = new Job(new Configuration())
        job.setMapperClass(classOf[MyMapper])
      }
    }
    
    0 讨论(0)
提交回复
热议问题