我们如何通过Spark-Core中的两个不同字段实现排序?

时间:2017-03-10 09:51:31

标签: apache-spark

我正在做一些基本的编程

InputFile:

if (is_object($dataDecode)) {
   $em->createQuery('DELETE FROM AppBundle:Machine_1')->execute();
   $em->createQuery('DELETE FROM AppBundle:Machine_2')->execute();
   $em->createQuery('DELETE FROM AppBundle:Machine_3')->execute();
   .....
}

我的Spark代码:

display: inline-table;

我希望输出按年份按升序排序,并且每年要按降序排序值

预期输出:

display: inline-block;

有人可以帮助我获得这个结果吗?

1 个答案:

答案 0 :(得分:0)

您必须定义包含年份和年份值的类。该类应该通过重写比较方法来扩展Ordered。比使用此类的对象作为键值并应用sortBy操作。

class TwoKeys(var first: Int, var second: Int) extends Ordered[TwoKeys] {
    def compare(that: TwoKeys): Int = {
      if(first == that.first){
        that.second - second 
      }else{
        first - that.first
      }
    }
  }
...
val keyValueRDD = mapRDD.map(elem => (TwoKeys(elem(0), elem(1)), TwoKeys(elem(0), elem(1))))
val sortRDD = keyValueRDD.sortByKey(true,1)