如何在Spark Scala中找到类型为元组的迭代器中的元素总和？

Question

我正在使用spark scala中的不同功能和技术。我想从scala spark

中找到类型（int，int）的迭代器中的第一个元素的总和

请告诉我上述语法中的问题

Answer 1

在map函数中， x._2 是（Int，Int）的迭代器，你需要使用 map 从每个元组中提取第一个元素，然后总结一下：

import React, { Component } from 'react';

import {
StyleSheet,
Text,
View,
Image, // we want to use an image
PanResponder, // we want to bring in the PanResponder system
Animated // we wil be using animated value
} from 'react-native';

export default class Tag extends React.Component {

constructor(props) {
  super(props);

  this.state = {
  pan: new Animated.ValueXY(),
  scale: new Animated.Value(1)
  };
}
_handleStartShouldSetPanResponder(e, gestureState) {
  return true;
}

_handleMoveShouldSetPanResponder(e, gestureState) {
 return true;
}

componentWillMount() {
  this._panResponder = PanResponder.create({
    onStartShouldSetPanResponder: 
  this._handleStartShouldSetPanResponder.bind(this),
    onMoveShouldSetPanResponder: 
  this._handleMoveShouldSetPanResponder.bind(this),

  onPanResponderGrant: (e, gestureState) => {
    // Set the initial value to the current state
    this.state.pan.setOffset({x: this.state.pan.x._value, y: this.state.pan.y._value});
    this.state.pan.setValue({x: 30*Math.random(), y: 0});
    Animated.spring(
      this.state.scale,
      { toValue: 1.1, friction: 1 }
    ).start();
  },

  // When we drag/pan the object, set the delate to the states pan position
  onPanResponderMove: Animated.event([
    null, {dx: this.state.pan.x, dy: this.state.pan.y},
  ]),

  onPanResponderRelease: (e, {vx, vy}) => {
    // Flatten the offset to avoid erratic behavior
    this.state.pan.flattenOffset();
    Animated.spring(
      this.state.scale,
      { toValue: 1, friction: 1 }
    ).start();
    }
   });
  }

  render() {
// Destructure the value of pan from the state
let { pan, scale } = this.state;

// Calculate the x and y transform from the pan value
let [translateX, translateY] = [pan.x, pan.y];

let rotate = '0deg';
// Calculate the transform property and set it as a value for our style which we add below to the Animated.View component
let imageStyle = {transform: [{translateX}, {translateY}, {rotate}, {scale}]};

return (
    <Animated.View style={[imageStyle, styles.container]} {...this._panResponder.panHandlers} >
    <View style={styles.rect}>
      <Text style={styles.txt} >tgyyHH</Text>
      </View>
    </Animated.View>
);
}

}

const styles = StyleSheet.create({
  container: {
  width:50,
  height:50,
  position: 'absolute'
},
rect: {
  borderRadius:4,
  borderWidth: 1,
  borderColor: '#fff',
  width:50,
  height:50,
  backgroundColor:'#68a0cf', 

  },
  txt: {
    color:'#fff',
    textAlign:'center'
  }

 });

Answer 2

看起来你使用groupByKey - 这是一个非常糟糕的选择。

您应该做的是mapValues后跟reduceByKey：

val rdd = sc.parallelize(Seq(("first", (1, 1)), ("first", (1, 1))))

rdd.mapValues(_._1).reduceByKey(_ + _)

或aggregateByKey

rdd.aggregateByKey(0)({ case (acc, (x, _)) => acc + x}, _ + _)