Question

我有一个文件ip_details.txt，其中包含以下内容;

Peer "ATBBB010.domain.com:1111;transport=tcp" ConnectAddress="10.184.88.29,10.184.88.30" LocalPort="0"
Peer "ATBBB020.domain.com:1111;transport=tcp" ConnectAddress="10.184.88.61,10.184.88.62" LocalPort="0"
Peer "CHBBB010.domain.com:1111;transport=tcp" ConnectAddress="10.161.144.5,10.161.144.6" LocalPort="0"
Peer "CHBBB020.domain.com:1111;transport=tcp" ConnectAddress="10.161.144.21,10.161.144.22" LocalPort="0"
Peer "DABBB010.domain.com:1111;transport=tcp" ConnectAddress="10.160.130.5,10.160.130.6" LocalPort="0"
Peer "DABBB020.domain.com:1111;transport=tcp" ConnectAddress="10.160.130.21,10.160.130.22" LocalPort="0"
Peer "ATCCC010.domain.com:1111;transport=tcp" ConnectAddress="10.199.88.29,10.199.88.30" LocalPort="0"
Peer "ATCCC020.domain.com:1111;transport=tcp" ConnectAddress="10.199.88.61,10.199.88.62" LocalPort="0"
Peer "CHCCC010.domain.com:1111;transport=tcp" ConnectAddress="10.161.155.5,10.161.155.6" LocalPort="0"
Peer "CHCCC020.domain.com:1111;transport=tcp" ConnectAddress="10.161.155.21,10.161.155.22" LocalPort="0"
Peer "DACCC010.domain.com:1111;transport=tcp" ConnectAddress="10.199.130.5,10.199.130.6" LocalPort="0"
Peer "DACCC020.domain.com:1111;transport=tcp" ConnectAddress="10.199.130.21,10.199.130.22" LocalPort="0"

我目前使用这两个脚本/命令来ping特定类型节点的IP地址。

首先对BBB节点进行Ping（逗号分隔符之前的IP）;

for i in `cat ip_details.txt | grep BBB | awk '{print $3}' | cut -d= -f2 | cut -d, -f1 | tr -d '[="=]'`; do ping -I eth0 -c 3 $i >> /dev/null ; [ $? != 0 ] && echo $i || echo "Ping to $i Good"; done

为BBB节点Ping第二个IP（逗号分隔符后面的IP）;

for i in `cat ip_details.txt | grep BBB | awk '{print $3}' | cut -d= -f2 | cut -d, -f2 | tr -d '[="=]'`; do ping -I eth0 -c 3 $i >> /dev/null ; [ $? != 0 ] && echo $i || echo "Ping to $i Good"; done

我得到以下内容;

Ping to 10.184.88.29 Good
Ping to 10.184.88.61 Good
.
.

同样地，我为CCC节点grep CCC。

我需要一个脚本来ping每个节点类型或两种类型的IP地址，并得到这样的输出;

Ping from <hostname> to ATBBB010 IP1 <ip address> is Good
....
....
....

Ping from <hostname> to ATBBB010 IP2 <ip address> is Good 
....
....
....

如果ping失败;

Ping from <hostname> to ATBBB010 IP1 <ip address> failed
.....
.....

Ping from <hostname> to ATBBB010 IP2 <ip address> failed
.....
.....

Answer 1

尝试以下方法：

object STRPartitioner extends Serializable{
  def apply(expectedParNum: Int,
        sampleRate: Double,
        originRdd: RDD[Vertex]): Unit= {
    val bound = computeBound(originRdd)
    val rdd = originRdd.mapPartitions(
      iter => iter.map(row => {
        val cp = row
        (cp.coordinate, cp.copy())
      }
      )
    )
    val partitioner = new STRPartitioner(expectedParNum, sampleRate, bound, rdd)
    val shuffled = new ShuffledRDD[Coordinate, Vertex, Vertex](rdd,  partitioner)
    shuffled.setSerializer(new KryoSerializer(new SparkConf(false)))
    val result = shuffled.collect()
  }

class STRPartitioner(expectedParNum: Int,
                     sampleRate: Double,
                     bound: MBR,
                     rdd: RDD[_ <: Product2[Coordinate, Vertex]])
  extends Partitioner with  Serializable {
    ... 
}

bash脚本 - 循环中的2个变量

1 个答案: