Question

我的团队正在构建基于flink的实时计算平台。我们将flink作业提交给Yarn。我们创建一个流程并使用CLI运行commit命令。为了获得纱线应用程序ID，我们创建一个线程并解析过程输出。应用程序ID用于其他方法。

例如，我们通过以下命令提交作业：

nohup flink run  -m yarn-cluster -d -yqu root.default 
-ynm BDP_RTC_FLINK_10457_MultiOutputTestFrontEnd -yjm 1024 
-yn 2 -ytm 1024 -ys 2

输出如下所示：

2018-10-10 11:21:04 [info] 2018-10-10 11:21:04,629 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Submitting application master application_1536669298614_67675
2018-10-10 11:21:04 [info] 2018-10-10 11:21:04,654 INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl         - Submitted application application_1536669298614_67675
2018-10-10 11:21:04 [info] 2018-10-10 11:21:04,656 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deploying cluster, current state ACCEPTED
2018-10-10 11:21:12 [info] 2018-10-10 11:21:12,699 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - YARN application has been deployed successfully.
2018-10-10 11:21:12 [info] 2018-10-10 11:21:12,700 INFO  org.apache.flink.yarn.AbstractYarnClusterDescriptor           - The Flink YARN client has been started in detached mode.

我们解析流程输出并获取应用程序ID：application_1536669298614_67675。

在我们这种情况下，是否还有其他优雅的解决方案来获取应用程序ID？

Answer 1

也许您可以获得纱线应用程序和flink作业之间的关系。

首先，列出纱线的应用。

package calculator;

import java.util.Stack; 

public class EvaluateString 
{ 
    public static int evaluate(String expression) 
    { 
            System.out.println(expression);
        char[] tokens = expression.toCharArray(); 
                System.out.println(tokens);
        // Stack for numbers: 'values' 
        Stack<Integer> values = new Stack<Integer>(); 

        // Stack for Operators: 'ops' 
        Stack<Character> ops = new Stack<Character>(); 

        for (int i = 0; i < tokens.length; i++) 
        { 

            // Current token is a number, push it to stack for numbers 
            if (tokens[i] >= '0' && tokens[i] <= '9') 
            { 
                StringBuffer sbuf = new StringBuffer(); 
                // There may be more than one digits in number 
                while (i < tokens.length && tokens[i] >= '0' && tokens[i] <= '9') 
                    sbuf.append(tokens[i++]); 
                values.push(Integer.parseInt(sbuf.toString())); 
            } 


            // Current token is an operator. 
            else if (tokens[i] == '+' || tokens[i] == '-' || 
                    tokens[i] == '*' || tokens[i] == '/') 
            { 
                // While top of 'ops' has same or greater precedence to current 
                // token, which is an operator. Apply operator on top of 'ops' 
                // to top two elements in values stack 
                while (!ops.empty() && hasPrecedence(tokens[i], ops.peek())) 
                values.push(applyOp(ops.pop(), values.pop(), values.pop())); 

                // Push current token to 'ops'. 
                ops.push(tokens[i]); 
            } 
        } 

        // Entire expression has been parsed at this point, apply remaining 
        // ops to remaining values 
        while (!ops.empty()) 
            values.push(applyOp(ops.pop(), values.pop(), values.pop())); 

        // Top of 'values' contains result, return it 
        return values.pop(); 
    } 

    // Returns true if 'op2' has higher or same precedence as 'op1', 
    // otherwise returns false. 
    public static boolean hasPrecedence(char op1, char op2) 
    { 

        if ((op1 == '*' || op1 == '/') && (op2 == '+' || op2 == '-')) 
            return false; 
        else
            return true; 
    } 

    // A utility method to apply an operator 'op' on operands 'a' 
    // and 'b'. Return the result. 
    public static int applyOp(char op, int b, int a) 
    { 
        switch (op) 
        { 
        case '+': 
            return a + b; 
        case '-': 
            return a - b; 
        case '*': 
            return a * b; 
        case '/': 
            if (b == 0) 
                throw new
                UnsupportedOperationException("Cannot divide by zero"); 
            return a / b; 
        } 
        return 0; 
    } 

    // Driver method to test above methods 
    public static void main(String[] args) 
    { 
        System.out.println(EvaluateString.evaluate("100*2+12")); 
    } 
}

然后，您将获得应用程序列表，并可以在yarn应用程序上列出flink作业。

yarn application -list

顺便说一句，您可以使用

./bin/flink list -m yarn-cluster -yid <Yarn Application Id>

不使用

./bin/flink run -d

在使用命令行界面将Flink作业提交到Yarn中时，如何获取应用程序ID？

1 个答案: