带有 eol 错误的笔记本参数记录错误代码

时间:2021-04-28 20:05:28

标签: pyspark jupyter-notebook azure-synapse

我正在尝试在 Azure Synapse 工作区的笔记本中捕获错误日志记录。我尝试了几种方法,将活动中的字符串值作为动态内容传递。
我尝试过和“学到的”:

<块引用>
 - @activity('Raise an Error').error.message in pyspark returns and EOL error
 - @trim(activity('Raise an Error').error.message) - OK for simple Strings but when I get anything close to real exception message I get
EOL errors in pyspark 
 - @binary(trim(activity('Raise an Error').error.message))  

[我在这里抓着稻草!] 我明白

<块引用>
ErrorDescription": {
            "value": {
                "$content-type": "application/octet-stream",
                "$content": "U3lzdGVtLkludmFsaWRPcGVyYXRpb25FeGNlcHRpb246IEZpbGUgY3JlYXRpb24KICAgYXQgU3VibWlzc2lvbiMxOC48PEluaXRpYWxpemU+PmRfXzAuTW92ZU5leHQoKQotLS0gRW5kIG9mIHN0YWNrIHRyYWNlIGZyb20gcHJldmlvdXMgbG9jYXRpb24gd2hlcmUgZXhjZXB0aW9uIHdhcyB0aHJvd24gLS0tCiAgIGF0IE1pY3Jvc29mdC5Db2RlQW5hbHlzaXMuU2NyaXB0aW5nLlNjcmlwdEV4ZWN1dGlvblN0YXRlLlJ1blN1Ym1pc3Npb25zQXN5bmNbVFJlc3VsdF0oSW1tdXRhYmxlQXJyYXlgMSBwcmVjZWRpbmdFeGVjdXRvcnMsIEZ1bmNgMiBjdXJyZW50RXhlY3V0b3IsIFN0cm9uZ0JveGAxIGV4Y2VwdGlvbkhvbGRlck9wdCwgRnVuY2AyIGNhdGNoRXhjZXB0aW9uT3B0LCBDYW5jZWxsYXRpb25Ub2tlbiBjYW5jZWxsYXRpb25Ub2tlbik="
            },
            "type": "string"
        }, 

这让我在 .NET Spark (C#) 中出现 EOL 错误,我怀疑我是否可以访问输入的 #content 部分,然后我可以对其进行解码并将其写入文件

所以我的问题是如何最好地捕捉错误。我有 4 种类型,我可以将参数作为 (string, int, float, bool) 字符串传递,这是我最好的选择。我通过的内容类似于:

var ErrorDescription = \"System.InvalidOperationException: Failure happened on Source side. ErrorCode=RestSourceCallFailed,Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The HttpStatusCode 400 indicates failure.\nRequest URL: https://restapp.name.provider-og.cloud/rest/v1/domain/data/SOMEMETHOD?qc=SOMEVALUE,=,134578\nResponse payload:<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body>\r\n<center><h1>400 Bad Request</h1></center>\r\n</body>\r\n</html>\r\n,Source=Microsoft.DataTransfer.ClientLibrary,\n   at Submission#18.<<Initialize>>d__0.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n   at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)\"

作为一个字符串,我无法弄清楚如何避免 EOL (pyspark) 或多个 .NET 错误(CS1010、CS1002、CS1525、CS1003)用于转义字符。

我不介意“不回答”,即如果您有合理的建议,我会尝试并提供反馈。我不害怕工作只是出于想法。也就是说,如果您有答案,请随时分享。

非常感谢。

更新 我试图将相同的错误作为参数添加到数据流中(承认笔记本方法失败)这也是错误:

{"message":"Job failed due to reason: Expression cannot be parsed. Details:Parameter stream  has parsing errors\nLine 18 Position 17: extraneous input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', OPEN_INTERPOLATE}","failureType":"UserError","target":"Data flow1","errorCode":"DF-Executor-ParseError"}

0 个答案:

没有答案