我正在尝试在 Azure Synapse 工作区的笔记本中捕获错误日志记录。我尝试了几种方法,将活动中的字符串值作为动态内容传递。
我尝试过和“学到的”:
- @activity('Raise an Error').error.message in pyspark returns and EOL error
- @trim(activity('Raise an Error').error.message) - OK for simple Strings but when I get anything close to real exception message I get
EOL errors in pyspark
- @binary(trim(activity('Raise an Error').error.message))
[我在这里抓着稻草!] 我明白
<块引用>ErrorDescription": {
"value": {
"$content-type": "application/octet-stream",
"$content": "U3lzdGVtLkludmFsaWRPcGVyYXRpb25FeGNlcHRpb246IEZpbGUgY3JlYXRpb24KICAgYXQgU3VibWlzc2lvbiMxOC48PEluaXRpYWxpemU+PmRfXzAuTW92ZU5leHQoKQotLS0gRW5kIG9mIHN0YWNrIHRyYWNlIGZyb20gcHJldmlvdXMgbG9jYXRpb24gd2hlcmUgZXhjZXB0aW9uIHdhcyB0aHJvd24gLS0tCiAgIGF0IE1pY3Jvc29mdC5Db2RlQW5hbHlzaXMuU2NyaXB0aW5nLlNjcmlwdEV4ZWN1dGlvblN0YXRlLlJ1blN1Ym1pc3Npb25zQXN5bmNbVFJlc3VsdF0oSW1tdXRhYmxlQXJyYXlgMSBwcmVjZWRpbmdFeGVjdXRvcnMsIEZ1bmNgMiBjdXJyZW50RXhlY3V0b3IsIFN0cm9uZ0JveGAxIGV4Y2VwdGlvbkhvbGRlck9wdCwgRnVuY2AyIGNhdGNoRXhjZXB0aW9uT3B0LCBDYW5jZWxsYXRpb25Ub2tlbiBjYW5jZWxsYXRpb25Ub2tlbik="
},
"type": "string"
},
这让我在 .NET Spark (C#) 中出现 EOL 错误,我怀疑我是否可以访问输入的 #content 部分,然后我可以对其进行解码并将其写入文件
所以我的问题是如何最好地捕捉错误。我有 4 种类型,我可以将参数作为 (string, int, float, bool) 字符串传递,这是我最好的选择。我通过的内容类似于:
var ErrorDescription = \"System.InvalidOperationException: Failure happened on Source side. ErrorCode=RestSourceCallFailed,Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The HttpStatusCode 400 indicates failure.\nRequest URL: https://restapp.name.provider-og.cloud/rest/v1/domain/data/SOMEMETHOD?qc=SOMEVALUE,=,134578\nResponse payload:<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body>\r\n<center><h1>400 Bad Request</h1></center>\r\n</body>\r\n</html>\r\n,Source=Microsoft.DataTransfer.ClientLibrary,\n at Submission#18.<<Initialize>>d__0.MoveNext()\n--- End of stack trace from previous location where exception was thrown ---\n at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)\"
作为一个字符串,我无法弄清楚如何避免 EOL (pyspark) 或多个 .NET 错误(CS1010、CS1002、CS1525、CS1003)用于转义字符。
我不介意“不回答”,即如果您有合理的建议,我会尝试并提供反馈。我不害怕工作只是出于想法。也就是说,如果您有答案,请随时分享。
非常感谢。
更新 我试图将相同的错误作为参数添加到数据流中(承认笔记本方法失败)这也是错误:
{"message":"Job failed due to reason: Expression cannot be parsed. Details:Parameter stream has parsing errors\nLine 18 Position 17: extraneous input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', OPEN_INTERPOLATE}","failureType":"UserError","target":"Data flow1","errorCode":"DF-Executor-ParseError"}