Question

我创建了一个自定义的log4j附加程序，用于数据块中，以记录作业错误到松弛状态：

package aaa.bbb.slackAppender

import org.apache.http.client.methods.{CloseableHttpResponse, HttpPost}
import org.apache.http.entity.StringEntity
import org.apache.http.impl.client.HttpClientBuilder
import org.apache.http.util.EntityUtils
import org.apache.log4j.{AppenderSkeleton, Level}
import org.apache.log4j.spi.LoggingEvent
import org.json.simple.JSONValue

class SlackAppender extends AppenderSkeleton {
  override def append(event: LoggingEvent): Unit = {

    val level = event.getLevel

    if (level == Level.ERROR) {

      val escaped = JSONValue.escape(event.getMessage.toString)

      val escapedMessage = "{\"text\":\"" + escaped + "\"}"

      val post = new HttpPost("https://hooks.slack.com/services/XXXX/YYYY/ZZZZ")
      post.setHeader("Content-type", "application/json")
      post.setEntity(new StringEntity(escapedMessage))

      val client = HttpClientBuilder.create.build
      val response:CloseableHttpResponse = client.execute(post)
      val entity = response.getEntity
      val str = EntityUtils.toString(entity,"UTF-8")
      println("Error post response code is " + str)

    }
  }

  override def close(): Unit = {}

  override def requiresLayout(): Boolean = true
}

从笔记本或类似作业中运行时，它基本上可以正常工作：

import org.apache.log4j.{LogManager, Level, Logger}
import org.apache.commons.logging.LogFactory
import aaa.bbb.slackAppender.SlackAppender

Logger.getRootLogger().addAppender(new SlackAppender());

val log = LogFactory.getLog("misc-test-log")
log.error("errorA")
log.error("errorB")
log.error("errorC")

但是，从笔记本电脑运行时，如果我已经运行了N次笔记本电脑，则会将日志复制N次。换句话说，第一次运行后，我在Slack中看到了

errorA
errorB
errorC

运行5次后，我看到类似以下内容的

errorA
errorA
errorA
errorA
errorA
errorB
errorB
errorB
errorB
errorB
errorC
errorC
errorC
errorC
errorC

似乎日志已以某种方式保存并每次都重新记录。幸运的是，当我将笔记本作为作业运行时，看不到相同的东西。

有人知道为什么会这样和/或我如何阻止它发生吗？

Answer 1

基于李维·拉姆齐（Levi Ramsey）的上述评论，进行了以下更改：

Logger.getRootLogger().addAppender(new SlackAppender());

收件人：

new SlackAppender().initializeAppender()

其中initializeAppender（）为：

  def initializeAppender() {
    val rootLogger = Logger.getRootLogger

    val en: util.Enumeration[_] = rootLogger.getAllAppenders()
    while ( {en.hasMoreElements}) {
      val appender: Any = en.nextElement
      if (appender.isInstanceOf[SlackAppender]) {
        val slackAppender: Appender = appender.asInstanceOf[Appender]
        rootLogger.removeAppender(slackAppender)
      }
    }
    rootLogger.addAppender(new SlackAppender)

  }

仍然进行测试，但这似乎是一个可能的答案。

使用自定义log4j附加程序从databricks笔记本复制日志

1 个答案: