FLUME IllegalStateException:当事务处于打开状态时调用的begin()

时间:2016-01-22 13:28:15

标签: java flume illegalstateexception flume-ng flume-twitter

我编写了名为MySink的自定义水槽,其处理方法在下面的第一个片段中显示。我得到一个IllegalStateException如下(详细的堆栈跟踪在下面的第二个片段中可用):

  

引起:java.lang.IllegalStateException:begin()时调用   交易是开放的!

问题:我在编写流程方法时遵循了KafkaSink和类似的现有接收器实现,我正在应用与那些现有接收器相同的事务处理逻辑。你能告诉我这里的处理方法有什么问题吗?我该如何解决这个问题?

PROCESS方法(我已标记抛出异常的位置):

@Override
public Status process() throws EventDeliveryException {
    Status status = Status.READY;
    Channel ch = getChannel();
    Transaction txn = ch.getTransaction();
    Event event = null;

    try {
        LOG.info(getName() + " BEFORE txn.begin()");
    //!!!! EXCEPTION IS THROWN in the following LINE !!!!!!
        txn.begin();
        LOG.info(getName() + " AFTER txn.begin()");
        LOG.info(getName() + " BEFORE ch.take()");
        event = ch.take();
        LOG.info(getName() + " AFTER ch.take()");

        if (event == null) {
            // No event found, request back-off semantics from the sink runner
            LOG.info(getName() + " - EVENT is null! ");
            return Status.BACKOFF;
        }

        Map<String, String> keyValueMapInTheMessage = event.getHeaders();
        if (!keyValueMapInTheMessage.isEmpty()) {
            mDBWriter.insertDataToDB(keyValueMapInTheMessage);
        }

        LOG.info(getName() + " - EVENT: " + EventHelper.dumpEvent(event));
        if (txn != null) {
            txn.commit();                
        }

    } catch (Exception ex) {
        String errMsg = getName() + " - Failed to publish events. Exception: ";
        LOG.info(errMsg);
        status = Status.BACKOFF;
        if (txn != null) {
            try {
                txn.rollback();
            } catch (Exception e) {
                LOG.info(getName() + " - EVENT: " + EventHelper.dumpEvent(event));
                throw Throwables.propagate(e);
            }
        }
        throw new EventDeliveryException(errMsg, ex);
    } finally {
        if (txn != null) {
            txn.close();
        }
    }

    return status;
}

EXCEPTION STACK:

2016-01-22 14:01:15,440 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]  

Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: MySink - Failed to publish events.
Exception:    at com.XYZ.flume.maprdb.MySink.process(MySink.java:116)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: begin() called when transaction is OPEN!
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
at com.XYZ.flume.maprdb.MySink.process(MySink.java:82)
... 3 more

1 个答案:

答案 0 :(得分:0)

&#13;
&#13;
if (event == null) {
    // No event found, request back-off semantics from the sink runner
    LOG.info(getName() + " - EVENT is null! ");
    return Status.BACKOFF;
 }
&#13;
&#13;
&#13;

此代码导致此问题。当event为null时,你只需返回它。但是,正确的方法是提交或回滚。事务应该经历三个阶段:开始,提交或回滚,最后关闭。我们可以看到以下源代码来查找它如何实现

BasicChannelSemantics:

&#13;
&#13;
  public Transaction getTransaction() {

    if (!initialized) {
      synchronized (this) {
        if (!initialized) {
          initialize();
          initialized = true;
        }
      }
    }

    BasicTransactionSemantics transaction = currentTransaction.get();
    if (transaction == null || transaction.getState().equals(
            BasicTransactionSemantics.State.CLOSED)) {
      transaction = createTransaction();
      currentTransaction.set(transaction);
    }
    return transaction;
  }
&#13;
&#13;
&#13;

当currentTransaction为null或其State为close时,channel将创建一个新的,否则返回旧的。此异常不会立即发生。当第一次执行process方法时,你得到一个新的事务,但是事件是null,你只是返回并最终关闭,close方法因为它的实现而不起作用。所以第二次执行process方法,你不要#39;得到一个新的交易,它是旧的交易。下面的代码是关于如何实现交易。

BasicTransactionSemantics:

&#13;
&#13;
  protected BasicTransactionSemantics() {
    state = State.NEW;
    initialThreadId = Thread.currentThread().getId();
  }
  
  public void begin() {
    Preconditions.checkState(Thread.currentThread().getId() == initialThreadId,
        "begin() called from different thread than getTransaction()!");
    Preconditions.checkState(state.equals(State.NEW),
        "begin() called when transaction is " + state + "!");

    try {
      doBegin();
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new ChannelException(e.toString(), e);
    }
    state = State.OPEN;
  }

  public void commit() {
    Preconditions.checkState(Thread.currentThread().getId() == initialThreadId,
        "commit() called from different thread than getTransaction()!");
    Preconditions.checkState(state.equals(State.OPEN),
        "commit() called when transaction is %s!", state);

    try {
      doCommit();
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new ChannelException(e.toString(), e);
    }
    state = State.COMPLETED;
  }

 public void rollback() {
    Preconditions.checkState(Thread.currentThread().getId() == initialThreadId,
        "rollback() called from different thread than getTransaction()!");
    Preconditions.checkState(state.equals(State.OPEN),
        "rollback() called when transaction is %s!", state);

    state = State.COMPLETED;
    try {
      doRollback();
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new ChannelException(e.toString(), e);
    }
  }
  
 public void close() {
    Preconditions.checkState(Thread.currentThread().getId() == initialThreadId,
        "close() called from different thread than getTransaction()!");
    Preconditions.checkState(
            state.equals(State.NEW) || state.equals(State.COMPLETED),
            "close() called when transaction is %s"
            + " - you must either commit or rollback first", state);

    state = State.CLOSED;
    doClose();
  }
&#13;
&#13;
&#13;

创建时,状态是新的。

开始时,状态必须是新的,然后状态变为开放状态。

当提交或回滚时,状态必须打开,然后状态变为完成。

关闭时,状态必须完成,然后状态变得接近。

所以当你以正确的方式执行close方法时,下次你将获得一个新的事务,否则旧的一个状态不能是新的,所以你不能执行transaction.begin(),它需要一个新的。