使用allenai / bilm-tf进行问答

时间:2019-08-07 09:29:17

标签: python tensorflow nlp

我正在尝试使用allenai/bilm-tf进行问答,并试图从usage_character.py获得结果,但是我不知道如何从此示例继续。

raw_context = [
    'Pretrained biLMs compute representations useful for NLP tasks .',
    'They give state of the art performance for many tasks .'
]
tokenized_context = [sentence.split() for sentence in raw_context]
tokenized_question = [
    ['What', 'are', 'biLMs', 'useful', 'for', '?'],
]

with tf.Session() as sess:
    # It is necessary to initialize variables once before running inference.
    sess.run(tf.global_variables_initializer())

    # Create batches of data.
    context_ids = batcher.batch_sentences(tokenized_context) #shape: [2, 11, 50]
    question_ids = batcher.batch_sentences(tokenized_question)

    # Compute ELMo representations (here for the input only, for simplicity).
    elmo_context_input_, elmo_question_input_ = sess.run(
        [elmo_context_input['weighted_op'], elmo_question_input['weighted_op']],
        feed_dict={context_character_ids: context_ids,
                   question_character_ids: question_ids})

输出sess.run()返回使我感到困惑,因为我期望一个带有概率/对数的2d数组执行argmax来获得tokenized_context的跨度,这就是我的答案。但这种情况并非如此。 elmo_context_input_的形状为[2,11,32]。

我的问题现在是:

  • 我是否需要执行其他预测步骤(不列颠哥伦比亚作者提到:“此处仅用于输入,为简单起见”)
  • 如何将输出投射回原始文本和/或词汇文件以接收答案

谢谢。

0 个答案:

没有答案