Question

我在理解如何运行GIZA ++的基础知识方面遇到了问题。

我通过StackOverflow（Is there a tutorial about giza++?）和人们在那里提供的链接进行了讨论。我已经从Moses-SMT Github下载并编译了最新的吉萨。

git clone https://github.com/moses-smt/giza-pp.git
cd giza-pp
make

编译成功后，我编写了一个简单的脚本用于测试目的。

#!/bin/bash
SRC=french
TRG=english
PREFIX=out
GIZA=../giza-pp

# Cleaning from previous run ...
rm -f *.log
rm -f *.vcb
rm -f *.snt
rm -f *.vcb.classes
rm -f *.vcb.classes.cats
rm -f *.gizacfg
rm -f *.cooc
rm -f ${PREFIX}*

# Converting plain text into sentence format using the "plain2snt.out" tool ...
${GIZA}/GIZA++-v2/plain2snt.out ${SRC} ${TRG}

# Generating word clusters using the "mkcls" tool ...
${GIZA}/mkcls-v2/mkcls -p${SRC} -V${SRC}.vcb.classes
${GIZA}/mkcls-v2/mkcls -p${TRG} -V${TRG}.vcb.classes

# Generating coocurrence using the "snt2cooc" tool ...
${GIZA}/GIZA++-v2/snt2cooc.out ${SRC}.vcb ${TRG}.vcb ${SRC}_${TRG}.snt > ${SRC}_${TRG}.cooc

# Running "GIZA++" ...
${GIZA}/GIZA++-v2/GIZA++ -S ${SRC}.vcb -T ${TRG}.vcb -C ${SRC}_${TRG}.snt -CoocurrenceFile ${SRC}_${TRG}.cooc -o ${PREFIX} >> giza.log 2>&1

现在，这是我运行脚本后目录的内容。

jakub@jakub-virtual-machine:~/Master/giza-pp_test$ ls
english                   french_english.snt       out.d3.final        out.perp
english_french.snt        french.vcb               out.d4.final        out.t3.final
english.vcb               french.vcb.classes       out.D4.final        out.trn.src.vcb
english.vcb.classes       french.vcb.classes.cats  out.Decoder.config  out.trn.trg.vcb
english.vcb.classes.cats  giza.log                 out.gizacfg         out.tst.src.vcb
french                    out.a3.final             out.n3.final        out.tst.trg.vcb
french_english.cooc       out.A3.final             out.p0_3.final      run_test.sh

重点是下面列出的输出（对我来说很重要）。

out.ti.final
out.actual.ti.final

现在我一直在寻找GIZA的Main.cpp（行：260 - 273）并且可以看到应该创建这些文件的行。

cerr << "writing Final tables to Disk \n";
  string t_inv_file = Prefix + ".ti.final" ;
  if( !FEWDUMPS)
    m1.getTTable().printProbTableInverse(t_inv_file.c_str(), m1.getEnglishVocabList(), 
                     m1.getFrenchVocabList(), 
                     m1.getETotalWCount(), 
                     m1.getFTotalWCount());
  t_inv_file = Prefix + ".actual.ti.final" ;
  if( !FEWDUMPS )
    m1.getTTable().printProbTableInverse(t_inv_file.c_str(), 
                     eTrainVcbList.getVocabList(), 
                     fTrainVcbList.getVocabList(), 
                     m1.getETotalWCount(), 
                     m1.getFTotalWCount(), true);

我也在日志中打印了“cerr”行，但我无法找出输出中不存在这些文件的原因。

jakub@jakub-virtual-machine:~/Master/giza-pp_test$ cat giza.log | tail -n 25
p0_count is 4.0073 and p1 is 5.99635; p0 is 0.400584 p1: 0.599416
Model4: TRAIN CROSS-ENTROPY 0.80096 PERPLEXITY 1.74226
Model4: (10) TRAIN VITERBI CROSS-ENTROPY 0.801289 PERPLEXITY 1.74266
Dumping alignment table (a) to file:out.a3.final
Dumping distortion table (d) to file:out.d3.final
Dumping nTable to: out.n3.final

Model4 Viterbi Iteration : 10 took: 0 seconds
H3333344444 Training Finished at: Fri Oct 23 16:24:44 2015


Entire Viterbi H3333344444 Training took: 0 seconds
==========================================================
writing Final tables to Disk 
Writing PERPLEXITY report to: out.perp
Writing source vocabulary list to : out.trn.src.vcb
Writing source vocabulary list to : out.trn.trg.vcb
Writing source vocabulary list to : out.tst.src.vcb
Writing source vocabulary list to : out.tst.trg.vcb
writing decoder configuration file to out.Decoder.config

Entire Training took: 0 seconds
Program Finished at: Fri Oct 23 16:24:44 2015

==========================================================

有人请遇到类似的问题吗？这是某种错误还是我做错了什么？

编辑：

现在我在-DBINARY_SEARCH_FOR_TTABLE的{{1}}内没有CFLAGS选项重新编译了整个GIZA ++。并更改了脚本，以便它不会生成并向GIZA ++提供共存文件。重新运行脚本后，输出确实包含Makefile和out.actual.ti.final。有谁知道如何解释这种行为？我教过我会使用共存文件获得更好的分配和概率估算，请问有什么需要吗？或者它只是为了提高性能的速度？

Answer 1

我之前遇到过同样的问题。我认为缺少的一步是在位于。\ giza-pp \ GIZA ++ - v2 \的Makefile中，替换以下行： CFLAGS_OPT = $（CFLAGS）-O3 -funroll-loops -DNDEBUG -DWORDINDEX_WITH_4_BYTE -DBINARY_SEARCH_FOR_TTABLE -DWORDINDEX_WITH_4_BYTE

与该行： CFLAGS_OPT = $（CFLAGS）-O3 -funroll-loops -DNDEBUG -DWORDINDEX_WITH_4_BYTE -DWORDINDEX_WITH_4_BYTE

检查此放置并祝你好运

GIZA ++输出缺少* .ti.final和* actual.ti.final文件

1 个答案: