使用Perl将段落转换为句子

时间:2013-04-01 02:53:26

标签: perl text-segmentation

我在做Perl编程。我需要阅读一个段落并将每个句子打印出来作为一行。

任何人都知道怎么做?

以下是我的代码:

#! /C:/Perl64/bin/perl.exe

use utf8;

if (! open(INPUT, '< text1.txt')){
die "cannot open input file: $!";
}

if (! open(OUTPUT, '> output.txt')){
die "cannot open input file: $!";
}

select OUTPUT;

while (<INPUT>){
print "$_";
}

close INPUT;
close OUTPUT;
select STDOUT;

3 个答案:

答案 0 :(得分:6)

我不会处理文件名,而是让Perl这样做。

这在多个层面非常粗糙,而且完整的工作无疑是艰难的。

sentence.pl

#!/usr/bin/env perl
use strict;
use warnings;
use Lingua::EN::Sentence qw(get_sentences);

sub normalize
{
    my($str) = @_;
    $str =~ s/\n/ /gm;
    $str =~ s/\s\s+/ /gm;
    return $str;
}

{
    local $/ = "\n\n";
    while (<>)
    {
        chomp;
        print "Para: [[$_]]\n";
        my @sentences = split m/(?<=[.!?])\s+/m, $_;
        foreach my $sentence (@sentences)
        {
            $sentence = normalize $sentence;
            print "Ad Hoc Sentence: $sentence\n";
        }
        my $sref = get_sentences($_);
        foreach my $sentence (@$sref)
        {
            $sentence = normalize $sentence;
            print "Lingua Sentence: $sentence\n";
        }
    }
}

split正则表达式查找一个或多个前面带有句号(句号),感叹号或问号的空格,并匹配多行。后视(?<=[.!?])意味着标点符号与句子一致。 normalize函数只是将换行符展平为空格,并将多个空格渲染为单个空格。 (请注意,这不能正确识别括号句。)这将被视为上一句的一部分,因为.后面没有空格。

示例输入

This is a paragraph with more than one sentence in it.  How many will be
determined later.  Mr. A. P. McDowney has been rather busy.  This
incomplete sentence will still be counted as one

This is the second paragraph.  With three sentences in it, it is a lot
less exciting than the first paragraph, but the middle sentence extends
over multiple lines and   there   is     some         wonky spacing too.
But 'tis time to finish.

示例输出

Para: [[This is a paragraph with more than one sentence in it.  How many will be
determined later.  Mr. A. P. McDowney has been rather busy.  This
incomplete sentence will still be counted as one]]
Ad Hoc Sentence: This is a paragraph with more than one sentence in it.
Ad Hoc Sentence: How many will be determined later.
Ad Hoc Sentence: Mr.
Ad Hoc Sentence: A.
Ad Hoc Sentence: P.
Ad Hoc Sentence: McDowney has been rather busy.
Ad Hoc Sentence: This incomplete sentence will still be counted as one
Lingua Sentence: This is a paragraph with more than one sentence in it.
Lingua Sentence: How many will be determined later.
Lingua Sentence: Mr. A. P. McDowney has been rather busy.
Lingua Sentence: This incomplete sentence will still be counted as one
Para: [[This is the second paragraph.  With three sentences in it, it is a lot
less exciting than the first paragraph, but the middle sentence extends
over multiple lines and   there   is     some         wonky spacing too.
But 'tis time to finish.
]]
Ad Hoc Sentence: This is the second paragraph.
Ad Hoc Sentence: With three sentences in it, it is a lot less exciting than the first paragraph, but the middle sentence extends over multiple lines and there is some wonky spacing too.
Ad Hoc Sentence: But 'tis time to finish.
Lingua Sentence: This is the second paragraph.
Lingua Sentence: With three sentences in it, it is a lot less exciting than the first paragraph, but the middle sentence extends over multiple lines and there is some wonky spacing too.
Lingua Sentence: But 'tis time to finish.

请注意Lingua::EN::Sentence如何设法处理'先生' A. P. McDowney'比简单的正则表达式更好。

答案 1 :(得分:4)

识别句子非常困难并且特定于语言。你需要帮助。也许Lingua::EN::Sentence是要走的路?

答案 2 :(得分:-1)

如果您将段落作为字符串给出,则可以在标记句子结尾的字符上split()

例如:

my @sentences = split /[.?!]/, $paragraph;