根据正则表达式,第二意见将大文件拆分为较小的文件

时间:2011-06-27 21:07:31

标签: perl

好的,所以我已经阅读了不同的方法,但我只想检查一下我做过的方式是否存在看不见的问题,或者是否有更好的方法(也许是grep?)

这是我的工作代码:

#!usr/bin/perl

use strict;
use warnings;

my $chapternumber;
open my $corpus, '<', "/Users/jon/Desktop/chpts/chpt1-8/Lifeprocessed.txt" or die $!;
while (my $sentence = <$corpus>) 
    {
    if ($sentence =~ /\~\s(\d*F*[\.I_]\w+)\s/ ) 
        {
        $chapternumber = $1;
        $chapternumber =~ s/\./_/;
        }

    open my $outfile, '>>', "/Users/jon/Desktop/chpts/chpt$chapternumber.txt" or die $!;
    print $outfile $sentence;
    }

该文件是一本教科书,我在下面列出了新的章节:~ 1.1 Organisms Have Changed over Billions of Years 1.1.~ 15Intro ...~ F_14我想将其作为新文件的开头:chpt1_1.txt(或其他chpt15Intro等....)。当我找到下一章分隔符时,哪个结束。

1选项:也许不是逐行,只是像这样得到整个块? :

 local $/ = "~";
 open...
 while...
 next unless ($sentenceblock =~ /\~\s([\d+F][\.I_][\d\w]+)\s/);
....

非常感谢。

3 个答案:

答案 0 :(得分:8)

首先,好事:

enabled strict and warnings
using 3-arg open and lexical filehandles
checking the return value from open()

但是你的正则表达毫无意义。

~ is not "meta" in regexes, so it does not need escaping
. is not "meta" in a character class, so it does not need escaping
[\d+F] is equivalent to [+F\d] (what is the "F" for? + matches a literal plus character in a character class, it does NOT mean "one or more" here
[\.I_] what is the "I" for? What is the underscore for?
[\d\w] is equivalent to [\w] and even to just \w

您的代码调用open()的方式需要更多次。

tr ///优于s ///用于处理单个字符。

希望这会让你走上正轨:

#!/usr/bin/perl
use warnings;
use strict;

my $outfile;
while (<DATA>) {
    if ( my($chapternumber) = /^~\s([\d.]+)/) {
        $chapternumber =~ tr/./_/;
        close $outfile if $outfile;
        open $outfile, '>', "chpt$chapternumber.txt"
            or die "could not open 'chpt$chapternumber.txt' $!";
    }
    print {$outfile} $_;
}

__DATA__
~ 1.1 Organisms Have Changed over Billions of Years 1.1
stuff
about changing
organisms
~ 1.2 Chapter One, Part Two 1.2
part two
stuff is here

答案 1 :(得分:1)

嗯...也许是csplit?

将以下内容保存到文件中,例如splitter.sh

csplit -s -f tmp - '/^~ [0-9][0-9]*\./'
ls tmp* | while read file
do
    title=($(head -1 $file))
    mv $file chpt${title[1]//./_}.txt
done

并使用它

bash splitter.sh < book.txt

答案 2 :(得分:0)

为什么不在整个内容中啜饮?然后你可以匹配每个章节标题。 /m使^匹配多行字符串中的所有行开头,/g匹配while中所有匹配的相同模式,直到不再匹配出现。 man perlre

#!/usr/bin/perl

use strict;
use warnings;

open my $corpus, '<', '/Users/jon/..../Lifeprocessed.txt' or die $!;
undef $/;
my $contents = <$corpus>;
close($corpus);

while ( $contents =~ /^\~\s([\d+F][\.I_][\d\w]+)\s/mg ) {
    ( my $chapternumber = $1 ) =~ s/\./_/;
    open my $outfile, '>>', "/Users/jon/Desktop/chpts/chpt$chapternumber.txt" or die $!;
    print $outfile $sentence;
    close $outfile;
}