删除重复的单词

时间:2016-12-14 03:16:53

标签: duplicates notepad++ programmatically-created programmatically

我有一个句子列表。

我想处理这样的重复:

  • 白鞋女
  • 女鞋白色
  • 女士白鞋

我想进入这个:

  • 白鞋女

我可以在Notepad ++中执行此操作吗?

或者其他一些软件?

2 个答案:

答案 0 :(得分:1)

我认为你不能在Npp做这样的工作。

这是一种使用perl来完成工作的方法,它保留了第一行的大小写和顺序 (感谢@jwpfox的输入示例)。

use Modern::Perl;

my $prev = '';
while(<DATA>) {
    chomp;
    my $str = join'',sort split' ',lc$_;
    say $_ if $str ne $prev;
    $prev = $str;
}

__DATA__
White shoes women
Shoes women white
Women white shoes
White shoes women
Shoes women white
Women white shoes
Men black boots
Black boots men
Boots men black
girl yellow shirt
yellow girl shirt
pants blue boy

<强>输出:

White shoes women
Men black boots
girl yellow shirt
pants blue boy

PHP中的一个版本:

$s = array(
'White shoes women',
'Shoes women white',
'Women white shoes',
'White shoes women',
'Shoes women white',
'Women white shoes',
'Men black boots',
'Black boots men',
'Boots men black',
'girl yellow shirt',
'yellow girl shirt',
'pants blue boy');

$prev = '';
foreach($s as $line) {
    $list = explode(' ', strtolower($line));
    sort($list);
    $str = implode('',$list);
    if ($str != $prev) echo $line,"\n";
    $prev = $str;
}

<强>输出:

White shoes women
Men black boots
girl yellow shirt
pants blue boy

答案 1 :(得分:0)

使用&#34;其他一些软件&#34;选项。

input.txt档案的内容:

White shoes women
Shoes women white
Women white shoes
Men black boots
Black boots men
Boots men black
girl yellow shirt
yellow girl shirt
pants blue boy

Python 3:

sentences = []

with open('input.txt', mode='r') as infile:
    for line in infile:
        wordlist = line.split(' ')
        words = []
        for word in wordlist:
            word = word.strip()
            words.append(word.lower())

        words.sort()

        if words not in sentences:
            sentences.append(words)

with open('output.txt', mode='w') as outfile:
    for sentence in sentences:
        for word in sentence:
            outfile.write(word + ' ')
        outfile.write('\n')

output.txt档案的内容:

shoes white women 
black boots men 
girl shirt yellow 
blue boy pants