Question

我搜索谷歌无济于事。这里的任何人都可以告诉我如何在Prolog中编写术语频率代码？越“逻辑”越好。给定一个文本文件，忽略非字母数字字符，检测单词，删除停用词（可能在外部文件中给出），计算文件中每个单词的出现次数，并输出类似“word：freq”的内容，按降低频率排序。

谢谢！

（这不是家庭作业，顺便说一句，这是我正在做的一个项目）

Answer 1

这是让你入门的东西：

假设单词各自位于输入文件的单独行

不处理“停用词”

不实现排序

; Print the term frequency for words in file Path
print_freq_from_file(Path) :-
    open( Path, read, Stream ),
    read_file( Stream, Words ),
    calc_freq( Words, Freq ),
    sort_freq( Freq, SortedFreq ),
    print_freq( SortedFreq ).

; calc_freq(Words,Freq) Freq is the term frequency of the words in list Words
calc_freq( [], [] ).
calc_freq( [Word|Words], Freq ) :-
    calc_freq( Words, FreqRest ),
    add_word( Word, FreqRest, Freq ).

; add_word( Word, Freq1, Freq2 ) Freq2 is Freq1 with Word added
add_word( Word, [], [(Word,1)] ).
add_word( Word, [(Word,N)|Rest], [(Word,N1)|Rest] ) :-
    N1 is N+1.
add_word( Word, [Term|Rest], [Term|NewRest] ) :-
    add_word( Word, Rest, NewRest ).

; Print the given term frequency
print_freq( [] ).
print_freq( [(W,N)|Rest] ) :-
    write( W ), write( ' : ' ), write( N ), nl,
    print_freq( Rest ).

Answer 2

我在这个答案中使用SWI-Prolog广泛的库支持，所以在你的情况下可能不合适。

当然，您如何解决任何编程任务受到您所选语言中可用内容的强烈影响，和来自您使用它的能力。

这里我使用library(assoc)，如果您的系统错过了它，那么您可以使用列表或使用断言/撤消进行模拟。

这个片段只显示（惯用） count 字的方式，计算频率可以通过库（聚合）或一些算术轻松完成，也许你想尝试自己写下来锻炼语言...

/*  File:    frequency_of_words.pl
    Author:  Carlo,,,
    Created: May 23 2012
    Purpose: http://stackoverflow.com/questions/10711483/calculating-term-frequency-in-prolog
*/

:- module(frequency_of_words, [count_words/2, count_words/1]).
:- [library(assoc)].

count_words(File, Assoc) :-
    empty_assoc(Empty),
    open(File, read, Stream),
    frequency_of_words(Stream, Empty, Assoc, ""),
    close(Stream).
count_words(File) :-
    count_words(File, Assoc),
    assoc_to_list(Assoc, List),
    maplist(writeln, List).

frequency_of_words(Stream, SoFar, Words, CurrWord) :-
    get_code(Stream, Code),
    (   Code == -1
    ->  update_dictionary(SoFar, Words, CurrWord)
    ;   use_character(Code, SoFar, Updated, CurrWord, NextWord),
        frequency_of_words(Stream, Updated, Words, NextWord)
    ).

update_dictionary(SoFar, SoFar, Word) :-
    skip_word(Word).
update_dictionary(SoFar, Updated, Codes) :-
    atom_codes(Word, Codes),
    ( get_assoc(Word, SoFar, CountSoFar) ; CountSoFar = 0 ),
    WordCount is CountSoFar + 1,
    put_assoc(Word, SoFar, WordCount, Updated).

use_character(Code, SoFar, Updated, CurrWord, NextWord) :-
    (   word_character(Code)
    ->  Updated = SoFar,
        NextWord = [Code|CurrWord]
    ;   reverse(CurrWord, Forward),
        update_dictionary(SoFar, Updated, Forward),
        NextWord = ""
    ).

word_character(Code) :-
    [Code] @>= "A", [Code] @=< "Z" ;
    [Code] @>= "a", [Code] @=< "z" ;
    [Code] @>= "0", [Code] @=< "9" ;
    [Code] == "_".

skip_word(""). % a trick on EOF or consecutive blanks: not really a skipword
skip_word("is").

试验：

?- count_words('frequency_of_words.pl').
0-2
1-3
2012-1
23-1
9-1
A-1
Author-1
Carlo-1
Code-14
Codes-2
CountSoFar-3
Created-1
CurrWord-6
EOF-1
Empty-2
File-3
Forward-2
Full-2
List-2
May-1
NextWord-5
Purpose-1
SoFar-11
StackOverflow-1
Stream-6
Updated-7
Word-5
WordCount-2
Words-3
Z-1
_-1
a-3
answer-1
assoc-1
assoc_to_list-1
atom_codes-1
blanks-1
close-1
consecutive-1
count_words-2
empty_assoc-1
frequency_of_words-5
get_assoc-1
get_code-1
library-1
maplist-1
module-1
not-1
on-1
open-1
or-1
pl-1
put_assoc-1
read-1
really-1
reverse-1
skip_word-3
skipword-1
trick-1
update_dictionary-4
use_character-2
word_character-2
writeln-1
z-1

Answer 3

对于过滤停用词，一种好方法是通过assert：

read_sw(Path) :-
    open( Path, read, Stream ),
    read_words_file( Stream, Words ),
    add_sw(Words).

add_sw([]).
add_sw([Word|Rest]) :-
    assertz(stop_word(Word)),
    add_sw(Rest).

完成此操作后，您可以使用stop_word/1检查单词是否为禁用词并忽略它。例如，如果您正在实施Scott Hunter的答案，您可以在calc_freq/2中添加一个附加条款来过滤掉这些词：

calc_freq( [], [] ).
calc_freq( [Word|Words], Freq ) :-
    stop_word(Word), !,
    calc_freq( Words, Freq ).
calc_freq( [Word|Words], Freq ) :-
    calc_freq( Words, FreqRest ),
    add_word( Word, FreqRest, Freq ).

计算Prolog中的术语频率

3 个答案: