Question

我是Python的新手，我正在处理下面的推文：

@PrincessSuperC Hey Cici sweetheart! Just wanted to let u know I luv u! OH! and will the mixtape drop soon? FANTASY RIDE MAY 5TH!!!!  
@Msdebramaye I heard about that contest! Congrats girl!! 
UNC!!! NCAA Champs!! Franklin St.: I WAS THERE!! WILD AND CRAZY!!!!!! Nothing like it...EVER http://tinyurl.com/49955t3
Do you Share More #jokes #quotes #music #photos or #news #articles on #Facebook or #Twitter?
Good night #Twitter and #TheLegionoftheFallen.  5:45am cimes awfully early!
I just finished a 2.66 mi run with a pace of 11'14"/mi with Nike+ GPS. #nikeplus #makeitcount
Disappointing day. Attended a car boot sale to raise some funds for the sanctuary, made a total of 88p after the entry fee - sigh
no more taking Irish car bombs with strange Australian women who can drink like rockstars...my head hurts.
Just had some bloodwork done. My arm hurts

它应该具有如下特征向量的输出：

featureList = ['hey', 'cici', 'luv', 'mixtape', 'drop', 'soon', 'fantasy', 'ride', 'heard', 
'congrats', 'ncaa', 'franklin', 'wild', 'share', 'jokes', 'quotes', 'music', 'photos', 'news',
'articles', 'facebook', 'twitter', 'night', 'twitter', 'thelegionofthefallen', 'cimes', 'awfully',
'finished', 'mi', 'run', 'pace', 'gps', 'nikeplus', 'makeitcount', 'disappointing', 'day', 'attended',
'car', 'boot', 'sale', 'raise', 'funds', 'sanctuary', 'total', 'entry', 'fee', 'sigh', 'taking',
'irish', 'car', 'bombs', 'strange', 'australian', 'women', 'drink', 'head', 'hurts', 'bloodwork', 
'arm', 'hurts']

然而，我得到的当前输出只是

hey, cici, luv, mixtape, drop, soon, fantasy, ride

仅来自第一条推文。它只是在一条推文中循环而没有进入下一行。我试图使用nextLine但显然它不适用于Python。我的代码如下：

#import regex
import re
import csv
import pprint
import nltk.classify

#start replaceTwoOrMore
def replaceTwoOrMore(s):
    #look for 2 or more repetitions of character
    pattern = re.compile(r"(.)\1{1,}", re.DOTALL) 
    return pattern.sub(r"\1\1", s)
#end

#start process_tweet
def processTweet(tweet):
    # process the tweets

    #Convert to lower case
    tweet = tweet.lower()
    #Convert www.* or https?://* to URL
    tweet = re.sub('((www\.[\s]+)|(https?://[^\s]+))','URL',tweet)
    #Convert @username to AT_USER
    tweet = re.sub('@[^\s]+','AT_USER',tweet)    
    #Remove additional white spaces
    tweet = re.sub('[\s]+', ' ', tweet)
    #Replace #word with word
    tweet = re.sub(r'#([^\s]+)', r'\1', tweet)
    #trim
    tweet = tweet.strip('\'"')
    return tweet
#end 

#start getStopWordList
def getStopWordList(stopWordListFileName):
    #read the stopwords
    stopWords = []
    stopWords.append('AT_USER')
    stopWords.append('URL')

    fp = open(stopWordListFileName, 'r')
    line = fp.readline()
    while line:
        word = line.strip()
        stopWords.append(word)
        line = fp.readline()
    fp.close()
    return stopWords
#end

#start getfeatureVector
#start getfeatureVector
def getFeatureVector(tweet):
    featureVector = []
    #split tweet into words
    words = tweet.split()
    for w in words:
        #replace two or more with two occurrences
        w = replaceTwoOrMore(w)
        #strip punctuation
        w = w.strip('\'"?,.')
        #check if the word stats with an alphabet
        val = re.search(r"^[a-zA-Z][a-zA-Z0-9]*$", w)
        #ignore if it is a stop word
        if(w in stopWords or val is None):
            continue
        else:
            featureVector.append(w.lower())
    return featureVector
#end

#Read the tweets one by one and process it
fp = open('data/sampleTweets.txt', 'r')
line = fp.readline()

st = open('data/feature_list/stopwords.txt', 'r')
stopWords = getStopWordList('data/feature_list/stopwords.txt')

while line:
    processedTweet = processTweet(line)
    featureVector = getFeatureVector(processedTweet)
    with open('data/niek_corpus_feature_vector.txt', 'w') as f:
        f.write(', '.join(featureVector))
#end loop
fp.close()

更新：尝试按照以下建议更改循环后：

st = open('data/feature_list/stopwords.txt', 'r')
stopWords = getStopWordList('data/feature_list/stopwords.txt')

with open('data/sampleTweets.txt', 'r') as fp:
    for line in fp:
        processedTweet = processTweet(line)
        featureVector = getFeatureVector(processedTweet)
        with open('data/niek_corpus_feature_vector.txt', 'w') as f:
            f.write(', '.join(featureVector))
fp.close()

我得到了以下输出，这只是推文最后一行的字样。

bloodwork, arm, hurts

我仍然想弄明白。

Answer 1

line = fp.readline()

只读取文件中的一行。然后，您可以在while中处理该行，然后立即退出。您需要读取文件中的每一行。读完整个文件后，您应该按照已经完成的方式处理每一行。

lines = fp.readlines()

# Now process each line

for line in lines:
  # Now process the line as you do in your original code
  while line:
    processedTweet = processTweet(line)

Python File readlines() Method

方法readlines()使用readline()读取EOF 并返回包含这些行的列表。如果是可选的sizehint 论证存在，而不是读到EOF，整行总计近似sizehint字节（可能在四舍五入后读取内部缓冲区大小。

以下是readlines（）方法的语法：

fileObject.readlines( sizehint ); Parameters sizehint -- This is the number of bytes to be read from the file.

Return Value: This method returns a list containing the lines.

示例以下示例显示了readlines()方法的用法。

 #!/usr/bin/python

 # Open a file 
fo = open("foo.txt", "rw+") print "Name of the file: ", fo.name

 # Assuming file has following 5 lines
 # This is 1st line
 # This is 2nd line
 # This is 3rd line
 # This is 4th line
 # This is 5th line

line = fo.readlines() print "Read Line: %s" % (line)

line = fo.readlines(2) print "Read Line: %s" % (line)

# Close opend file 

fo.close()

让我们编译并运行上面的程序，这将产生以下结果：

 Name of the file:  foo.txt Read Line: ['This is 1st line\n', 'This is
 2nd line\n', 
             'This is 3rd line\n', 'This is 4th line\n', 
             'This is 5th line\n'] 
Read Line: []

Answer 2

如果您只想使用readline（）而不是readlines，请按如下方式使用循环。

st = open('data/feature_list/stopwords.txt', 'r')
stopWords = getStopWordList('data/feature_list/stopwords.txt')
with open('data/sampleTweets.txt', 'r') as fp:
    for line in fp:
        processedTweet = processTweet(line)
        featureVector = getFeatureVector(processedTweet)
        with open('data/niek_corpus_feature_vector.txt', 'ab') as f:
            f.write(', '.join(featureVector))

在PYTHON中读取行，列表处理和写入文件

2 个答案: