在python中预处理文本后如何删除空值

时间:2019-05-18 13:52:25

标签: python text-mining preprocessor

例如 我有一条推文“ @cintya @groot @smanela https://blog ...” 并且我做了一个预处理过程,删除了链接和提及,我认为它应该丢失。 但是在CSV中,它们返回一个空值。我该如何解决? 这是我的代码

def replaceMultiple(mainString, toBeReplaces, newString):
    for elem in toBeReplaces :
        if elem in mainString :
            mainString = mainString.replace(elem, newString)
    return  mainString

with open('datalatihNegatif.csv', encoding='utf-8') as csvfile:
    readCSV = csv.reader(csvfile, delimiter=',')
    for row in readCSV:
        _word = []
        username = row[0]
        date = row[1]
        text = row[2].lower()
        text = re.sub(r'@[A-Za-z0-9_]+','',text)
        text = re.sub(r'http\S+', '',text)

        text = replaceMultiple(text, ["!","@","#","$","%","^","&","*","(",
                                      ")","_","-","+","=","{","}","[","]",
                                      "\\","/",",",".","?","<",">",":",";",
                                      "'",'"',"~","0","1","2","3","4","5","6","7","8","9"], '')
        text = text.strip()
        nltk_tokens = nltk.word_tokenize(text)
        stop_words = set(stopwords.words("indonesian"))
        stop_words_new = ['i','liked','video','an','at','ba','da','do','ka','ma','ta','uh','yg','al','eh','ha','ah','ng']
        new_stopwords_list = stop_words.union(stop_words_new)

        print(username)
        print(date)

        for word in nltk_tokens:
            if word not in new_stopwords_list:
                if stemmer.stem(word) != "":
                    _word.append(stemmer.stem(word))
        print(_word)
        csvFile = open('preprocessingDLNegatif.csv', 'a', newline='')
        csvWriter = csv.writer(csvFile)
        csvWriter.writerow(_word)
        csvFile.close()

我希望CSV中的结果被删除,但实际输出为CSV中的空值1行 481 is empty value, how can i remove it?

1 个答案:

答案 0 :(得分:0)

在将其写出之前,如何检查_word中的内容:

if len(_word) != 0:
    csvFile = open('preprocessingDLNegatif.csv', 'a', newline='')
    csvWriter = csv.writer(csvFile)
    csvWriter.writerow(_word)
    csvFile.close()

我也不会为您写入的每条记录打开一个关闭输出文件。循环前将其打开一次,完成后将其关闭。这样做会使我的答案看起来像:

if len(_word) != 0:
    csvWriter.writerow(_word)