list index out of range error for large data set

时间:2018-03-25 18:59:51

标签: python-3.x

I'm trying to implement sentiment analysis model for analysing data. The code below is working for dataset below 50, but there are 10000+ of data in the CSV file.

I have googled a lot but still couldn't find any solution.

Here's the code:

import re
import csv
fh=open("dataset.csv","r")
dataset={}
no_of_items={}
for row in reader:
     no_of_items.setdefault(row[1],0)
     no_of_items[row[1]]+=1
     dataset.setdefault(row[1],{})
     split_data=re.split('[^a-zA-Z\']',row[0])

for i in split_data:

    if len(i) > 2:          
        dataset[row[1]].setdefault(i.lower(),0)         
        dataset[row[1]][i.lower()]+=1           
        feature_set.setdefault(i.lower(),{})            
        feature_set[i.lower()].setdefault(row[1],0)         
        feature_set[i.lower()][row[1]]+=1

Output:

no_of_items.setdefault(row[1],0) IndexError: list index out of range

0 个答案:

没有答案