我使用SVM分类器将文本分类为好文本和乱码。我正在使用python的scikit-learn并按如下方式进行操作：
'''
Created on May 5, 2017
'''
import re
import random
import numpy as np
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.linear_model import SGDClassifier
from sklearn import metrics
# Prepare data
def prepare_data(data):
"""
data is expected to be a list of tuples of category and texts.
Returns a tuple of a list of lables and a list of texts
"""
random.shuffle(data)
return zip(*data)
# Format training data
training_data = [
("good", "rain a lot the packs maybe damage."),
("good", "15107 Lane Pflugerville, TX customer called me and his phone number and my phone numbers were not masked. thank you customer has had a stroke and items were missing from his delivery the cleaning supplies for his wet vacuum steam cleaner. he needs a call back from customer support "),
("gibber", "wh. screen"),
("gibber", "How will I know if I"),
("good", "I have problems scheduling blocks they are never any available. Can I do full time? Can I get scheduled more than one day a month?"),
("good", "Suggestion: easier way to sign in due alleviate the tediousness of periodically having to sign back in to the app to check for blocks."),
("good", "I am so glad to hear from you. "),
("good", "loading on today's itinerary takes ages!!!!!! time consuming when you have 150+ packages to deliver!!!!!"),
("good", "due to the new update that makes hours available at 10 pm. if you worked 8 hours that day you can't see next day hours due to 8 hour limit. please fix this"),
("good", "omg, PLEASE make it so we don't have to sign in every time we need to go into the app. At least make it good for a week. Thanks."),
("good", "Constantly being logged out of app, if we could have a continuous login so we could receive notifications if blocks are available that would be ideal."),
("good", "I am having problems with the App. Every time I exit the App and reopen it asks for my login info."),
("good", "15 minute service time due to 33rd floor and 20l lbs of cargo"),
("good", "I have been sceduled 1 block in 3 weeks. I check for new block availability multiple times a day and have not seen 1 available in three weeks. is there any way to get more blocks."),
("good", "When will delivery jobs be available? Everytime I open this app, it says nothing is available. Have deliveries in Cincinnati started yet?"),
("good", "During delivery had to call customer support and after 10 minutes support person couldn't find my pick up location Kirkland /Bellevue and told me to hang up and call different support team. Support person were unprofessional and rude, which is not acceptable."),
("good", "can you please remove the pick up from my phone"),
("good", "Dear friends: I'm very very happy it's a big oportunitt"),
("good", "THANK YOU so much for the block you assigned me for next week. If you have an additional 5 blocks please go ahead and assign them to me for next week. My availability is updated and current. You guys are awesome!!!"),
("good", "after update every time I open app I have too log in! I used to be able to stay logged in unless I logged out, can you return stay logged in option."),
("good", "It looks like my app is not installed properly on my android phone, Note 5. I cannot access or do not see the tab to swipe to start delivering and the map or help button that should be visible for me to work today 5/6 at rpm"),
("gibber", "AF0000"),
("good", "awesome app, awesome hiring process, awesome delivery warehouse , awesome team and help in the field! lets deliver I would like more more more delivers , looking forward to the future ! I just bought a new delivery vehical !"),
("good", "I will like to ask why I can't get more delivery's only one in two weeks"),
("good", "device too slow software crashing all day"),
("good", "it doesn't work sometimes."),
("good", "can you please remove the old sprouts pick up from my phone"),
("good", "They ability to zoom in on text screens would be very helpful. Am example would be customer notes when viewing in certain lighting conditions can be difficult."),
("good", "I missed out on a delivery day when I clicked check in and waited for my turn to get an order only to find out that not only did my check in not register but the gps showed me down the street. I encountered this issue again when one of the warehouse employees placed an order for that location and the app wanted me to drive in a big circle to get back to where I was standing."),
("good", "i am a little concerned that i didn't receive any blocks of time for this coming week, even though i had a perfect delivery score from this past pay period. Did the Cincinnati market over hire drivers where there are many people being shut completely out of any delivery blocks for an entire week? i really enjoy this type of work and the app makes it quite convenient."),
("good", "I've arrived at the pick up restaurant but the staff did not have the barecode for me to scan, however I pick up the package and deliver but my is still not let me move on"),
("good", "might want to check my assigned hours for next week. 5am to 1pm??"),
("good", "hi team--just want to give some positive feedback. I have had nothing but positive feedback from customers. Great support when calling help line. Thank you for this opportunity and if there is ever a situation where you need drivers immediately I will drop what I'm doing and help. You guys are the best."),
("good", "Allow days or blocks throughout the day to be modified after General availability is set up for time off like doctors appointments."),
("gibber", "AL001234"),
("good", "Please, enlight me."),
("good", "it only shows my schedule starting in two weeks. when will we be able to start work"),
("good", "include more packages for one block, if the packages can be fitted into the car, so driver don't have to come back and pickup every two hours. 25% of the time is wasted coming back for pick up."),
("gibber", "BBB h"),
("gibber", "AG0003006033SDgCJ12344"),
("gibber", "How will I know if I"),
("good", "please bring back some sort of hours cap! or possibly stagger the hour drops from 1200 to 1203 so that people with slower internet/slower phone arent at a disadvantage!"),
("good", "when the hours released tonight all of the people who didn't have 40 hours could see them. however the drivers that are capped at 40 were unable to see them due to a flawed system. please fix the system so that we are not continually treated unfairly like all of the drivers that whined so much and got us in to this mess. the cap system is unfair to people that want to work and it caused problems with a lack of drivers to deliver today at the hub. obviously this is not a good system and benefits no one."),
("good", "You have seriously messed up the whole scheduling process. Why can't I get any blocks at 10 even if I wait exactly until 10? Midnight was much better. So now that scheduling is a huge random pain in the ass, why would people want to keep doing this? I haven't been able to schedule work for three days now, it's quite frustrating when I don't get a chance to sign up, even when I'm diligent with timing."),
("good", "Seriously, that's all I'm going to get is one lousy day? Tell me again why you need drivers if all we get is one day. I'm not sure this is gonna work out for me. I waited forever to get my background check back and this is what I get? smh"),
("good", "doesn't save updated access codes"),
("good", "the scheduling of my route is nor done very accurately. it keeps me driving back and forth"),
("good", "can't understand how to pick up a block. my availability is wide open. when you guys send the alerts about blocks available I open it real quick and there is nothing there. I do it in a matter of seconds"),
("good", "My availability keeps disappearing from my calender. I set my availability for three weeks in advance. The gray dots are visible but disappear on Wednesday or Thursday. This makes it impossible for me to see and choose available blocks for the upcoming week. How can I get it fix. Mike"),
("good", "GPS blank screen"),
("gibber", "sea swq"),
("gibber", "hiw o"),
("gibber", "Dr a"),
("gibber", "quick to quick to u uhu wu just us"),
("gibber", "Awa what's"),
("gibber", "wxdfcs"),
("gibber", "7k9opu"),
("gibber", "o.m.day day"),
("gibber", "GGT part his h"),
("gibber", "aawfhg"),
("gibber", "seesaw 2s"),
("gibber", "wawaa"),
("gibber", "of ll"),
("gibber", "rewards"),
("gibber", "mmqqm5my"),
("gibber", ".in w"),
("gibber", "play r"),
("gibber", "was wwnw www www n"),
("gibber", "wqq2fwqq2fz22"),
("gibber", "not"),
("gibber", "I by yu I"),
("gibber", "Hi just wanted to let you know that it's bee"),
("gibber", "I erroneously v"),
("gibber", "I find it"),
("gibber", "bqyyx I a"),
("gibber", "are are"),
("gibber", "wawi waarnnnkwn"),
("gibber", "t Petey ueteu he"),
("gibber", "ews ri"),
("gibber", "bd xd"),
("gibber", "hatpa"),
("gibber", "se wests tasgt"),
("gibber", "wa vgcx azc Jo of"),
("gibber", "2w222"),
("gibber", "her u t b"),
("gibber", "ddddedc"),
("gibber", "just juju in hiking"),
("gibber", "wew2ww2wwwew2i2wkkk"),
("gibber", "meleeee"),
("gibber", "Aaq wqXD"),
]
training_labels, training_texts = prepare_data(training_data)
# Format test data
test_data = [
("gibber", "an quality"),
("good", "Can't check in. Time was 4:06. I didn't drive out here for no reason."),
("good", "can you do view all full address including postal code how it's in old app that helps do correctly delivery and not waist customer time"),
("good", "i am available again starting at 10am to 10pm. thanks"),
("gibber", "Hello, I encountered"),
("good", "I want to know how we are notified if there is a block I have been signed in and haven't been given a block yet"),
("gibber", "aawaaw"),
("gibber", "eeeeeeeeene"),
("good", "I am not getting enough shifts"),
("gibber", "hey e75k"),
("good", "my screen had went black or inverted"),
("good", "maps packed up again in sr20ls"),
("good", "how to clear my itinerary from old pickup address ?"),
("good", "keep signing me out."),
("good", "For alcohol delivery, where does customer sign?"),
("gibber", "t Petey ueteu he"),
("good", "can't get blocks. too many drivers ??"),
("good", "got a new phone how do i download to new phone")
]
test_labels, test_texts = prepare_data(test_data)
# Create feature vectors
"""
Convert a collection of text documents to a matrix of token counts.
See: http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html
"""
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(training_texts)
y = training_labels
# Train the classifier
clf = SGDClassifier()
clf.fit(X, y)
# Test performance
X_test = vectorizer.transform(test_texts)
y_test = test_labels
# Generates a list of labels corresponding to the samples
test_predictions = clf.predict(X_test)
# Convert back to the usual format
annotated_test_data = list(zip(test_predictions, test_texts))
print(annotated_test_data)
# evaluate predictions
y_test = np.array(test_labels)
print(metrics.classification_report(y_test, test_predictions))
print("Accuracy: %0.4f" % metrics.accuracy_score(y_test, test_predictions))
但是，每次运行时我都会保持不同的准确度。为什么会这样？
更新： 所以我将training_data移动到一个文本文件中，我在上面的代码中读取它：
lines = [line.rstrip('\n') for line in open("file.txt")]
training_data=[]
for i in lines:
result = i.rstrip(',')
l = literal_eval(result)
training_data.append(l)
training_labels, training_texts = prepare_data(training_data)
我在上面的代码中也改变了这个：
clf = SGDClassifier(random_state=5000)
所以，现在random_state不是None。但是，我每次都会得到不同的准确度!!
答案 0 :(得分：3)
这是因为在prepare_data()方法中，您随机改组数据。这就是你在做的事情：
random.shuffle(data)
因此它会影响估算器的训练，从而影响结果。
尝试评论或删除该行以及random_state中设置的SGDClassifier。每次都会得到完全相同的结果。
建议：尝试使用不同的估算工具来查看效果最佳的估算工具。如果您热衷于使用SGDClassifier，那么我建议您查看并了解n_iter参数。尝试将其更改为更大的值，您将看到准确度的差异将变得越来越小（即使您的数据混乱）。
您可以查看此答案以获取更多详细信息：
答案 1 :(得分：1)
所有分割或混洗数据的分类器（h / t到Vivek）在构造函数中都有一个可选的random_state变量，默认值为None。传递random_state时，会由内部check_random_state函数检查。
来自文档：
check_random_state：从a创建一个np.random.RandomState对象 参数random_state。如果random_state是None或np.random，那么a 随机初始化的RandomState对象被返回。如果random_state是一个整数，然后它用于播种新的RandomState对象。如果random_state是一个RandomState对象，然后通过它。
因为您使用默认的None，所以代码中会出现一些不受控制的随机噪音。
传递种子以获得再现性。