使用池计数检查文件时的酸洗问题

时间:2017-09-11 18:47:01

标签: python pickle

我的代码包含多个进程,用于检查文件计数和维护数据库中的记录。正在运行的代码如下:

import multiprocessing  as mp
from multiprocessing import Pool
import os
import time
import mysql.connector

"""Function to check the count of the file"""
def file_wc(fname):
        with open('/home/vaibhav/Desktop/Input_python/'+ fname) as f:
            count = sum(1 for line in f)            
        return (fname,count)    

class file_audit:

    def __init__(self):      
        """Initialising the constructor for getting the names of files 
        and refrencing the outside class function"""

        folder = '/home/vaibhav/Desktop/Input_python'
        self.fnames = (name for name in os.listdir(folder))
        self.file_wc=file_wc   

    def count_check(self):
        "Creating 4 worker threads to check the count of the file parallelly"

        pool = Pool(4)
        self.m=list(pool.map(self.file_wc, list(self.fnames),4))
        pool.close()
        pool.join()

    def database_updation(self):
        """To maintain an entry in the database with details 
        like filename and recrods present in the file"""

        self.db = mysql.connector.connect(host="localhost",user="root",password="root",database="python_showtime" )
# prepare a cursor object using cursor() method
        self.cursor = self.db.cursor()
        query_string = ("INSERT INTO python_showtime.audit_capture"
                "(name,records)"
                "VALUES(%s,%s)")
        #data_user = (name,records)
        for each in self.m:    

            self.cursor.execute(query_string, each)
            self.db.commit()
        self.cursor.close()
        start_time = time.time()
        print("My program took", time.time() - start_time, "to run")

#if __name__ == '__main__':
x=file_audit()
x.count_check()  #To check the count by sprawning multiple processes
x.database_updation() #To maintain the entry in the database

要点

现在如果我将我的函数放在类中并在构造函数部分中注释 self.file_wc = file_wc ,我得到 错误不能发泡在生成器对象。我得到了一些公平的理解,就像我们不能腌制一些物体一样,所以想要用非常简单的术语知道背景究竟发生了什么。我从here here获得了引用以使代码正常工作

0 个答案:

没有答案