Question

我的ruby应用程序使用大约97％的CPU，最终会被杀死。程序正在从文件夹中读取文件，如果数据库中存在文件名，它会跳过它并检查另一个文件。执行此过程时，应用程序通常会被杀死。

COMMAND      %CPU
ruby         96.5

即使我插入几乎所有文件并尝试再次午餐（因为它被杀死），系统往往会更快地杀死它。如何减少％CPU？

task :process_data, [:data_directory] => :environment do |_task, args|
    # add data to a database
    saver = CsvToSqlSaver.new
    saver.fill_files_names

    Dir.foreach(args.data_directory) do |filename|
      # if not present in records already we read it
      Base.logger.info "> Found #{filename}."
      next if saver.files_names.to_s.include?(filename) ||
              !filename.include?('csv')
      Base.logger.info "> Reading #{filename}."

      begin
        saver.generate_db_rows_from_csv_file(args.data_directory, filename)
      # handle Malformed .csv exception
      rescue CSV::ArgumentError, CSV::MalformedCSVError => e
       Base.logger.info e.message
        next
      end # we continue csv file loop?

      unless integrator.insert_data_to_database
       Base.logger.info '> No new data saved.'
      end
    end
end

这是fill_files_names：

 def fill_files_names
      @files_names = []
      files_names = MyFilesTable.select(:filename).distinct
      files_names.each do |row|
        @files_names.push(row[:filename])
      end
 end

这是基地：

class Base
    class << self
      attr_accessor :logger
    end
    @logger ||= Logger.new(STDERR)
end

这是generate_db_rows_from_csv_file

def generate_db_rows_from_csv_file(directory, filename)
      @incoming_data = []

      CSV.foreach("#{directory}/#{filename}",
                  headers: true, quote_char: "\x00") do |csv_record|
        # if invalid record, go further
        next if record_invalid?(csv_record)

        generate_row_in_the_database(csv_record, filename)
      end
end

Ruby任务使用97％的CPU

0 个答案: