当乘客收到多个请求时,Sidekiq被杀

时间:2017-08-30 07:49:53

标签: ruby postgresql nginx passenger sidekiq

我有一个rails应用程序,它部署在带有nginx和passenger的ec2实例上。对于后台工作,我使用了redki的sidekiq。我在服务器上有1 GB的内存。我还在nginx配置中为乘客设置了max_pool_size为1,并将sidekiq并发设置为1.

当乘客获得多个请求时,它会获取最大内存,而sidekiq因内存不足而被杀死。

Passenger和sidekiq在理想位置占用大约54%的内存,并且在完成此过程后没有释放内存。在理想位置的日志下面:

public class SparkDataSets {
private static SparkConf conf;
private static JavaSparkContext sc;
private static SQLContext sqlsc;
private static void run(String path) {
    JavaRDD<Employee> rdd = sc.textFile(path).map((t) -> {
        String[] values = t.split(",", -1);
        Employee emp = new Employee();
        SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-yy");
        Date startdate = dateFormat.parse(values[1]);
        Date Enddate = dateFormat.parse(values[5]);
        long start = startdate.getTime();
        long end = Enddate.getTime();
        emp.setId(Integer.parseInt(values[0]));
        emp.setStart_Date(new Timestamp(start));
        emp.setFirstName(values[2]);
        emp.setLastName(values[3]);
        emp.setGender(values[4]);
        emp.setEndDate(new Timestamp(end));
        emp.setDepartMent_id(values[6]);
        return emp;
    });
    Dataset<Employee> set = sqlsc.createDataset(rdd.rdd(),Encoders.bean(Employee.class));
    set.createOrReplaceTempView("Employee");
    Dataset<Row> filterset = sqlsc.sql("select * from Employee where lastName='Facello'");
    filterset.show();
    Dataset<Employee> bean = sqlsc.createDataset(rdd.rdd(),Encoders.bean(Employee.class));
    bean.show();
}
public static void main(String... args) {
    conf = new SparkConf().setMaster("local").setAppName("GroupBy");
    sc = new JavaSparkContext(conf);
    sqlsc = new SQLContext(sc);
    run("data/input/Employee.txt");
}

}

当乘客和sidekiq收到多个请求时,由于内存不足导致sidekiq被杀。

ps aux --sort=-%mem | awk 'NR<=10{print $0}'
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
ubuntu   13975  1.5 24.8 774064 251908 ?       Sl   11:04   0:01 Passenger RubyApp: /var/www/dairylac/Quickmilk/current/public (staging)
ubuntu   14108  5.1 21.4 914332 217596 ?       Sl   11:04   0:03 sidekiq 4.1.2 Quickmilk [0 of 1 busy]
ubuntu   13858  3.8 19.9 491072 202544 ?       Sl   11:03   0:04 Passenger AppPreloader: /var/www/dairylac/Quickmilk/current
postgres 13986  0.2  4.4 192528 45048 ?        Ss   11:04   0:00 postgres: enbake quickmilk_dev_one 127.0.0.1(32902) idle
postgres  1489  0.0  3.5 185752 36044 ?        Ss   Aug29   0:06 postgres: writer process   
ubuntu   13952  0.0  1.9  42764 19376 ?        Ssl  11:04   0:00 /var/www/dairylac/Quickmilk/shared/bundle/ruby/2.1.0/gems/appsignal-1.2.5/ext/appsignal-agent
root     13811  0.1  1.6 677924 16744 ?        Sl   11:03   0:00 Passenger core
postgres 14117  0.0  1.4 187736 14316 ?        Ss   11:04   0:00 postgres: enbake quickmilk_dev_one 127.0.0.1(32924) idle
postgres 13959  0.0  1.4 187736 14308 ?        Ss   11:04   0:00 postgres: enbake quickmilk_dev_one 127.0.0.1(32900) idle

我没有找到为什么乘客和sidekiq被杀,我该怎么解决。

请给我任何建议。

提前致谢。

0 个答案:

没有答案