Question

反正我可以优化读取此文件吗？

这是一个Laravel应用程序。 data_fill()是Laravel的特定功能。这只是我的php数组构建逻辑。

https://github.com/laravel/framework/blob/3414dcfcbe27cf0f4deee0670f022983e8016392/src/Illuminate/Support/helpers.php#L427

我拿了15万行，16列文件，并将其切碎成更小的4261行样本，大约需要10到15秒。完整的文件需要几分钟的时间（必须对此进行大量修改nginx和php）。

这是较小文件-https://blackfire.io/profiles/c4087f40-dd5c-42ed-9258-3c6d5a1ace51/graph

的blackfire.io报告

看起来它正在较小的文件349380次（较深的红色框中）多次读取所有68176个单元格（4261行* 16列）。

已附加电子表格。

spreadsheets.zip

这是我用来处理文件的代码。我已经硬编码了一些参数进行测试。通常，这些将是请求变量，以允许根据标题行选择不同的列。

public function report02()
    {
        ini_set('max_execution_time', '400');

        // Time how long script takes to run
        $executionStartTime = microtime(true);

        $reader = ReaderFactory::create(Type::XLSX); // for XLSX files

        $reader->open(storage_path('smaller.xlsx'));

        $headers = [];
        $header_tech = strtoupper('INSTALLER NBR');
        $header_tech_index = 0;
        $header_equipment_type = strtoupper('ITEM');
        $header_equipment_type_index = 0;
        $header_equipment_sn = strtoupper('SERIAL NUMBER');
        $header_equipment_sn_index = 0;
        $header_equipment_status = strtoupper('EQUIP STS');
        $header_equipment_status_index = 0;
        $header_equipment_age = strtoupper('DAYS ASSIGNED');
        $header_equipment_age_index = 0;
        $show_old_equipment_limit_in_days = 21;
        // convert comma delimitied tech numbers to an array and trim white space
        $techs = array_map('trim', explode(",", '9448,69091,69165,69327,69430,69445,69449,69711,70056'));
        $data = [
            'old_equipment' => [
                'total' => 0
            ]
        ];

        foreach ($reader->getSheetIterator() as $sheet) {
            foreach ($sheet->getRowIterator() as $row => $values) {
                // Check for last row - end of file
                $values2 = array_map('trim', $values);
                if (! empty($values2)) {
                    // Header row position
                    if ($row == 2) {
                        $headers = $values2;
                        $header_tech_index = array_search($header_tech, $headers);
                        $header_equipment_type_index = array_search($header_equipment_type, $headers);
                        $header_equipment_sn_index = array_search($header_equipment_sn, $headers);
                        $header_equipment_status_index = array_search($header_equipment_status, $headers);
                        $header_equipment_age_index = array_search($header_equipment_age, $headers);
                    }

                    // Data after header
                    if ($row > 2) {
                        // This row contains data for a tech we requested
                        if (in_array($values2[$header_tech_index], $techs)) {
                            // 7 Status
                            if ($values2[$header_equipment_status_index] == 7) {
                                if (array_has($data, 'techs.' . $values2[$header_tech_index] . '.counts.7 Status.' . $values2[$header_equipment_type_index])) {
                                    $data['techs'][$values2[$header_tech_index]]['counts']['7 Status'][$values2[$header_equipment_type_index]]++;
                                } else {
                                    data_fill($data, 'techs.' . $values2[$header_tech_index] . '.counts.7 Status.' . $values2[$header_equipment_type_index], 1);
                                }

                                // All tech equipment
                                data_fill($data, 'techs.' . $values2[$header_tech_index] . '.equipment.7 Status.' . $values2[$header_equipment_sn_index], [
                                    'type' => $values2[$header_equipment_type_index],
                                    'age' => $values2[$header_equipment_age_index],
                                ]);
                            }

                            // T Status
                            if ($values2[$header_equipment_status_index] == 'T') {
                                if (array_has($data, 'techs.' . $values2[$header_tech_index] . '.counts.T Status.' . $values2[$header_equipment_type_index])) {
                                    $data['techs'][$values2[$header_tech_index]]['counts']['T Status'][$values2[$header_equipment_type_index]]++;
                                } else {
                                    data_fill($data, 'techs.' . $values2[$header_tech_index] . '.counts.T Status.' . $values2[$header_equipment_type_index], 1);
                                }

                                // All tech equipment
                                data_fill($data, 'techs.' . $values2[$header_tech_index] . '.equipment.T Status.' . $values2[$header_equipment_sn_index], [
                                    'type' => $values2[$header_equipment_type_index],
                                    'age' => $values2[$header_equipment_age_index],
                                ]);
                            }

                            // X Days or older
                            if ($values2[$header_equipment_age_index] >= $show_old_equipment_limit_in_days && ($values2[$header_equipment_status_index] !== 'L')) {
                                data_fill($data, 'old_equipment.techs.' . $values2[$header_tech_index] . '.' . $values2[$header_equipment_sn_index], [
                                        'type' => $values2[$header_equipment_type_index],
                                        'status' => $values2[$header_equipment_status_index],
                                        'age' => $values2[$header_equipment_age_index],
                                    ]);

                                $data['old_equipment']['total']++;
                            }
                        }
                    }
                }
                unset($values2);
            }
        }

        $reader->close();

        $executionEndTime = microtime(true);
        $data['runtime_in_seconds'] = round($executionEndTime - $executionStartTime, 2);

        return response()->json($data, 200);
    }

较小文件输出的屏幕截图

还创建了一个github问题请求，但它可能不是软件包问题，因为它是一个实现-https://github.com/box/spout/issues/585。

感谢我收到的任何帮助。

盒式出水嘴-处理大量Xlsx Excel工作表崩溃

0 个答案: