Question

I am working with laravel project which uses a mysql database.It has some tables which has over 5 million data.it takes too much time to get these data to frontend. following is the function i use to get data. i have some filters used such as date range (from,to) search by name($seach) pagination amount(records_number) etc. but when i try to get all records it takes too much time. is there any solutions/optimizations for this matter?

Thanks.

public function transactionListBetween($from, $to, $sort, $search, $records_number, $filter, $previous_sort) {

    $query = $this->accountTransactions
        ->with('transactionType', 'giver', 'recipient')
        ->leftJoin('tbdb_users as recipient', 'recipient.id', '=', 'tbdb_account_transaction.recipient_id')
        ->leftJoin('tbdb_users as giver', 'giver.id', '=', 'tbdb_account_transaction.giver_id')
        ->leftJoin('tbdb_account_transaction_type', 'tbdb_account_transaction.account_transaction_type_id', '=', 'tbdb_account_transaction_type.id')
        ->where('recipient.name', 'like', "%$search%")
        ->select('tbdb_account_transaction.*');

    if($filter) {
        if($filter == -1) {
            $transactionTypeModel = \App::make('App\Models\AccountTransactionTypeModel');
            $depositTypeIds = $transactionTypeModel->whereIn('name', ['PayPal Deposit', 'Eway Deposit', 'Bank Deposit', 'BPay Deposit', 'Poli Deposit'])->lists('id');
            $query = $query->whereIn('tbdb_account_transaction.account_transaction_type_id', $depositTypeIds);
        } else {
            $query = $query->where('tbdb_account_transaction.account_transaction_type_id', $filter);
        }
    }

    if($from) {
        $query = $query->where('tbdb_account_transaction.created_date', '>=', $from);
    }

    if($to) {
        $query = $query->where('tbdb_account_transaction.created_date', '<=', $to->endOfDay());
    }

    if($records_number == 'no_paginate') {
        return $query->orderBy($sort, $previous_sort)
            ->get();
    } else {
        $totalAmount = $query->sum('amount');
        $collection = $query->orderBy($sort, $previous_sort)->paginate($records_number);
        $collection->totalAmount = number_format($totalAmount / 100, 2);
        return $collection;
    }

}

Answer 1

您似乎要进行3个左联接，并获得与with相同的数据。

您可以放下它们：

public function transactionListBetween($from, $to, $sort, $search, $records_number, $filter, $previous_sort) {
    $query = $this->accountTransactions->->with('transactionType', 'giver', 'recipient')
                ->whereHas('recipient', function ($query) use ($search) { //Query the related model
                     $query->where('name', 'LIKE', "%$search%"); //Note this will not use an index
                 });

    if ($filter) {
       if ($filter == -1) {            
             $query->whereHas('transactionType', function ($query) {
                 $query->whereIn('id', \DB::raw("(SELECT id FROM tbdb_account_transaction_type WHERE name IN ('PayPal Deposit', 'Eway Deposit', 'Bank Deposit', 'BPay Deposit', 'Poli Deposit')");
             });
       } else {
            $query->whereHas('transactionType', function ($query) use ($filter) {
               $query->where('id', $filter);
            });
       }

    }


    if($from) {
        $query = $query->where('created_date', '>=', $from);
    }

    if($to) {
        $query = $query->where('created_date', '<=', $to->endOfDay());
    }

    if($records_number == 'no_paginate') {
        return $query->orderBy($sort, $previous_sort)
            ->get();
    } else {
        $totalAmount = $query->sum('amount'); 
        $collection = $query->orderBy($sort, $previous_sort)->paginate($records_number);
        $collection->totalAmount = number_format($totalAmount / 100, 2);
        return $collection;
    }

 }

这将大大减少主要结果集，使其更易于使用。

Answer 2

就我而言，使用mysql数据库时我也遇到了这个问题。与关系表中的这些很多记录花费太多时间。您必须使用原始查询来获取结果。您必须使用分页来显示数据。因为获取所有数据会减慢速度。

但是我强烈建议您规范化您的表结构。就我而言，我确实通过对表进行年度归一化来对表进行归一化。

Answer 3

找出原因。这是因为分页。分页需要很多时间。有超过50000页，因此需要时间。我使用了simplePaginate() instead of paginate()。然后加载速度比平常快

laravel retrieving data from database takes too much time

3 个答案: