无法在 GAE/云存储/PHP 上创建大型 zip 文件

时间:2021-06-03 16:21:48

标签: php google-app-engine google-cloud-storage ziparchive

我在 ZipArchive 和 GAE 上读过类似的帖子,但它们没有解决我的问题。将 .zip 创建到 GAE tmp 文件夹会耗尽内存,因为 zip 可能是 1G 或更多。因此,我正在尝试直接写入存储桶。

下面的输出显示了运行我的 zip 函数的结果。我对 GAE/Storage 有点陌生,所以希望更多的人能帮助我!

我注意到的一件事是 $zip->filename = /workspace/gs:/X... 为什么 "workspace" 在那里,为什么在 "/" 之后只有一个 gs:

如果有其他方法可以实现这一点,我愿意接受建议!

我在 LAMP 中有这个工作,但需要转移到 GAE。这是我的压缩函数代码:

<?php
    function create_zip($files = array(),$fileDestinations = array(),$destination='',$overwrite = false) {
        
        $valid_files = array();
        
        if(is_array($files)) {
            //cycle through each file
            foreach($files as $file) {
                //make sure the file exists OR THE ZIP WON'T BE CREATED
                if(file_exists($_SESSION['devStorage'].$file)) {
                    $valid_files[] = $_SESSION['devStorage'].$file;
                }else{
                    $valid_files[] = $_SERVER['DOCUMENT_ROOT']."/images/default.jpg";
                }
            }
        }
    
        //if we have good files...
        if(count($valid_files)) {
    
            if (is_writable(str_replace("https://storage.googleapis.com/","gs://",$_SESSION['devStorage']))) {
                echo '<br>'.str_replace("https://storage.googleapis.com/","gs://",$_SESSION['devStorage']).' Folder is writable<br>';
            } else {
                echo '<br>'.str_replace("https://storage.googleapis.com/","gs://",$_SESSION['devStorage']).' Folder not writable<br>';
            }
            
            //create the archive
            $zip = new ZipArchive();
            if($zip->open($destination, ZIPARCHIVE::CREATE | ZIPARCHIVE::OVERWRITE)!==true) {
                return "<br>can't open ".$destination;
            }
    
            //add the files
            foreach($valid_files as $file) {
                $zip->addFile($file,current($fileDestinations));
                next($fileDestinations);
            }
    
            //debug
            echo '<br>The zip archive contains ',$zip->numFiles,' files with a status of ',$zip->status;
            echo '<br>zip->filename = '.$zip->filename;
    
            //close the zip -- done!
            if($zip->close()!==true) {
                echo '<br>zip->close error';
            }
            
            //check to make sure the file exists
            if (!file_exists($destination)) {
                return "<br>File does not exist at ".$destination;
            }else{
                return true;                    
            }
            
        } else {
            return "no valid files";
        }
    }
?>

结果输出:

gs://X.appspot.com/uploads/ Folder is writable

The zip archive contains 139 files with a status of 0

zip->filename = /workspace/gs:/X.appspot.com/uploads/XSpring2021asof122920_06-03-2021_11-18-45

zip->close error

File does not exist at gs://X.appspot.com/uploads/XSpring2021asof122920_06-03-2021_11-18-45

感谢您提供的任何帮助!

1 个答案:

答案 0 :(得分:0)

最终使用了 ZipStream。我的压缩功能如下:

    /* creates a compressed zip file */
function create_zip($files = array(),$fileDestinations = array(),$destination='') {
    // Autoload the dependencies
    require 'vendor/autoload.php';

    // enable output of HTTP headers
    $options = new ZipStream\Option\Archive();
    $options->setSendHttpHeaders(true);
    $options->setFlushOutput(true);
    $options->setZeroHeader(true);
    $options->setStatFiles(true);

    // create a new zipstream object
    $zip = new ZipStream\ZipStream($destination, $options);

    $valid_files = array();
    
    if(is_array($files)) {
        //cycle through each file
        foreach($files as $file) {
            if (($_SERVER["REMOTE_ADDR"]=="127.0.0.1") || ($_SESSION["LAMP"]==1)) {
                // //make sure the file exists OR THE ZIP WON'T BE CREATED
                //if(file_exists($_SERVER["REQUEST_SCHEME"]."://".$_SERVER['HTTP_HOST'].$file)) {
                    $valid_files[] = $_SERVER["REQUEST_SCHEME"]."://".$_SERVER['HTTP_HOST'].$file;
                //}else{
                //  $valid_files[] = $_SERVER["REQUEST_SCHEME"]."://".$_SERVER['HTTP_HOST']."/images/default.jpg";
                //}
            }else{
                // //make sure the file exists OR THE ZIP WON'T BE CREATED
                //if(file_exists($file)) {
                    $valid_files[] = $file;
                //}else{
                //  $valid_files[] = $_SERVER["REQUEST_SCHEME"]."://".$_SERVER['HTTP_HOST']."/images/default.jpg";
                //}
            }
        }
    }

    //if we have good files...
    if(count($valid_files)) {

        foreach($valid_files as $url) {
            // Create Temp File
            $fp = tmpfile();
        
            // Download File
            $ch = curl_init();
            curl_setopt($ch, CURLOPT_FILE, $fp);
            curl_setopt($ch, CURLOPT_URL, $url);
            curl_exec($ch);
        
            // Force to write all data
            fflush($fp);
        
            // Rewind to the beginning, otherwise the stream is at the end from the start
            rewind($fp);
    
            // Add File
            $zip->addFileFromStream(current($fileDestinations), $fp);
            next($fileDestinations);
        
            // Close the Temp File
            curl_close($ch);
            fclose($fp);
        }
  
        // Finish ZIP
        $zip->finish();

        return true;

    }else{

        return "no valid files";

    }

}
相关问题