从网址复制的文件被截断

时间:2019-01-31 02:28:36

标签: php file copy php-stream-wrappers

复制文件时遇到问题。我的代码:

$file = "https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml";

$newfile = $_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';

if(copy($file, $newfile)) {
    echo "salvato<br>";
} else {
    echo "ERROR inport file PO_Offerte_".$data.".".$ext."<br>";
    die;
}

copy()是正确的,已创建文件,但是缺少文件末尾的某些行...文件大小为3.6MB,缺少文件末尾的0.3 ...

如果我手动下载文件,一切都很好,因此源代码是完整的...

如果我用file_get_contents()获取文件内容,并尝试使用文件写入功能将其保存到文件中,我实际上会遇到同样的问题...

我认为upload_max_filesize中实际上没有涉及post_max_sizecopy(),但已将它们设置为20MB

任何提示?

谢谢

8 个答案:

答案 0 :(得分:7)

我能够使用file_get_contents 并强制使用HTTP / 1.1协议

$context = stream_context_create([
    'http' => [
      'protocol_version' => '1.1',
      'header' => 'Connection: Close'
    ],
]);
$content = file_get_contents('https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml', false, $context);
file_put_contents('document.xml', $content);

话虽如此,我建议使用CURL:

$ch = curl_init();
$curlopts = array();
$curlopts[CURLOPT_RETURNTRANSFER] = true;
$curlopts[CURLOPT_VERBOSE] = true;
$curlopts[CURLOPT_URL] = 'https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';
curl_setopt_array($ch, $curlopts);
$content = curl_exec($ch);
file_put_contents('document.xml', $content);
curl_close($ch);

答案 1 :(得分:0)

您有一个问题,因为您使用HTTPS协议。 copy()file_get_content()函数在使用HTTPS时存在一些问题。 加载远程文件的更可靠方法是使用CURL。例如:

$url = 'https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';    
$filePath = $_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';

$ch = curl_init($url);
$fp = fopen($filePath, 'wb');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);

答案 2 :(得分:0)

  

请尝试使用此连接超时功能,希望这个东西对您有用。

$url=curl_init("https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml"); 
$set_timeout=400; 
$newfile=$_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';
curl_setopt($url, CURLOPT_CONNECTTIMEOUT, $set_timeout); 
$content = curl_exec($url);//execute request
if($content)
{
    $copied_file = fopen($newfile, "w"); 
    if(fwrite($copied_file , $content)){
        echo "Done successfully";
    }
    else{
        echo "unable to write file";
        fclose($copied_file ); 
    }
}
else {
    echo "Something Went wrong";
}

答案 3 :(得分:0)

看起来像memory_limit。尝试在脚本开始时将其设置为更高的值。

ini_set('memory_limit' '1024m')

答案 4 :(得分:0)

问题出在Http 1.0上

这应该解决它:

$sc = stream_context_create(['http' => ['protocol_version' => '1.1']]);

copy($file, $newfile, $sc);

答案 5 :(得分:0)

由于该URL已启用SSL,因此通常需要打包其他信息并与请求一起发送-对于copy函数,有一个context参数。 context允许您指定方法,协议以及更多内容来支持请求。

$url='https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';

/* download a copy from: https://curl.haxx.se/ca/cacert.pem */
$cacert=__DIR__ . DIRECTORY_SEPARATOR . 'cacert.pem';
/* define where files are to be stored */
$dir='c:/temp/downloads/';

使用stream_context_create()函数为请求建立上下文,并使用copy实际发出请求。

$filepath=$dir . basename( $url );

$args = array(
    'http'  =>  array( 'method' => 'GET', 'protocol_version' => '1.1' ),
    'ssl'   =>  array( 'verify_peer' => true, 'verify_peer_name' => true, 'allow_self_signed' => false, 'cafile' => $cacert )
);
$ctxt = stream_context_create( $args );
$status = copy( $url, $filepath, $ctxt );

if( $status && file_exists( $filepath ) ){
    printf( 
        'The file "%s" downloaded successfully. %sMb written to disk.',
        $filepath,
        round( filesize( $filepath ) / pow( 1024, 2 ),2 )
    );
}

另一个也许是更好的选择是卷曲:

function downloadfile( $url=false, $dir=false, $cacert=false ){
    if( $url && $dir ){

        /* define the save path */
        $filepath = $dir . basename( $url );

        /* time how long the download takes */
        $start=time();

        /* open a file pointer for use by curl */
        $fp = fopen( $filepath, 'w+' );

        /* create the curl request - write file directly */
        $ch = curl_init( $url );
        curl_setopt($ch, CURLOPT_HEADER, 0 );
        curl_setopt($ch, CURLOPT_TIMEOUT, 10 );
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
        curl_setopt($ch, CURLOPT_BINARYTRANSFER, true );
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true );
        curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36' );
        if( parse_url( $url, PHP_URL_SCHEME )=='https' ){
            curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true );
            curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2 );
            curl_setopt($ch, CURLOPT_CAINFO, $cacert );
        }
        curl_setopt($ch, CURLOPT_ENCODING, '' );
        curl_setopt($ch, CURLOPT_FILE, $fp );# write to file

        /* the response */
        $obj=(object)array(
            'response'  =>  curl_exec($ch),
            'info'      =>  (object)curl_getinfo($ch),
            'error'     =>  curl_error($ch),
            'filepath'  =>  $filepath
        );

        /* tidy up */
        curl_close($ch);
        fclose($fp);

        /* calculate time operation took */
        $obj->duration=round( time() - $start, 2 );

        return $obj;
    }
}



/* run the function */
$obj = downloadfile( $url, $dir, $cacert );

if( $obj->info->http_code==200 ){
    printf( 
        'The file "%s" downloaded successfully in approximately %ss. %sMb written to disk.',
        $obj->filepath,
        $obj->duration,
        round( filesize( $obj->filepath ) / pow( 1024, 2 ),2 )
    );
} else {
    printf(
        'Error: A problem was encountered downloading %s. The response code is: %d and error message: "%s"',
        $url,
        $obj->info->http_code,
        $obj->error
    );
}

以上两种方法以及手动下载的文件均为3.19Mb

答案 6 :(得分:-1)

看看您的php.ini。 可能您必须增加这两个参数

upload_max_filesizepost_max_size

答案 7 :(得分:-1)

http://php.net/manual/en/function.copy.php

我看到了:exec(“ xcopy $ source $ destination”);

尝试一下!