PHP flock()替代方案

时间:2011-08-06 14:43:54

标签: php locking flock

PHP flock()的文档页面表明在IIS下使用它是不安全的。如果我在任何情况下都不能依赖flock,还有另一种方法可以安全地实现同样的目标吗?

7 个答案:

答案 0 :(得分:7)

在所有可能的情况下,没有其他方法可以安全地实现相同的目标。这是计算机系统的设计和the job is not trivial for cross-platform code

如果您需要安全使用flock(),请记录您的应用程序的要求。

或者,您可以创建自己的锁定机制,但必须确保它是原子的。这意味着,您必须测试锁定,如果它不存在,请建立锁定,同时确保没有其他任何东西可以获取锁定。

这可以通过创建表示锁的锁文件来完成,但前提是它不存在。不幸的是,PHP没有提供这样的功能来以这种方式创建文件。

或者,您可以使用mkdir()创建目录并使用结果,因为它在创建目录时将返回true,如果已存在则返回false

答案 1 :(得分:3)

您可以基于 mkdir 围绕读/写操作实现文件锁定 - 解锁模式,因为这是原子的,非常快。我已经对此进行了压力测试,并且不像mgutt那样找不到瓶颈。你必须要处理死锁情况,这可能是mgutt所经历的。死锁是指两次锁定尝试一直在等待。它可以通过锁定尝试的随机间隔来补救。像这样:

// call this always before reading or writing to your filepath in concurrent situations
function lockFile($filepath){
   clearstatcache();
   $lockname=$filepath.".lock";
   // if the lock already exists, get its age:
   $life=@filectime($lockname);
   // attempt to lock, this is the really important atomic action:
   while (!@mkdir($lockname)){
         if ($life)
            if ((time()-$life)>120){
               //release old locks
               rmdir($lockname);
               $life=false;
         }
         usleep(rand(50000,200000));//wait random time before trying again
   }
}

然后在文件路径中处理您的文件,完成后,请致电:

function unlockFile($filepath){
   $unlockname= $filepath.".lock";   
   return @rmdir($unlockname);
}

我已经选择在最长的PHP执行时间之后移除旧锁,以防脚本在解锁之前退出。更好的方法是在脚本失败时始终删除锁。有一种巧妙的方法,但我已经忘记了。

答案 2 :(得分:1)

我的建议是使用mkdir()代替flock()。这是阅读/编写缓存的真实示例,显示了差异:

$data = false;
$cache_file = 'cache/first_last123.inc';
$lock_dir = 'cache/first_last123_lock';
// read data from cache if no writing process is running
if (!file_exists($lock_dir)) {
    // we suppress error messages as the cache file exists in 99,999% of all requests
    $data = @include $cache_file;
}
// cache file not found
if ($data === false) {
    // get data from database
    $data = mysqli_fetch_assoc(mysqli_query($link, "SELECT first, last FROM users WHERE id = 123"));
    // write data to cache if no writing process is running (race condition safe)
    // we suppress E_WARNING of mkdir() because it is possible in 0,001% of all requests that the dir already exists after calling file_exists()
    if (!file_exists($lock_dir) && @mkdir($lock_dir)) {
        file_put_contents($cache_file, '<?php return ' . var_export($data, true) . '; ?' . '>')) {
        // remove lock
        rmdir($lock_dir);
    }
}

现在,我们尝试使用flock()

实现相同目标
$data = false;
$cache_file = 'cache/first_last123.inc';
// we suppress error messages as the cache file exists in 99,999% of all requests
$fp = @fopen($cache_file, "r");
// read data from cache if no writing process is running
if ($fp !== false && flock($fp, LOCK_EX | LOCK_NB)) {
    // we suppress error messages as the cache file exists in 99,999% of all requests
    $data = @include $cache_file;
    flock($fp, LOCK_UN);
}
// cache file not found
if (!is_array($data)) {
    // get data from database
    $data = mysqli_fetch_assoc(mysqli_query($link, "SELECT first, last FROM users WHERE id = 123"));
    // write data to cache if no writing process is running (race condition safe)
    $fp = fopen($cache_file, "c");
    if (flock($fp, LOCK_EX | LOCK_NB)) {
        ftruncate($fp, 0);
        fwrite($fp, '<?php return ' . var_export($data, true) . '; ?' . '>');
        flock($fp, LOCK_UN);
    }
}

重要的部分是LOCK_NB,以避免阻止所有连续请求:

  

还可以将LOCK_NB作为位掩码添加到上述之一   如果您不希望flock()在锁定时阻塞,则执行操作。

没有它,代码会产生巨大的瓶颈!

另一个重要部分是if (!is_array($data)) {。这是因为$ data可能包含:

  1. array()作为db查询的结果
  2. false失败的include
  3. 或空字符串(race condition
  4. 如果第一位访客执行此行,则会出现竞争条件:

    $fp = fopen($cache_file, "c");
    

    ,另一位访客在一毫秒后执行此行:

    if ($fp !== false && flock($fp, LOCK_EX | LOCK_NB)) {
    

    这意味着第一个访问者创建了空文件,但第二个访问者创建了锁,因此include返回一个空字符串。

    所以你看到很多陷阱都可以通过使用mkdir()来避免,而且速度也提高了7倍:

    $filename = 'index.html';
    $loops = 10000;
    $start = microtime(true);
    for ($i = 0; $i < $loops; $i++) {
        file_exists($filename);
    }
    echo __LINE__ . ': ' . round(microtime(true) - $start, 5) . PHP_EOL;
    $start = microtime(true);
    for ($i = 0; $i < $loops; $i++) {
        $fp = @fopen($filename, "r");
        flock($fp, LOCK_EX | LOCK_NB);
    }
    echo __LINE__ . ': ' . round(microtime(true) - $start, 5) . PHP_EOL;
    

    结果:

    file_exists: 0.00949
    fopen/flock: 0.06401
    

    P.S。如您所见,我在file_exists()前面使用了mkdir()。这是因为my tests(德语)仅使用mkdir()就会产生瓶颈。

答案 3 :(得分:1)

  

这是我的“ PHP flock()替代方案”-建立在mkdir()上。

使用mkdir()的想法来自herehere

我的版本

  • 检查我是否已经具有锁定访问权限。如果我为同一个basedir.name
  • 多次创建和使用该类,还可以防止阻塞自己
  • 检查是否创建了我要用来进行锁定访问的锁定文件
  • 让我按照要求的顺序获得锁定访问权限
  • 如果在我指定的时间内无法获得锁访问,则停止等待并循环播放
  • 删除无效的锁定文件(= PID的SID不再存在的文件)

您可以像这样使用PHP类:

//$dir        (string) = base-directory for the lock-files (with 'files' I mean directories => mode 0644)
// 2       (float/int) = time to wait for lock-access before returning unsuccessful (default is 0 <= try once and return)
//'.my_lock'  (string) = the way you want to name your locking-dirs (default is '.fLock')
$lock = new FileLock($dir, 2, '.my_lock');

//start lock - a locking directory will be created looking like this:
//$dir/.my_lock-1536166146.4997-22796
if ($lock->lock()) {
    //open your file - modify it - write it back
} else { /* write alert-email to admin */ }

//check if I had locked before
if ($lock->is_locked) { /* do something else with your locked file */ }

//unlock - the created dir will be removed (rmdir)
$lock->unlock();

这是工人阶级:

//build a file-locking class
define('LOCKFILE_NONE', 0);
define('LOCKFILE_LOCKED', 1);
define('LOCKFILE_ALREADY_LOCKED', 2);
define('LOCKFILE_ALREADY_LOCKED_IN_OTHER_CLASS', 3);
define('LOCKFILE_FAILED_TO_OBTAIN_LOCK', false);
define('LOCKFILE_FAILED_TO_OBTAIN_LOCK_BY_TIMEOUT', '');


class FileLock {
    //FileLock assumes that there are no other directories or files in the
    //lock-base-directory named "$name-(float)-(int)"
    //FileLock uses mkdir() to lock. Why?
    //- mkdir() is atomic, so the lock is atomic and faster then saving files.
    //  Apparently it is faster than flock(), that requires several calls to the
    //  file system.
    //- flock() depends on the system, mkdir() works everywhere.

    private static $locked_memory = array();

    public function __construct($lockbasedir, $wait_sec=0, $name='.fLock') {
        $this->lockbasedir = (string)$lockbasedir;
        $this->wait        = (float)$wait_sec;
        $this->name        = (string)$name;

        $this->pid         = (int)getmypid();

        //if this basedir.name was locked before and is still locked don't try to lock again
        $this->is_locked   = empty(self::$locked_memory[$this->lockbasedir . $this->name]) ? LOCKFILE_NONE : LOCKFILE_ALREADY_LOCKED;
    }

    public function lock() {
        if ($this->is_locked) return $this->is_locked;

        $break_time = microtime(true);

        //create the directory as lock-file NOW
        $this->lockdir = "{$this->name}-" . number_format($break_time, 4, '.', '') . "-{$this->pid}";
        @mkdir("{$this->lockbasedir}/{$this->lockdir}", 0644);

        $break_time += $this->wait;

        //try to get locked
        while ($this->wait == 0 || microtime(true) < $break_time) {

            //get all locks with $this->name
            $files = preg_grep("/^{$this->name}-\d+\.\d+-\d+$/", scandir($this->lockbasedir));

            //since scandir() is sorted asc by default
            //$first_file is the next directory to obtain lock
            $first_file = reset($files);

            if (!$first_file) {
                //no lock-files at all
                return $this->is_locked = LOCKFILE_FAILED_TO_OBTAIN_LOCK;
            } elseif ($first_file == $this->lockdir) {
                //Its me!! I'm getting locked :)
                self::$locked_memory[$this->lockbasedir . $this->name] = 1;
                return $this->is_locked = LOCKFILE_LOCKED;
            } elseif (preg_match("/^{$this->name}-\d+\.\d+-{$this->pid}$/", $first_file)) {
                //my process-ID already locked $this->name in another class before
                rmdir("{$this->lockbasedir}/{$this->lockdir}");
                $this->lockdir = $first_file;
                self::$locked_memory[$this->lockbasedir . $this->name] = 1;
                return $this->is_locked = LOCKFILE_ALREADY_LOCKED_IN_OTHER_CLASS;
            }

            //missing lock-file for this job
            if (array_search($this->lockdir, $files) === false) return LOCKFILE_FAILED_TO_OBTAIN_LOCK;

            //run only once
            if ($this->wait == 0) break;

            //check if process at first place has died
            if (!posix_getsid(explode('-', $first_file)[2])) {
                //remove dead lock
                @rmdir("{$this->lockbasedir}/$first_file");
            } else {
                //wait and try again after 0.1 seconds
                usleep(100000);
            }
        }

        return $this->is_locked = LOCKFILE_FAILED_TO_OBTAIN_LOCK_BY_TIMEOUT;
    }

    public function unlock($force=false) {
        if ($force || $this->is_locked == 1) {
            rmdir("{$this->lockbasedir}/{$this->lockdir}");
            self::$locked_memory[$this->lockbasedir . $this->name] = $this->is_locked = LOCKFILE_NONE;
        }
    }
}

答案 4 :(得分:0)

我很欣赏这个问题已有几年了,但我觉得一个有效的例子/替代羊群可能值得建立。我基于其他答案,但是对于那些纯粹想要替换flock功能的人(而不是同时写一个文件(虽然这确实反映了PHP手册flock的例子))我相信以下就足够了< / p>

function my_flock ($path,$release = false){
    if ($release){
        @rmdir($path);
    } else {
        return !file_exists($path) && @mkdir($path);
    }
}

答案 5 :(得分:0)

基于mkdir:

// call this always before reading or writing to your filepath in concurrent situations
function lockFile($filepath){
   clearstatcache();
   $lockname=$filepath.".lock";
   // if the lock already exists, get its age:
   $life=@filectime($lockname);
   // attempt to lock, this is the really important atomic action:
   while (!@mkdir($lockname)){
     if ($life)
        if ((time()-$life)>120){
           //release old locks
           rmdir($lockname);
     }else $life=@filectime($lockname);
     usleep(rand(50000,200000));//wait random time before trying again
   }
}

为避免一个脚本死锁,以防一个脚本在解锁之前退出并且同时一个(或多个)脚本在$ life = @ filectime($ lockname)上没有结果;因为所有脚本都同时启动,然后尚未创建目录。 要解锁然后致电:

function unlockFile($filepath){
   $unlockname= $filepath.".lock";   
  return @rmdir($unlockname);
}

答案 6 :(得分:0)

这些方法都不是完全原子的。

我做了一些测试,confirming this

enter image description here

enter image description here

enter image description here

T7的代码,使用7个以kB大小命名的文件:

clearstatcache();
$_DEBUG_ = false;

echo "Lock and flush tester.".time()."<br>";
$time_constant = 1570787996;
die; // Remove this line when you set time_constant 

while ( time()<$time_constant )
 {
 usleep(500);
 }


function test($n, $p, $_DEBUG_){
//  $delay_multiplier = $n*2.5;
  $sname = "$n";    // source
  $tname = "$n.txt";// target
  echo "<h4>$n at ".time()."</h4>";
  for ($i = 0; $i<50; $i++ ){
    $start = microtime(true);
    clearstatcache(); // needed for filesize and touch    
    $st = stat("$sname");
    $original_size = $st['size'];
    if ( $_DEBUG_ )
      echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
    $fsize = filesize($sname);
    if ( $original_size <> $fsize )
      die("; fsize total FAILTURE; ");
    if ($fsize === 0)
     echo "! <b>The fsize is 0</b>: stat(): ".$st['size']." ;";    
    else
      {
      // READ OPERATION AND LOCK FOR SHARE
       $locked = false;     
       for ($c = 0; !$locked; $c++):      
         if ( $c > 400)
           break;
         $fp = fopen($sname, "r");
         $locked = flock($fp, LOCK_SH);
         if ($locked)
           break;
         else
           {
           echo "failed to get LOCK_SH;<br>";
           usleep(5000);
           }
       endfor;
       $s = fread($fp, $fsize );
       $success = flock($fp, LOCK_UN);
       if ( $success === false  )
         die("; r flock release failed; ");
       $success = fclose($fp);
       if ( $success === false  )
         die("; fclose failed; ");
       // 10 - loaded data , $p - broser
       if ( $success )
         { 
         $result = touch("$sname",strlen($s),$p);
         if ( $_DEBUG_ )
            echo "; TOUCH: $result;";
         }
       else
         die("fclose FAIL.");
       if ( strlen($s)<60 ) 
          echo "*$s LENGTH:".strlen($s)."<br>";
      }
    clearstatcache();
    $st = stat("$tname");                               
    if ( $_DEBUG_ )
      echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";

    // WRITE OPERATION WITH LOC_EX
    $fp = fopen($tname, "w");
    $locked = false; 
    /*
    // TOTO NEMÁ VLIV NA ZAMKNUTÍ
    for ($c = 0; !$locked; $c++ ):
      $c++;
      if ( $c > 400)
        break;
      $locked = flock($fp, LOCK_EX);
      if ($locked)
        break;
      else
        {
        echo "failed to get LOCK_EX;<br>";
        usleep(5000);
        }
    endfor;
    */
    $locked = flock($fp, LOCK_EX);
    if ( $locked ) {  // acquire an exclusive lock
        $success = fwrite($fp, $s);
        if ( $success === false)
          echo "; w FAILED;";
        else
          if ( $_DEBUG_ )
                echo " $success B written; ";
        $success = fflush($fp);// flush output before releasing the lock
        if ( $success === false ) 
          echo "; flush FAILED; ";
        $success = flock($fp, LOCK_UN);    // release the lock
        if ( $success === false ) 
          echo "; release FAILED; ";
        $success = fclose($fp);
        if ( $success === false ) 
          echo "; fclose FAILED; ";
        clearstatcache(); // needed for filesize and touch
        $fsize = filesize($tname);
        if ($original_size>$fsize)
            {
            echo "; <b>WRITE FAILED, restoring</b>;";
            $original_fname = "$n";
            $result = copy($original_fname, $tname);
            if ($result == false )
              die(" <b>TOTAL FAILTURE: copy failed.</b>");
            else
              echo " <b>RESTORED</b>;";
            }
        else
        {
          if ($fsize === 0)
           echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp<br>";    
          if ( $success ) 
              touch("$tname",$fsize,$p);
        }
    } else {
        echo "Couldn't get the lock!";
    }
     $time_elapsed_secs = microtime(true) - $start;
     //usleep( $delay_multiplier + $n*rand(2,6) ); 
     if ( $time_elapsed_secs === 0 )
       echo " FAILED ";
    echo "time: $time_elapsed_secs s<br>"; 
  }
}
// headers to identify originator of the request
switch ( $_SERVER['HTTP_USER_AGENT'] ):
  // FF 1:
  case "Mozilla/5.0 (Windows NT 5.1;) Gecko": 
    $p = 1; break;
  // Chrome:
  case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit Chrome  Safari":
    $p = 2; break;
  // OPERA:
  case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit Chrome Safari":  
    $p = 3; break;
endswitch;

copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");

test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);

T8(mkdir锁定测试)的代码:

clearstatcache();
$_DEBUG_ = false;

echo "Atomicity tester.".time()."<br>";
$time_constant = 1570787996;
die; // Remove this line when you set time_constant 

while ( time()<$time_constant )
 {
 usleep(500);
 }

/*
c is counter for optimalization
first call must have c = 0;
*/
function atomicFuse($n, $c, $disableDelay = false){
  $start = false;
  if ( !file_exists("$n.t") ) 
   $start = mkdir("$n.t");
  if ( !$disableDelay ){
    if ( $start == false )
     {
     $n = $n*30;
     switch($c):      // Delay example increase:
       case 0: break; // 0,01569 total
       case 1: break; // 0,03138 total
       case 2: $n = $n*2; break; // 0,06276 total
       case 3: $n = $n*4; break; // 0,12552 total
       // case 4: You need at least *6 or *8 to get out of problems with extrem times
       case 4: $n = $n*8; break; // 0,25104 t.(upper limit)
       // In case of heavy traffic:
       case 5: $n = $n*8; break; // 0,36087 total extrem
       case 6: $n = $n*10; break; // 0,51777 total extrem
       case 7: $n = $n*20; break; // 1,03554 total extrem
       default: $n = $n*8; break;
     endswitch;
     usleep($n);
     echo ($n)."<br>";
     }
    }
  return $start;
}
function test($n, $p, $_DEBUG_){
  $fp = null;
  $sname = "$n";    // source
  $tname = "$n.txt";// target
  echo "<h4>$n at ".time()."</h4>";
  for ($i = 0; $i<50; $i++ ){
    $start_time = microtime(true);
      {
      $start = atomicFuse($n,0);
      if (!$start) $start = atomicFuse($n,1);
      if (!$start) $start = atomicFuse($n,2);
      if (!$start) $start = atomicFuse($n,3);
      if (!$start) $start = atomicFuse($n,4);
      if (!$start) $start = atomicFuse($n,5);
      if (!$start) $start = atomicFuse($n,6);
      if (!$start) $start = atomicFuse($n,7);
      if (!$start) $start = atomicFuse($n, false);
      if (!$start) echo "<b>Atomicity failed.</b> ";
      if ( $start )
         {
         echo "<b>Atomicity OK.</b> ";
         /////////////////////////////
         // CHECK FILESIZE VALIDITY //
         /////////////////////////////
         clearstatcache(); // needed for filesize and touch    
         $st = stat("$sname");
         $original_size = $st['size'];
         if ( $_DEBUG_ )
           echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
         $fsize = filesize($sname);
         if ( $original_size <> $fsize )
           die("; fsize total FAILTURE; ");
         if ($fsize === 0)
          echo "! <b>The fsize is 0</b>: stat(): ".$st['size']." ;";    
         ///////////////////
         // OPEN THE FILE //
         ///////////////////
         $fp = fopen($sname, "r");
         $s = fread($fp, $fsize );
         $success = fclose($fp);
         if ( $success === false  )
           die("; fclose failed; ");
         // 10 - loaded data, $p - browser
         if ( $success )
           { 
           $result = touch("$sname",strlen($s),$p);
           if ( $_DEBUG_ )
              echo "; TOUCH: $result;";
           }
         else
           die("fclose FAIL.");
         if ( strlen($s)<60 ) 
            echo "*$s LENGTH:".strlen($s)."<br>";
         }  
      }
    if ( $start )
      {
      clearstatcache();
      $st = stat("$tname");                               
      if ( $_DEBUG_ )
        echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";

      // WRITE OPERATION WITH LOC_EX
      $fp = fopen($tname, "w");
      if ( true ) {  // acquire an exclusive lock
          $success = fwrite($fp, $s);
          if ( $success === false)
            echo "; w FAILED;";
          else
            if ( $_DEBUG_ )
                  echo " $success B written; ";
          $success = fflush($fp);// flush output before releasing the lock
          if ( $success === false ) 
            echo "; flush FAILED; ";
          if ( $success === false ) 
            echo "; release FAILED; ";
          $success = fclose($fp);
          if ( $success === false ) 
            echo "; fclose FAILED; ";
          clearstatcache(); // needed for filesize and touch
          $fsize = filesize($tname);
          if ($original_size>$fsize)
              {
              echo "; <b>WRITE FAILED, restoring</b>;";
              $original_fname = "$n";
              $result = copy($original_fname, $tname);
              if ($result == false )
                die(" <b>TOTAL FAILTURE: copy failed.</b>");
              else
                echo " <b>RESTORED</b>;";
              }
          else
            {
              if ($fsize === 0)
               echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp<br>";    
              if ( $success ) 
                  touch("$tname",$fsize,$p);
            }
          } else {
              echo "Couldn't get the lock!";
             }
      $success = rmdir("$n.t"); // remove atomic fuse
      if ( $success )
        echo "<h4>DIR REMOVED</h4>";
      else
        echo "<h4>DIR NOT REMOVED</h4>";
      } // start
     else 
       echo "skipped"; 
     $time_elapsed_secs = microtime(true) - $start_time;
     if ( $time_elapsed_secs === 0 )
       echo " FAILED ";
     echo "time: $time_elapsed_secs s<br>"; 
  } // for
}

switch ( $_SERVER['HTTP_USER_AGENT'] ):
  case "": 
    $p = 1; break;
  case "":
    $p = 2; break;
  case "":  
    $p = 3; break;
endswitch;

copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");

test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);

注意:T5-T7-我没有确定文件损坏是由fflush还是fwrite造成的,但是在这些测试中会发生这些错误。

注意:T8-此测试的特定问题是,它通常在测试块开始时(在测试功能开始时)等待太长时间。甚至有7秒的等待时间。但是我也尝试删除这些数字,并且平均变化不会太大,因此T8的曲线在此变化之后将保持不变。这里的问题是,在循环中使用延迟不是解决问题的理想方法,这会使失败的可能性更高。请注意,“失败”并不是说文件损坏,而是由于超时而跳过了给定的原子任务。