Question

我想同步/＆＃34;顺序化＆＃34;多个实例同时运行特定shell。

可选地，参数子集可以确定进程是立即运行还是必须等待。

要明确：

如果我同时运行以下三个命令（在此示例中为schema和table选项确定锁定）：

loadTable --schema dev --table person --file mydata1.txt
loadTable --schema dev --table person --file mydata2.txt
loadTable --schema dev --table enterprise --file mydata3.txt

我想：

第一个和第三个命令可以同时运行
因为部分参数不同（schema和table）
第二个命令必须等待第一个结束
因为部分参数相同（schema和table）

对我来说有两个解决方案：

loadTable shell必须自己管理一个等待队列。
父代理 shell必须管理它

我有一些想法，但似乎有点复杂......

感谢您的帮助

Answer 1

我已经制作了同步功能。

我现在可以在我的脚本开头调用（在args解析之后）：

synchronize $myTable-$mySchema

它完成了这项工作，但我们可以简化它。

代码：

function synchronize() {
    key=${1:-noKey};    
    shell_fullname=$(readlink -f $0)
    shell_basename=$(basename $shell_fullname)    
    hash=$(echo "${shell_fullname}-${key}" | md5sum | cut -b-32)
    delay=2
    pid_file=/tmp/${shell_basename}.${hash}.pid    
    current_pid=$$        

    echo "synchronize$shell_basename($key)"
    (    
        # First step : lock file         
        isUnLock=true
        echo "trying to acquire lock"
        while $isUnLock
        do
            # Wait for lock on file desciptor 200 for 10 seconds
            flock -x -w $delay 200 && isUnLock=false
            $isUnLock && echo "Waiting lock for"                   
        done

        # here : isUnLock must normally be false
        $isUnLock && echo "unable to acquire lock" # not possible for me
        $isUnLock && exit 255 # bad luck...
        $isUnLock || echo "lock OK"         

        # Second step : waiting eventual previous process
        while [ -e ${pid_file} ] && kill -0 `cat $pid_file` > /dev/null 2>&1
        do
            echo "Another process already running with process_id $(cat ${pid_file})"        
            sleep $delay
        done

        # here : previous shell stop and the current process has lock the pid_file
        echo $current_pid > ${pid_file}

        # now we can unlock the pid file and the current shell can be running alone safely
        # (just fail if somebody delete the pid_file...)        

    ) 200>$pid_file.lock # add suffix, flock seems make empty file after lock... 

}

key=$1 # construct key using args
synchronize$key

最严格的版本

function synchronize() {
    key=${1:-noKey};    
    shell_fullname=$(readlink -f $0)
    shell_basename=$(basename $shell_fullname)    
    hash=$(echo "${shell_fullname}-${key}" | md5sum | cut -b-32)
    delay=10
    pid_file=/tmp/${shell_basename}.${hash}.pid    
    current_pid=$$        

    echo "synchronize $shell_basename($key) pid_file=$pid_file"
    (    
        # First step : lock file, Wait for lock on file desciptor 200
        echo "trying to acquire lock"
        flock -x 200 
        echo "lock acquired"

        # Second step : waiting eventual previous process        
        [ -s $pid_file ] && previous_pid=$(cat $pid_file) || previous_pid=00 # 00 is an impossible pid        
        [ -e /proc/${previous_pid} ] && echo "Another process already running with process_id $previous_pid"        
        while [ -e /proc/${previous_pid} ]; do sleep $delay; done

        # here : previous shell is stop and the current process has lock the pid_file        
        echo $current_pid > ${pid_file}        
        echo "current pid $current_pid is running"

        # now we can unlock the pid file and the current shell can be running several minutes      

    ) 200>$pid_file.lock # add suffix, flock seems make empty file after lock... 

}

Answer 2

您可以在第二步简化和安全性
（securise =＆gt;通过这种方式，即使在上次执行期间删除了aaa，它也能正常工作）

$pid_file

Answer 3

如果我做对了，你需要做这样的事情：

loadTable --schema dev --table person --file mydata1.txt &
loadTable --schema dev --table enterprise --file mydata3.txt &
wait %1 && loadTable --schema dev --table person --file mydata2.txt &
wait

在这种情况下，您将并行运行命令1和3，然后等待第一个命令完成并运行第二个命令。之后，等待所有人完成

如何同步不同的shell实例

3 个答案: