使用子进程的Python多处理停止工作

时间:2013-12-17 06:48:44

标签: python python-2.7 subprocess multiprocessing permutation

你能帮助我吗? - 避免程序<16.>排列后挂起
- 如果找到解决方案,所有流程 停止(参见输出),
- 给我一般性建议,使我的代码更好,更快,并学习更多关于python?

我试图强行使用我的True Crypt容器,因为我忘记了密码。我仍记得这些话,但我想念这个组合。因此我想到了这个剧本。

该怎么办?我把它一个单词列表,它应该a)建立所有排列和b)尝试,如果它找到正确的。因为我在Windows 7上工作,所以我使用True Crypt命令行界面进行尝试,我通过python子进程访问它。

写完single threaded version之后,我想到让它更快。我首先尝试了多个线程然后找到了GIL并使用了多处理。对我来说这是一次学习经历,我之前从未使用过Python,只知道Java和PHP。我之前从未做过并行编程。

代码:

import subprocess, os, sys, time, multiprocessing, Queue, itertools

wordlist = [
"foo",
"bar",
"zoo",
"hello",
"World",
]
tcFile = r"C:\dev\tc-brute-force\test.pa"
tcProg = r"C:\Program Files\TrueCrypt\TrueCrypt.exe"
tcMountLetter = "z"
verbose = 5 # as higher as more output is shown fatal=0-5=trace
counter = 0
numberofworkers = multiprocessing.cpu_count()*2
curenttime = time.time()

def getDuration(starttime):
        return time.time() - starttime

def callTC(password, event):
    commandArgs = [
        tcProg,
        '/a',
        '/s',
        '/q',
        '/v', tcFile,
        '/l', tcMountLetter,
        '/p',  password,
    ]

    child = subprocess.Popen(commandArgs, \
        stderr=open(os.devnull, 'w'), \
        stdout=open(os.devnull, 'w'))
    result = child.communicate() # Really important to get error code!

    if verbose > 4:
        print subprocess.list2cmdline(commandArgs).rstrip() + \
            " Status out=" + str(result[0]) + \
            " err=" + str(result[1]) + \
            ", code=" + str(child.returncode)

    if child.returncode == 0:
        event.set()
        print "Successfully opened TrueCrypt file with '%s' at iteration %d, duration %.3fs" % (password, counter, getDuration(curenttime))

def callTCDaemon(queue, event):
    while True:
        if queue.empty():
            break
        else:
            password = queue.get()
            callTC(password, event)

if __name__ == '__main__':

    manager = multiprocessing.Manager()
    event = manager.Event()
    worker = manager.Queue(numberofworkers)

    # start processes
    pool = []
    for i in xrange(numberofworkers):
        process = multiprocessing.Process(target=callTCDaemon, args=(worker, event))
        process.start()
        pool.append(process)

    # generate permutations
    for x in xrange(1, (len(wordlist)+1) ):
        for permutation in itertools.permutations(wordlist, x):

            # shutdown if result is found
            if event.is_set():
                # wait till finished
                for p in pool:
                    p.join(2)

                print "Finished TrueCrypt brute-force, after %d attempts, duration %.3fs" % (counter, getDuration(curenttime))
                sys.exit(1)

            counter += 1    
            combination = ""

            # build string from permutation
            for i in range(0, len(permutation)):
                combination += permutation[i]

            # output progress
            if verbose == 4 and counter%100 == 0:
                print "%15d|%15.3fs: %s" % (counter, getDuration(curenttime), combination)

            # avoid queue overload
            while worker.qsize() > 100:
                if verbose > 3: print "Wait because queue is full, size=%d" % (worker.qsize)
                time.sleep(4)           

            worker.put(combination)

示例输出(略有改动):

C:\dev\tc-brute-force>python TrueCryptBruteForceProcesses.py
            100|         23.013s: fooWorld
            200|         48.208s: barHelloWorld
Successfully opened TrueCrypt file with 'Worldfoo' at iteration 0, duration 50.218s
Successfully opened TrueCrypt file with 'Worldbar' at iteration 0, duration 50.249s
Successfully opened TrueCrypt file with 'Worldzoo' at iteration 0, duration 50.260s
Successfully opened TrueCrypt file with 'Worldhello' at iteration 0, duration 50.304s
Successfully opened TrueCrypt file with 'foobarzoo' at iteration 0, duration 50.354s
Successfully opened TrueCrypt file with 'helloWorld' at iteration 0, duration 50.433s
Successfully opened TrueCrypt file with 'foobarhello' at iteration 0, duration 50.438s
Successfully opened TrueCrypt file with 'foobarWorld' at iteration 0, duration 50.440s
Successfully opened TrueCrypt file with 'foozoobar' at iteration 0, duration 50.473s
Finished TrueCrypt brute-force, after 209 attempts, duration 50.733s

1 个答案:

答案 0 :(得分:3)

蛮力代码中有三个主要部分:

  • 生成密码
  • 检查单个密码
  • 同时检查多个密码

生成密码

产生所有可能的排列:

import itertools

def generate_passwords(wordlist):
    for password_length in range(1, len(wordlist) + 1): # no repeats
        for password in itertools.permutations(wordlist, password_length):
            yield " ".join(password)

检查密码

看看TRUECRYPT EXPLAINED。也许你不需要为每个密码生成一个子进程。

from subprocess import call

def valid_password(password):
    rc = call(true_crypt_command(password), close_fds=True)
    return rc == 0, password

同时检查多个密码

import sys
from multiprocessing.dummy import Pool # use threads

wordlist = "foo bar zoo hello World".split()
pool = Pool(20) # check 20 passwords in parallel
for i, (found, password) in enumerate(
    pool.imap_unordered(valid_password, generate_passwords(wordlist))):
    if i % 1000 == 0: # report progress
       sys.stderr.write("\rchecked %d" % i)
    if found:
       print("Found: '%s'" % password)
       break
else:
    sys.exit("failed to find")

pool.close() 
####pool.join() # uncomment if it is not the end

除了生成TrueCrypt命令行的true_crypt_command()函数外,它是完整的源代码。

另一个代码示例:Brute force http basic auth

  

如果到达rc == 0,imap_unordered会停止所有其他线程/进程吗?

如果在break之后立即退出程序,则操作系统会自动杀死所有剩余的线程(它们是守护程序线程,因此它们将无法生效)。

如果您的程序在循环之后继续运行,那么最多可以继续运行20个线程(池的大小)。如果您取消注释pool.join(),则可以等待它们结束。