Python相当于Golang在频道上的选择

时间:2013-10-02 06:16:09

标签: python go

Go有一个适用于频道的select语句。来自文档:

  

select语句允许goroutine等待多次通信   操作

     

一个选择块,直到其中一个案例可以运行,然后执行它   案件。如果多个准备就绪,它会随机选择一个。

是否有以下代码的Python等价物:

package main

import "fmt"

func main() {
    c1 := make(chan int)
    c2 := make(chan int)
    quit := make(chan int)

    go func() {
        for i := 0; i < 10; i++ {
            c1 <- i
        }
        quit <- 0
    }()

    go func() {
        for i := 0; i < 2; i++ {
            c2 <- i
        }
    }()

    for {
        select {
        case <-c1:
            fmt.Println("Received value from c1")
        case <-c2:
            fmt.Println("Received value from c2")
        case <-quit:
            fmt.Println("quit")
            return
        }
    }
}

该计划的输出:

Received value from c1
Received value from c1
Received value from c2
Received value from c1
Received value from c2
Received value from c1
Received value from c1
Received value from c1
Received value from c1
Received value from c1
Received value from c1
Received value from c1
quit

7 个答案:

答案 0 :(得分:11)

这是一个非常直接的翻译,但“选择哪个如果多个已经准备好”部分的工作方式不同 - 它只是采取了首先出现的内容。这也就像使用gomaxprocs(1)运行代码一样。

import threading
import Queue

def main():
    c1 = Queue.Queue(maxsize=0)
    c2 = Queue.Queue(maxsize=0)
    quit = Queue.Queue(maxsize=0)

    def func1():
        for i in range(10):
            c1.put(i)
        quit.put(0)

    threading.Thread(target=func1).start()

    def func2():
        for i in range(2):
            c2.put(i)

    threading.Thread(target=func2).start()

    combined = Queue.Queue(maxsize=0)

    def listen_and_forward(queue):
        while True:
            combined.put((queue, queue.get()))

    t = threading.Thread(target=listen_and_forward, args=(c1,))
    t.daemon = True
    t.start()
    t = threading.Thread(target=listen_and_forward, args=(c2,))
    t.daemon = True
    t.start()
    t = threading.Thread(target=listen_and_forward, args=(quit,))
    t.daemon = True
    t.start()

    while True:
        which, message = combined.get()
        if which is c1:
            print 'Received value from c1'
        elif which is c2:
            print 'Received value from c2'
        elif which is quit:
            print 'Received value from quit'
            return
main()

基本更改是使用组合消息的线程模拟select。如果您打算使用这种模式,可以编写一些选择代码:

import threading
import Queue

def select(*queues):
    combined = Queue.Queue(maxsize=0)
    def listen_and_forward(queue):
        while True:
            combined.put((queue, queue.get()))
    for queue in queues:
        t = threading.Thread(target=listen_and_forward, args=(queue,))
        t.daemon = True
        t.start()
    while True:
        yield combined.get()

def main():

    c1 = Queue.Queue(maxsize=0)
    c2 = Queue.Queue(maxsize=0)
    quit = Queue.Queue(maxsize=0)

    def func1():
        for i in range(10):
            c1.put(i)
        quit.put(0)

    threading.Thread(target=func1).start()

    def func2():
        for i in range(2):
            c2.put(i)

    threading.Thread(target=func2).start()

    for which, msg in select(c1, c2, quit):
        if which is c1:
            print 'Received value from c1'
        elif which is c2:
            print 'Received value from c2'
        elif which is quit:
            print 'Received value from quit'
            return
main()

但是...

请注意,这个选择不是很好的选择,虽然它对你的程序没有关系 - 如果我们并不总是如此,goroutine可以在一个将在select中排队的频道发送结果并丢失迭代选择完成!

答案 1 :(得分:9)

还要考虑Benoit Chesneau的offset library。它是Python的Go并发模型的一个端口,使用了光纤。

他在PyCon APAC 2013上发表了关于此事的演讲:

答案 2 :(得分:7)

您可以使用multiprocessing.Pipe代替chanthreading.Thread代替goselect.select代替select

这是使用这种方法在Python中重新实现你的go示例:

import random
from multiprocessing import Pipe
from select import select
from threading import Thread


def main():
    c1_r, c1_w = Pipe(duplex=False)
    c2_r, c2_w = Pipe(duplex=False)
    quit_r, quit_w = Pipe(duplex=False)

    def func1():
        for i in range(10):
            c1_w.send(i)
        quit_w.send(0)

    Thread(target=func1).start()

    def func2():
        for i in range(2):
            c2_w.send(i)

    Thread(target=func2).start()

    while True:
        ready, _, _ = select([c1_r, c2_r, quit_r], [], [])
        which = random.choice(ready)
        if which == c1_r:
            c1_r.recv()
            print 'Received value from c1'
        elif which == c2_r:
            c2_r.recv()
            print 'Received value from c2'
        elif which == quit_r:
            quit_r.recv()
            print 'Received value from quit'
            return

if __name__ == '__main__':
    main()

这个实现是基于@Thomas的实现,但与@Thomas不同,它不会产生额外的线程来执行选择。

使用Python 2.7.13在Linux上测试。 Windows可能表现不同,因为select是Unixy。

答案 3 :(得分:3)

使用Python 3.5,有关键字asyncawait可以使函数可以在执行时暂停,从而可以在evenloop而不是线程上运行。 asyncio std lib提供了一个。

要更直接地映射Go阻塞通道和select的行为,您可以使用this small library,然后您的示例代码在Python中看起来非常相似。

答案 4 :(得分:2)

这是另一个尝试模仿go语法:

from threading import Thread
from Queue import Queue

def main():

    c1 = Queue.Queue(maxsize=0)
    c2 = Queue.Queue(maxsize=0)
    quit = Queue.Queue(maxsize=0)

    Thread(target=lambda: [c1.put(i) for i in range(10)] or quit.put(0)).start()
    Thread(target=lambda: [c2.put(i) for i in range(2)]).start()

    for which, msg in select(c1, c2, quit):
        if which is c1:
            print 'Received value from c1'
        elif which is c2:
            print 'Received value from c2'
        elif which is quit:
            print 'Received value from quit'
            return

def select(*queues):
    combined = Queue.Queue(maxsize=0)
    def listen_and_forward(queue):
        while True:
            combined.put((queue, queue.get()))
    for queue in queues:
        t = Thread(target=listen_and_forward, args=(queue,))
        t.daemon = True
        t.start()
    while True:
        yield combined.get()

main()

答案 5 :(得分:2)

是的,goless可以实现一切。你可以尝试一下。

玩得开心; - )

以下是一个例子:

c1 = goless.chan()
c2 = goless.chan()

def func1():
    time.sleep(1)
    c1.send('one')
goless.go(func1)

def func2():
    time.sleep(2)
    c2.send('two')
goless.go(func2)

for i in range(2):
    case, val = goless.select([goless.rcase(c1), goless.rcase(c2)])
    print(val)

答案 6 :(得分:1)

为完整起见:pygolang中提供了包括工作选择在内的Go风格频道:

ch1 = chan()    # synchronous channel
ch2 = chan(3)   # channel with buffer of size 3

def _():
    ch1.send('a')
    ch2.send('b')
go(_)

ch1.recv()      # will give 'a'
ch2.recv_()     # will give ('b', True)

_, _rx = select(
    ch1.recv,           # 0
    ch2.recv_,          # 1
    (ch2.send, obj2),   # 2
    default,            # 3
)
if _ == 0:
    # _rx is what was received from ch1
    ...
if _ == 1:
    # _rx is (rx, ok) of what was received from ch2
    ...
if _ == 2:
    # we know obj2 was sent to ch2
    ...
if _ == 3:
    # default case
    ...

offset(请参阅https://stackoverflow.com/a/19143696/9456786)似乎也很有趣。

goless(请参阅https://stackoverflow.com/a/39269599/9456786)不幸的是,它有weak select implementation,其设计是does not work properly on synchronous channels