我需要一种方法从外部代码块“注入”名称到函数中,因此它们可以在本地访问和它们不需要由函数的代码专门处理(定义为函数参数,从*args
等加载。)
简化场景:提供一个框架,用户可以在其中定义(尽可能少的语法)自定义函数来操纵框架的其他对象(不必然{{1 }})。
理想情况下,用户定义
global
此处def user_func():
Mouse.eat(Cheese)
if Cat.find(Mouse):
Cat.happy += 1
,Cat
和Mouse
是框架对象,出于好的理由,它们不能与全局命名空间绑定。
我想为此函数编写一个包装器,表现如下:
Cheese
然后这个包装器可以应用于所有用户定义的函数(作为装饰器,由用户自己或自动应用,虽然我打算使用元类)。
def framework_wrap(user_func):
# this is a framework internal and has name bindings to Cat, Mouse and Cheese
def f():
inject(user_func, {'Cat': Cat, 'Mouse': Mouse, 'Cheese': Cheese})
user_func()
return f
我知道Python 3的@framework_wrap
def user_func():
关键字,但我仍然认为丑陋(从框架的用户角度来看)添加额外的一行:
nonlocal
并担心将他需要的每个对象添加到此行。
非常感谢任何建议。
答案 0 :(得分:11)
堆栈越乱,我越希望没有。不要破解全局变量来做你想做的事。而是破解字节码。我有两种方法可以做到这一点。
1)将包含所需引用的单元格添加到f.func_closure
。您必须重新组合函数的字节码才能使用LOAD_DEREF
而不是LOAD_GLOBAL
,并为每个值生成一个单元格。然后,将单元格元组和新代码对象传递给types.FunctionType
,并获得具有相应绑定的函数。函数的不同副本可以具有不同的本地绑定,因此它应该像您想要的那样是线程安全的。
2)在函数参数列表的末尾为新的locals添加参数。将LOAD_GLOBAL
的相应匹配项替换为LOAD_FAST
。然后使用types.FunctionType
构造一个新函数,并传入新的代码对象和所需的绑定元组作为默认选项。这在某种意义上是有限的,因为python将函数参数限制为255,并且它不能用于使用变量参数的函数。然而,它让我感到震惊,因为两者中更具挑战性的是我实施的那个(还有其他可以用这个完成的东西)。同样,您可以使用不同的绑定制作不同的函数副本,也可以使用每个调用位置所需的绑定调用该函数。所以它也可以像你想要的那样线程安全。
import types
import opcode
# Opcode constants used for comparison and replacecment
LOAD_FAST = opcode.opmap['LOAD_FAST']
LOAD_GLOBAL = opcode.opmap['LOAD_GLOBAL']
STORE_FAST = opcode.opmap['STORE_FAST']
DEBUGGING = True
def append_arguments(code_obj, new_locals):
co_varnames = code_obj.co_varnames # Old locals
co_names = code_obj.co_names # Old globals
co_argcount = code_obj.co_argcount # Argument count
co_code = code_obj.co_code # The actual bytecode as a string
# Make one pass over the bytecode to identify names that should be
# left in code_obj.co_names.
not_removed = set(opcode.hasname) - set([LOAD_GLOBAL])
saved_names = set()
for inst in instructions(co_code):
if inst[0] in not_removed:
saved_names.add(co_names[inst[1]])
# Build co_names for the new code object. This should consist of
# globals that were only accessed via LOAD_GLOBAL
names = tuple(name for name in co_names
if name not in set(new_locals) - saved_names)
# Build a dictionary that maps the indices of the entries in co_names
# to their entry in the new co_names
name_translations = dict((co_names.index(name), i)
for i, name in enumerate(names))
# Build co_varnames for the new code object. This should consist of
# the entirety of co_varnames with new_locals spliced in after the
# arguments
new_locals_len = len(new_locals)
varnames = (co_varnames[:co_argcount] + new_locals +
co_varnames[co_argcount:])
# Build the dictionary that maps indices of entries in the old co_varnames
# to their indices in the new co_varnames
range1, range2 = xrange(co_argcount), xrange(co_argcount, len(co_varnames))
varname_translations = dict((i, i) for i in range1)
varname_translations.update((i, i + new_locals_len) for i in range2)
# Build the dictionary that maps indices of deleted entries of co_names
# to their indices in the new co_varnames
names_to_varnames = dict((co_names.index(name), varnames.index(name))
for name in new_locals)
if DEBUGGING:
print "injecting: {0}".format(new_locals)
print "names: {0} -> {1}".format(co_names, names)
print "varnames: {0} -> {1}".format(co_varnames, varnames)
print "names_to_varnames: {0}".format(names_to_varnames)
print "varname_translations: {0}".format(varname_translations)
print "name_translations: {0}".format(name_translations)
# Now we modify the actual bytecode
modified = []
for inst in instructions(code_obj.co_code):
# If the instruction is a LOAD_GLOBAL, we have to check to see if
# it's one of the globals that we are replacing. Either way,
# update its arg using the appropriate dict.
if inst[0] == LOAD_GLOBAL:
print "LOAD_GLOBAL: {0}".format(inst[1])
if inst[1] in names_to_varnames:
print "replacing with {0}: ".format(names_to_varnames[inst[1]])
inst[0] = LOAD_FAST
inst[1] = names_to_varnames[inst[1]]
elif inst[1] in name_translations:
inst[1] = name_translations[inst[1]]
else:
raise ValueError("a name was lost in translation")
# If it accesses co_varnames or co_names then update its argument.
elif inst[0] in opcode.haslocal:
inst[1] = varname_translations[inst[1]]
elif inst[0] in opcode.hasname:
inst[1] = name_translations[inst[1]]
modified.extend(write_instruction(inst))
code = ''.join(modified)
# Done modifying codestring - make the code object
return types.CodeType(co_argcount + new_locals_len,
code_obj.co_nlocals + new_locals_len,
code_obj.co_stacksize,
code_obj.co_flags,
code,
code_obj.co_consts,
names,
varnames,
code_obj.co_filename,
code_obj.co_name,
code_obj.co_firstlineno,
code_obj.co_lnotab)
def instructions(code):
code = map(ord, code)
i, L = 0, len(code)
extended_arg = 0
while i < L:
op = code[i]
i+= 1
if op < opcode.HAVE_ARGUMENT:
yield [op, None]
continue
oparg = code[i] + (code[i+1] << 8) + extended_arg
extended_arg = 0
i += 2
if op == opcode.EXTENDED_ARG:
extended_arg = oparg << 16
continue
yield [op, oparg]
def write_instruction(inst):
op, oparg = inst
if oparg is None:
return [chr(op)]
elif oparg <= 65536L:
return [chr(op), chr(oparg & 255), chr((oparg >> 8) & 255)]
elif oparg <= 4294967296L:
return [chr(opcode.EXTENDED_ARG),
chr((oparg >> 16) & 255),
chr((oparg >> 24) & 255),
chr(op),
chr(oparg & 255),
chr((oparg >> 8) & 255)]
else:
raise ValueError("Invalid oparg: {0} is too large".format(oparg))
if __name__=='__main__':
import dis
class Foo(object):
y = 1
z = 1
def test(x):
foo = Foo()
foo.y = 1
foo = x + y + z + foo.y
print foo
code_obj = append_arguments(test.func_code, ('y',))
f = types.FunctionType(code_obj, test.func_globals, argdefs=(1,))
if DEBUGGING:
dis.dis(test)
print '-'*20
dis.dis(f)
f(1)
请注意,此代码的整个分支(与EXTENDED_ARG
相关)未经测试,但对于常见情况,它似乎非常可靠。我将攻击它,目前正在编写一些代码来验证输出。然后(当我绕过它时)我将对整个标准库运行它并修复任何错误。
我也可能正在实施第一个选项。
答案 1 :(得分:4)
编辑回答 - 调用user_func()
使用Python 2.7.5和3.3.2测试
文件framework.py:
# framework objects
class Cat: pass
class Mouse: pass
class Cheese: pass
_namespace = {'Cat':Cat, 'Mouse':Mouse, 'Cheese':Cheese } # names to be injected
# framework decorator
from functools import wraps
def wrap(f):
func_globals = f.func_globals if hasattr(f,'func_globals') else f.__globals__
@wraps(f)
def wrapped(*args, **kwargs):
# determine which names in framework's _namespace collide and don't
preexistent = set(name for name in _namespace if name in func_globals)
nonexistent = set(name for name in _namespace if name not in preexistent)
# save any preexistent name's values
f.globals_save = {name: func_globals[name] for name in preexistent}
# temporarily inject framework's _namespace
func_globals.update(_namespace)
retval = f(*args, **kwargs) # call function and save return value
# clean up function's namespace
for name in nonexistent:
del func_globals[name] # remove those that didn't exist
# restore the values of any names that collided
func_globals.update(f.globals_save)
return retval
return wrapped
使用示例:
from __future__ import print_function
import framework
class Cat: pass # name that collides with framework object
@framework.wrap
def user_func():
print('in user_func():')
print(' Cat:', Cat)
print(' Mouse:', Mouse)
print(' Cheese:', Cheese)
user_func()
print()
print('after user_func():')
for name in framework._namespace:
if name in globals():
print(' {} restored to {}'.format(name, globals()[name]))
else:
print(' {} not restored, does not exist'.format(name))
输出:
in user_func():
Cat: <class 'framework.Cat'>
Mouse: <class 'framework.Mouse'>
Cheese: <class 'framework.Cheese'>
after user_func():
Cheese not restored, does not exist
Mouse not restored, does not exist
Cat restored to <class '__main__.Cat'>
答案 2 :(得分:3)
听起来您可能希望使用exec code in dict
,其中code
是用户的功能,dict
是您提供的字典
exec的文档:http://docs.python.org/reference/simple_stmts.html#the-exec-statement
但是,我很确定只有当用户的代码以字符串形式引入并且您需要执行它时,这才会起作用。如果函数已经编译,则它已经设置了全局绑定。所以做exec "user_func(*args)" in framework_dict
这样的事情是行不通的,因为user_func
的全局变量已经设置为定义的模块。
由于func_globals
是只读的,我认为您必须执行类似what martineau suggests的操作才能修改函数全局变量。
我认为可能(除非你做了一些前所未有的令人敬畏的事情,或者我错过了一些关键的微妙之处),你可能最好将框架对象放入模块中,然后让用户代码导入该模块。在模块被import
编辑后,模块变量可以通过在该模块外部定义的代码非常容易地重新分配或变异或访问。
我认为这对于代码可读性也会更好,因为user_func
最终会为Cat
,Dog
等提供明确的命名空间,而不是不熟悉您的框架的读者想知道他们来自哪里。例如。 animal_farm.Mouse.eat(animal_farm.Cheese)
,或者像
from animal_farm import Goat
cheese = make_cheese(Goat().milk())
如果 做了一些非常棒的事情,我认为您需要使用C API将参数传递给代码对象。看起来函数PyEval_EvalCodeEx就是你想要的那个。
答案 3 :(得分:1)
如果您的应用程序严格来说是Python 3,我不会看到使用Python 3的nonlocal
比编写装饰器来操作函数的本地命名空间更为丑陋。我说试试nonlocal
解决方案或重新考虑这个策略。