遍历未知对象类型的图并变异一些对象属性

时间:2020-05-21 21:26:56

标签: c# asp.net-core graph-traversal

特定要求是,如果用户没有特定权限,则替换某些MVC模型属性中的某些值。

该解决方案应该适用于任何模型的任何图形,并且应该相当有效,因为它将用于掩盖大量对象中的值。

假设是:

  • 该图由未知类型的自定义对象组成(只有这些对象将是C#类)
  • 所有感兴趣的对象都具有公共属性,并且仅应检查公共属性(可能使用自定义[SensitiveData]属性作为过滤器,主要是出于性能方面的原因而忽略了大多数属性)
  • 这些对象可能还需要遍历其他[SensitiveData]属性的子对象
  • 对象可能具有需要遍历的不同类型的集合的属性(带有自定义对象的IEnumerable,IList,通用IEnumerable和IList)
  • 这些对象还可能包含.NET核心对象的集合,例如IEnumerable<int>IEnumerable<DateTime>,这对我来说并不重要,因此不应该遍历
  • 我不想遍历.NET核心对象的属性(无需检查DateTime的Date属性)
  • 我不想遍历.NET核心对象(无需检查字符串的每个字符,即IEnumerable)
  • 在大多数情况下,它将是树而不是图。尽管如此,将其实现为具有防止陷入无限循环的图形的图会更加安全。

这似乎都是很合理的逻辑和共同的要求,所以我认为应该有一些通用的,经过测试的解决方案,我可以根据自己的情况进行调整,只需传递一个回调函数和一些过滤器就可以定义感兴趣的属性,甚至开始遍历。

但是,到目前为止,我发现的所有内容都仅限于单个类似Node的类型,或者该实现不允许更改我选择的属性,或者它进行了深层递归并进行了反思,而没有任何性能方面的考虑。

我可以自己实现它,但结果可能是通过混乱的递归和反射来重新发明轮子。

难道没有任何东西已经存在并且众所周知吗?

此外,我听说反射SetValue和GetValue方法很慢,我应该更好地将setter和getter作为委托进行缓存,并在遇到相同类型时再次使用它们。而且我将再次遇到相同的类型,因为它是ASP.NET Core Web应用程序。因此,如果我缓存每个感兴趣的setter / getter以便将来重用,那么与天真的反射解决方案相比,可能会获得显着的性能提升。

1 个答案:

答案 0 :(得分:1)

花费了一些时间,但是有了awesome graph traversal example from Eric LippertFastMember library,我有了一些可行的方法:

[AttributeUsage(AttributeTargets.Property, AllowMultiple = false, Inherited = true)]
public class SensitiveDataAttribute : Attribute
{
}

public abstract class PocoGraphPropertyWalker
{
    private enum TypeKind
    {
        Primitive,
        IterablePrimitive,
        Poco,
        IterablePoco
    }

    private class TypeAccessDescriptor
    {
        public TypeAccessor accessor;
        public List<PropertyInfo> primitives;
        public List<PropertyInfo> iterables;
        public List<PropertyInfo> singles;
    }

    private static ConcurrentDictionary<Type, TypeAccessDescriptor> _accessorCache =
        new ConcurrentDictionary<Type, TypeAccessDescriptor>();

    public IEnumerable<object> TraversePocoList(IEnumerable<object> pocos)
    {
        if (pocos == null)
            return null;

        foreach (var poco in pocos)
            TraversePoco(poco);

        return pocos;
    }

    public object TraversePoco(object poco)
    {
        var unwound = Traversal(poco, ChildrenSelector).ToList();

        foreach(var unw in unwound)
            VisitPoco(unw);

        return poco;
    }

    public object VisitPoco(object poco)
    {
        if (poco == null)
            return poco;

        var t = poco.GetType();

        // the registry ignores types that are not POCOs
        var typeDesc = TryGetOrRegisterForType(t);

        if (typeDesc == null)
            return poco;

        // do not attempt to parse Keys and Values as primitives,
        // even if they were specified as such
        if (IsKeyValuePair(t))
            return poco;

        foreach (var prop in typeDesc.primitives)
        {
            var oldValue = typeDesc.accessor[poco, prop.Name];
            var newValue = VisitProperty(poco, oldValue, prop);
            typeDesc.accessor[poco, prop.Name] = newValue;
        }

        return poco;
    }

    protected virtual object VisitProperty(object model,
        object currentValue, PropertyInfo prop)
    {
        return currentValue;
    }

    private IEnumerable<object> Traversal(
            object item,
            Func<object, IEnumerable<object>> children)
    {
        var seen = new HashSet<object>();
        var stack = new Stack<object>();

        seen.Add(item);
        stack.Push(item);
        yield return item;

        while (stack.Count > 0)
        {
            object current = stack.Pop();
            foreach (object newItem in children(current))
            {
                // protect against cyclic refs
                if (!seen.Contains(newItem))
                {
                    seen.Add(newItem);
                    stack.Push(newItem);
                    yield return newItem;
                }
            }
        }
    }

    private IEnumerable<object> ChildrenSelector(object poco)
    {
        if (poco == null)
            yield break;

        var t = poco.GetType();

        // the registry ignores types that are not POCOs
        var typeDesc = TryGetOrRegisterForType(t);

        if (typeDesc == null)
            yield break;

        // special hack for KeyValuePair - FastMember fails to access its Key and Value
        // maybe because it's a struct, not class?
        // and now we have prop accessors stored in singles / primitives
        // so we extract it manually
        if (IsKeyValuePair(t))
        {
            // reverting to good old slow reflection
            var k = t.GetProperty("Key").GetValue(poco, null);
            var v = t.GetProperty("Value").GetValue(poco, null);

            if (k != null)
            {
                foreach (var yp in YieldIfPoco(k))
                    yield return yp;
            }

            if (v != null)
            {
                foreach(var yp in YieldIfPoco(v))
                    yield return yp;
            }
            yield break;
        }

        // registration method should have registered correct singles
        foreach (var single in typeDesc.singles)
        {
             yield return typeDesc.accessor[poco, single.Name];
        }

        // registration method should have registered correct IEnumerables
        // to skip strings as enums and primitives as enums
        foreach (var iterable in typeDesc.iterables)
        {
            if (!(typeDesc.accessor[poco, iterable.Name] is IEnumerable iterVals))
                continue;

            foreach (var iterval in iterVals)
                yield return iterval;
        }
    }

    private IEnumerable<object> YieldIfPoco(object v)
    {
        var myKind = GetKindOfType(v.GetType());
        if (myKind == TypeKind.Poco)
        {
            foreach (var d in YieldDeeper(v))
                yield return d;
        }
        else if (myKind == TypeKind.IterablePoco && v is IEnumerable iterVals)
        {
            foreach (var i in iterVals)
                foreach (var d in YieldDeeper(i))
                    yield return d;
        }
    }

    private IEnumerable<object> YieldDeeper(object o)
    {
        yield return o;

        // going slightly recursive here - might have IEnumerable<IEnumerable<IEnumerable<POCO>>>...
        var chs = Traversal(o, ChildrenSelector);
        foreach (var c in chs)
            yield return c;
    }

    private TypeAccessDescriptor TryGetOrRegisterForType(Type t)
    {
        if (!_accessorCache.TryGetValue(t, out var typeAccessorsDescriptor))
        {
            // blacklist - cannot process dictionary KeyValues
            if (IsBlacklisted(t))
                return null;

            // check if I myself am a real Poco before registering my properties
            var myKind = GetKindOfType(t);

            if (myKind != TypeKind.Poco)
                return null;

            var properties = t.GetProperties(BindingFlags.Public | BindingFlags.Instance);
            var accessor = TypeAccessor.Create(t);

            var primitiveProps = new List<PropertyInfo>();
            var singlePocos = new List<PropertyInfo>();
            var iterablePocos = new List<PropertyInfo>();

            // now sort all props in subtypes:
            // 1) a primitive value or nullable primitive or string
            // 2) an iterable with primitives (including strings and nullable primitives)
            // 3) a subpoco
            // 4) an iterable with subpocos
            // for our purposes, 1 and 2 are the same - just properties,
            // not needing traversion

            // ignoring non-generic IEnumerable - can't know its inner types
            // and it is not expected to be used in our POCOs anyway
            foreach (var prop in properties)
            {
                var pt = prop.PropertyType;
                var propKind = GetKindOfType(pt);

                // 1) and 2)
                if (propKind == TypeKind.Primitive || propKind == TypeKind.IterablePrimitive)
                    primitiveProps.Add(prop);
                else
                if (propKind == TypeKind.IterablePoco)
                    iterablePocos.Add(prop); //4)
                else
                    singlePocos.Add(prop); // 3)
            }

            typeAccessorsDescriptor = new TypeAccessDescriptor {
                accessor = accessor,
                primitives = primitiveProps,
                singles = singlePocos,
                iterables = iterablePocos
            };

            if (!_accessorCache.TryAdd(t, typeAccessorsDescriptor))
            {
                // if failed add, a parallel process added it, just get it back
                if (!_accessorCache.TryGetValue(t, out typeAccessorsDescriptor))
                    throw new Exception("Failed to get a type descriptor that should exist");
            }
        }

        return typeAccessorsDescriptor;
    }

    private static TypeKind GetKindOfType(Type type)
    {
        // 1) a primitive value or nullable primitive or string
        // 2) an iterable with primitives (including strings and nullable primitives)
        // 3) a subpoco
        // 4) an iterable with subpocos

        // ignoring non-generic IEnumerable - can't know its inner types
        // and it is not expected to be used in our POCOs anyway

        // 1)
        if (IsSimpleType(type))
            return TypeKind.Primitive;

        var ienumerableInterfaces = type.GetInterfaces()
            .Where(x => x.IsGenericType && x.GetGenericTypeDefinition() ==
            typeof(IEnumerable<>)).ToList();

        // add itself, if the property is defined as IEnumerable<x>
        if (type.IsGenericType && type.GetGenericTypeDefinition() ==
            typeof(IEnumerable<>))
            ienumerableInterfaces.Add(type);

        if (ienumerableInterfaces.Any(x =>
                IsSimpleType(x.GenericTypeArguments[0])))
            return TypeKind.IterablePrimitive;

        if (ienumerableInterfaces.Count() != 0)
            // 4) - it was enumerable, but not primitive - maybe POCOs
            return TypeKind.IterablePoco;

        return TypeKind.Poco;
    }

    private static bool IsBlacklisted(Type type)
    {
        return false;
    }

    public static bool IsKeyValuePair(Type type)
    {
        return type.IsGenericType && 
            type.GetGenericTypeDefinition() == typeof(KeyValuePair<,>);
    }

    public static bool IsSimpleType(Type type)
    {
        return
            type.IsPrimitive ||
            new Type[] {
        typeof(string),
        typeof(decimal),
        typeof(DateTime),
        typeof(DateTimeOffset),
        typeof(TimeSpan),
        typeof(Guid)
            }.Contains(type) ||
            type.IsEnum ||
            Convert.GetTypeCode(type) != TypeCode.Object ||
            (type.IsGenericType && type.GetGenericTypeDefinition() == typeof(Nullable<>) && IsSimpleType(type.GetGenericArguments()[0]))
            ;
    }
}

public class ProjectSpecificDataFilter : PocoGraphPropertyWalker
{
    const string MASK = "******";

    protected override object VisitProperty(object model,
            object currentValue, PropertyInfo prop)
    {
        if (prop.GetCustomAttributes<SensitiveDataAttribute>().FirstOrDefault() == null)
            return currentValue;

        if (currentValue == null || (currentValue is string &&
            string.IsNullOrWhiteSpace((string)currentValue)))
            return currentValue;

        return MASK;
    }
}

用于测试:

enum MyEnum
{
    One = 1,
    Two = 2
}

class A
{
    [SensitiveData]
    public string S { get; set; }
    public int I { get; set; }
    public int? I2 { get; set; }
    public MyEnum Enm { get; set; }
    public MyEnum? Enm1 { get; set; }
    public List<MyEnum> Enm2 { get; set; }
    public List<int> IL1 { get; set; }
    public int?[] IL2 { get; set; }
    public decimal Dc { get; set; }
    public decimal? Dc1 { get; set; }
    public IEnumerable<decimal> Dc3 { get; set; }
    public IEnumerable<decimal?> Dc4 { get; set; }
    public IList<decimal> Dc5 { get; set; }
    public DateTime D { get; set; }
    public DateTime? D2 { get; set; }
    public B Child { get; set; }
    public B[] Children { get; set; }
    public List<B> Children2 { get; set; }
    public IEnumerable<B> Children3 { get; set; }
    public IDictionary<int, int?> PrimDict { get; set; }
    public Dictionary<int, B> PocoDict { get; set; }
    public IDictionary<B, int?> PocoKeyDict { get; set; }
    public Dictionary<int, IEnumerable<B>> PocoDeepDict { get; set; }
}

class B
{
    [SensitiveData]
    public string S { get; set; }
    public int I { get; set; }
    public int? I2 { get; set; }
    public DateTime D { get; set; }
    public DateTime? D2 { get; set; }
    public A Parent { get; set; }
}

class Program
{

    static A root;

    static void Main(string[] args)
    {
        root = new A
        {
            D = DateTime.Now,
            D2 = DateTime.Now,
            I = 10,
            I2 = 20,
            S = "stringy",
            Child = new B
            {
                D = DateTime.Now,
                D2 = DateTime.Now,
                I = 10,
                I2 = 20,
                S = "stringy"
            },
            Children = new B[] {
                new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" },
                new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" },
            },
            Children2 = new List<B> {
                new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" },
                new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" },
            },
            PrimDict = new Dictionary<int, int?> {
                { 1, 2 },
                { 3, 4 }
            },
            PocoDict = new Dictionary<int, B> {
                { 1,  new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" } },
                { 3, new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" } }
            },
            PocoKeyDict = new Dictionary<B, int?> {
                { new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" }, 1 },
                { new B {
                    D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" }, 3 }
            },

            PocoDeepDict = new Dictionary<int, IEnumerable<B>>
            {
                { 1, new [] { new B {D = DateTime.Now,
                    D2 = DateTime.Now,
                    I = 10,
                    I2 = 20,
                    S = "stringy" } } }
            }
        };

        // add cyclic ref for test
        root.Child.Parent = root;

        var f = new VtuaSensitiveDataFilter();
        var r = f.TraversePoco(root);
    }
}

它替换标记的字符串,无论它们在POCO的内部有多深。 我也可以将沃克用于我能想到的所有其他属性访问/更改案例。 仍然不确定我是否在这里重新发明了轮子...