在嵌套数组的每个项目中重新映射ObjectIds数组

时间:2018-06-06 19:45:23

标签: node.js mongodb mongoose mongodb-query aggregation-framework

我有一个文档,其中包含用户生成的tags以及entries,其中包含每个条目的标记ID数组(或者可能没有):

// Doc (with redacted items I would like to project too)
{
    "_id": ObjectId("5ae5afc93e1d0d2965a4f2d7"),
    "entries" : [
        {
            "_id" : ObjectId("5b159ebb0ed51064925dff24"),
            // Desired:
            // tags: {[
            //   "_id" : ObjectId("5b142ab7e419614016b8992d"),
            //   "name" : "Shit",
            //   "color" : "#95a5a6"
            // ]}
            "tags" : [
                ObjectId("5b142ab7e419614016b8992d")
            ]
        },
    ],
    "tags" : [
        {
            "_id" : ObjectId("5b142608e419614016b89925"),
            "name" : "Outdated",
            "color" : "#3498db"
        },
        {
            "_id" : ObjectId("5b142ab7e419614016b8992d"),
            "name" : "Shit",
            "color" : "#95a5a6"
        },
    ],
}

如何使用tags数组中的相应值为每个条目“填充”标记数组?我尝试了$ lookup和聚合,但它太复杂了,无法做到正确。

1 个答案:

答案 0 :(得分:1)

从实际数据的外观来看,此处无需populate()$lookup,因为您想要加入的数据是"不仅是在同一个集合中,而且它实际上在同一个文档中。你想要的是$map甚至是Array.map(),只需在文档的一个数组中取值并将它们合并到另一个数组中。

聚合$ map transform

这里需要做的基本情况是$map来转换输出中的每个数组。这些是"entries"并且在每个"条目内#34;通过将值与父文档的"tags"数组中的值进行匹配来转换"tags"

Project.aggregate([
  { "$project": {
    "entries": {
      "$map": {
        "input": "$entries",
        "as": "e",
        "in": {
          "someField": "$$e.someField",
          "otherField": "$$e.otherField",
          "tags": {
            "$map": {
              "input": "$$e.tags",
              "as": "t",
              "in": {
                "$arrayElemAt": [
                  "$tags",
                  { "$indexOfArray": [ "$tags._id", "$$t" ] }
                ]
              }
            }
          }
        }
      }
    }
  }}
])

请注意,"someField""otherField"作为字段的占位符,"可能" 在每个"条目中出现在该级别&#34 34;数组的文档。 $map唯一的问题是"in"参数中指定的是您实际获得的 输出,因此需要明确命名您的"变量键中的每个潜在字段"结构,包括"tags"

自MongoDB 3.6以来,在现代版本中对此的反击是使用$mergeObjects而不是允许"合并" "重新映射"内部数组"tags"进入"条目"每个数组成员的文档:

Project.aggregate([
  { "$project": {
    "entries": {
      "$map": {
        "input": "$entries",
        "as": "e",
        "in": {
          "$mergeObjects": [
            "$$e",
            { "tags": {
              "$map": {
                "input": "$$e.tags",
                "as": "t",
                "in": {
                  "$arrayElemAt": [
                    "$tags",
                    { "$indexOfArray": [ "$tags._id", "$$t" ] }
                  ]
                }
              }
            }}
          ]
        }
      }
    }
  }}
])

关于"内部"的实际$map数组"tags",您可以使用$indexOfArray运算符与"根级别"进行比较。 "tags"的字段,基于_id属性与此"内部"的当前条目的值匹配的位置阵列。用那个"索引"返回后,$arrayElemAt运算符然后"提取"来自匹配的"索引"的实际数组条目位置,并使用该元素移植$map中的当前数组条目。

这里唯一的关注点是两个数组实际上由于某种原因没有匹配的条目。如果您已经处理过这个问题,那么这里的代码就可以了。如果不匹配,您可能需要$filter来匹配元素并转而使用索引0处的$arrayElemAt

    "in": {
      "$arrayElemAt": [
        { "$filter": {
          "input": "$tags",
          "cond": { "$eq": [ "$$this._id", "$$t" ] }
        }},
        0
      ]
    }

原因是这样做可以使null无法匹配,但$indexOfArray会返回-1,而$arrayElemAt使用的会返回&#34 ;最后"数组元素。并且"最后"在这种情况下,元素当然不是"匹配"结果,因为没有匹配。

客户端转换

所以从你的角度来看,只有"只有"返回"entries"内容"重新映射"并且从文档的根目录中丢弃"tags",聚合过程尽可能是更好的选择,因为服务器只返回您实际需要的元素。

如果您不能这样做,或者真的不关心是否还返回了现有的"tags"元素,那么根本就没有必要进行聚合转换。实际上"服务器"不需要做任何事情,并且可能"不应该" 考虑到所有数据已经​​在文档中并且"其他"转换只是添加到文档大小。

因此,一旦返回到客户端,这实际上可以用结果完成,并且对于文档的简单转换,就像上面的聚合管道示例所示,您实际需要的唯一代码是:

let results = await Project.find().lean();

results = results.map(({ entries, tags, ...r }) =>
  ({
    ...r,
    entries: entries.map(({ tags: etags, ...e }) =>
      ({
        ...e,
        tags: etags.map( tid => tags.find(t => t._id.equals(tid)) )
      })
    ),
    // tags
  })
);

这会为您提供完全相同的结果,甚至可以通过删除评论来保留tags。它甚至基本上都是"完全相同的过程"在每个数组上使用Array.map()以对每个数组进行转换。

&#34; merge&#34;的语法使用现代JavaScript object spread operations要简单得多,总体来说语言简洁得多。您使用Array.find()来查找&#34;查找&#34; tags的两个数组的匹配内容以及唯一需要注意的是ObjectId.equals()方法,实际比较这两个值并将其内置到返回的类型中。< / p>

当然,因为你正在&#34;转变&#34;文档,为了使你可以在任何mongoose操作上使用lean()返回结果进行操作,所以返回的数据实际上是纯JavaScript对象而不是绑定到模式的Mongoose Document类型,是默认的回报。

结论和示范

这里的一般教训是,如果您希望减少数据&#34;在返回的响应中,aggregate()方法适合您。但是,如果你决定想要&#34;整体&#34;无论如何都要记录数据,只是想&#34;增加&#34;响应中的这些其他数组条目,然后只需将数据带回&#34;客户端&#34;而是转而在那里。理想情况下是&#34;向前&#34;考虑到&#34;添加&#34;在这种情况下,只是增加了有效负载响应的权重。

完整的演示列表将是:

const { Schema, Types: { ObjectId } } = mongoose = require('mongoose');

const uri = 'mongodb://localhost/test';

mongoose.Promise = global.Promise;
mongoose.set('debug', true);

const tagSchema = new Schema({
  name: String,
  color: String
});

const projectSchema = new Schema({
  entries: [],
  tags: [tagSchema]
});

const Project = mongoose.model('Project', projectSchema);

const log = data => console.log(JSON.stringify(data, undefined, 2));

(async function() {

  try {
    const conn = await mongoose.connect(uri);

    let db = conn.connections[0].db;

    let { version } = await db.command({ buildInfo: 1 });
    version = parseFloat(version.match(new RegExp(/(?:(?!-).)*/))[0]);

    await Promise.all(Object.entries(conn.models).map(([k,m]) => m.remove()));

    await Project.insertMany(data);

    let pipeline = [
      { "$project": {
        "entries": {
          "$map": {
            "input": "$entries",
            "as": "e",
            "in": {
              "someField": "$$e.someField",
              "otherField": "$$e.otherField",
              "tags": {
                "$map": {
                  "input": "$$e.tags",
                  "as": "t",
                  "in": {
                    "$arrayElemAt": [
                      "$tags",
                      { "$indexOfArray": [ "$tags._id", "$$t" ] }
                    ]
                  }
                }
              }
            }
          }
        }
      }}
    ];

    let other = [
      {
        ...(({ $project: { entries: { $map: { input, as, ...o } } } }) =>
          ({
            $project: {
              entries: {
                $map: {
                  input,
                  as,
                  in: {
                    "$mergeObjects": [ "$$e", { tags: o.in.tags } ]
                  }
                }
              }
            }
          })
        )(pipeline[0])
      }
    ];

    let tests = [
      { name: 'Standard $project $map', pipeline },
      ...(version >= 3.6) ?
        [{ name: 'With $mergeObjects', pipeline: other }] : []
    ];

    for ( let { name, pipeline } of tests ) {
      let results = await Project.aggregate(pipeline);
      log({ name, results });
    }


    // Client Manipulation

    let results = await Project.find().lean();

    results = results.map(({ entries, tags, ...r }) =>
      ({
        ...r,
        entries: entries.map(({ tags: etags, ...e }) =>
          ({
            ...e,
            tags: etags.map( tid => tags.find(t => t._id.equals(tid)) )
          })
        )
      })
    );

    log({ name: 'Client re-map', results });

    mongoose.disconnect();

  } catch(e) {
    console.error(e)
  } finally {
    process.exit()
  }

})();

// Data

const data =[
  {
    "_id": ObjectId("5ae5afc93e1d0d2965a4f2d7"),
    "entries" : [
      {
        "_id" : ObjectId("5b159ebb0ed51064925dff24"),
        "someField": "someData",
        "tags" : [
          ObjectId("5b142ab7e419614016b8992d")
        ]
      },
    ],
    "tags" : [
      {
        "_id" : ObjectId("5b142608e419614016b89925"),
        "name" : "Outdated",
        "color" : "#3498db"
      },
      {
        "_id" : ObjectId("5b142ab7e419614016b8992d"),
        "name" : "Shitake",
        "color" : "#95a5a6"
      },
    ]
  },
  {
    "_id": ObjectId("5b1b1ad07325c4c541e8a972"),
    "entries" : [
      {
        "_id" : ObjectId("5b1b1b267325c4c541e8a973"),
        "otherField": "otherData",
        "tags" : [
          ObjectId("5b142608e419614016b89925"),
          ObjectId("5b142ab7e419614016b8992d")
        ]
      },
    ],
    "tags" : [
      {
        "_id" : ObjectId("5b142608e419614016b89925"),
        "name" : "Outdated",
        "color" : "#3498db"
      },
      {
        "_id" : ObjectId("5b142ab7e419614016b8992d"),
        "name" : "Shitake",
        "color" : "#95a5a6"
      },
    ]
  }
];

这将提供完整输出(使用支持的MongoDB 3.6实例的可选输出):

Mongoose: projects.remove({}, {})
Mongoose: projects.insertMany([ { entries: [ { _id: 5b159ebb0ed51064925dff24, someField: 'someData', tags: [ 5b142ab7e419614016b8992d ] } ], _id: 5ae5afc93e1d0d2965a4f2d7, tags: [ { _id: 5b142608e419614016b89925, name: 'Outdated', color: '#3498db' }, { _id: 5b142ab7e419614016b8992d, name: 'Shitake', color: '#95a5a6' } ], __v: 0 }, { entries: [ { _id: 5b1b1b267325c4c541e8a973, otherField: 'otherData', tags: [ 5b142608e419614016b89925, 5b142ab7e419614016b8992d ] } ], _id: 5b1b1ad07325c4c541e8a972, tags: [ { _id: 5b142608e419614016b89925, name: 'Outdated', color: '#3498db' }, { _id: 5b142ab7e419614016b8992d, name: 'Shitake', color: '#95a5a6' } ], __v: 0 } ], {})
Mongoose: projects.aggregate([ { '$project': { entries: { '$map': { input: '$entries', as: 'e', in: { someField: '$$e.someField', otherField: '$$e.otherField', tags: { '$map': { input: '$$e.tags', as: 't', in: { '$arrayElemAt': [ '$tags', { '$indexOfArray': [Array] } ] } } } } } } } } ], {})
{
  "name": "Standard $project $map",
  "results": [
    {
      "_id": "5ae5afc93e1d0d2965a4f2d7",
      "entries": [
        {
          "someField": "someData",
          "tags": [
            {
              "_id": "5b142ab7e419614016b8992d",
              "name": "Shitake",
              "color": "#95a5a6"
            }
          ]
        }
      ]
    },
    {
      "_id": "5b1b1ad07325c4c541e8a972",
      "entries": [
        {
          "otherField": "otherData",
          "tags": [
            {
              "_id": "5b142608e419614016b89925",
              "name": "Outdated",
              "color": "#3498db"
            },
            {
              "_id": "5b142ab7e419614016b8992d",
              "name": "Shitake",
              "color": "#95a5a6"
            }
          ]
        }
      ]
    }
  ]
}
Mongoose: projects.aggregate([ { '$project': { entries: { '$map': { input: '$entries', as: 'e', in: { '$mergeObjects': [ '$$e', { tags: { '$map': { input: '$$e.tags', as: 't', in: { '$arrayElemAt': [Array] } } } } ] } } } } } ], {})
{
  "name": "With $mergeObjects",
  "results": [
    {
      "_id": "5ae5afc93e1d0d2965a4f2d7",
      "entries": [
        {
          "_id": "5b159ebb0ed51064925dff24",
          "someField": "someData",
          "tags": [
            {
              "_id": "5b142ab7e419614016b8992d",
              "name": "Shitake",
              "color": "#95a5a6"
            }
          ]
        }
      ]
    },
    {
      "_id": "5b1b1ad07325c4c541e8a972",
      "entries": [
        {
          "_id": "5b1b1b267325c4c541e8a973",
          "otherField": "otherData",
          "tags": [
            {
              "_id": "5b142608e419614016b89925",
              "name": "Outdated",
              "color": "#3498db"
            },
            {
              "_id": "5b142ab7e419614016b8992d",
              "name": "Shitake",
              "color": "#95a5a6"
            }
          ]
        }
      ]
    }
  ]
}
Mongoose: projects.find({}, { fields: {} })
{
  "name": "Client re-map",
  "results": [
    {
      "_id": "5ae5afc93e1d0d2965a4f2d7",
      "__v": 0,
      "entries": [
        {
          "_id": "5b159ebb0ed51064925dff24",
          "someField": "someData",
          "tags": [
            {
              "_id": "5b142ab7e419614016b8992d",
              "name": "Shitake",
              "color": "#95a5a6"
            }
          ]
        }
      ]
    },
    {
      "_id": "5b1b1ad07325c4c541e8a972",
      "__v": 0,
      "entries": [
        {
          "_id": "5b1b1b267325c4c541e8a973",
          "otherField": "otherData",
          "tags": [
            {
              "_id": "5b142608e419614016b89925",
              "name": "Outdated",
              "color": "#3498db"
            },
            {
              "_id": "5b142ab7e419614016b8992d",
              "name": "Shitake",
              "color": "#95a5a6"
            }
          ]
        }
      ]
    }
  ]
}

请注意,这包括一些其他数据,用于演示&#34;变量字段&#34;。

的投影