Question

我有一个查询，每个查询需要运行300个以上的循环。即使在新数据库上，也要花费大约10秒钟才能完成通话。 WebAPI调用是不可接受的。

var isAbnormal = false;
var list = new List<String>();

//Check date range, log them & return if there is any abnormal.
foreach (DateTime day in DateHelper.EachDay(startDate, endDate))
{
  var isActive = db.Operations.Any(x=>x.IsActive && x.Day == day);
  var object;
  var queryable = db.ObjectA.Where(x=>x.Day == day);

  if(isActive){
    queryable = db.ObjectA.First(x=>x.Day == day);

  LogUtil.Info($"{object.Name}");

  var isLogicACorrect = queryable.Any(x=>x.ObjectACount == 5);
  var isLogicBCorrect = queryable.Any(x=>x.ObjectBCount == 3);
  var isLogicCCorrect = queryable.Any(x=>x.ObjectCCount == 2);
  var isLogicDCorrect = queryable.Any(x=>x.ObjectDCount == 8);
  var isLogicECorrect = queryable.Any(x=>x.ObjectECount == 1);
  if(!isLogicACorrect){
    list.Add("Logic A is incorrect");
    isAbnormal = true;
    }

  //More logic codes & db calls here, which is just to select & validate.

}
return list;

如何通过将所有查询合并为一个来优化速度？除了一天外，循环的内容实际上是相同的。每个循环总共要调用15个查询，一个完整的循环中要查询4500个db查询。

Answer 1

根据集合和关系数据而不是程序来考虑。从代码中准确地确定您想要的是不容易的（这与自身矛盾-queryable是通过调用db.ObjectA.Where(...)来设置的，而IQueryable<ObjectA>是db.ObjectA.First(...)，但是它也是通过调用来设置的到ObjectA，即IQueryable<ObjectA>；我假设您想要queryable.Any(...)，因为以后的代码引用了var days = DateHelper.EachDay( startDate, endDate ); var activeDaysIsLogicCorrectFlags = db.Operations // get days that are "active" .Where( op => op.IsActive && days.Contains( op.Day ) ) // join with ObjectA's to filter for active ObjectA's // is there a nav property you could use instead? // use GroupJoin for use with `Any(...)` in results .GroupJoin( db.ObjectA, op => op.Day, oa => oa.Day, ( op, oaGroup ) => new { //Operation = op, // projecting Operation.Day since that's what your foreach loop is using Day = op.Day, IsLogicACorrect = oaGroup.Any( oa => oa.ObjectACount == 5 ), // if IsLogicBCorrect can be determined from the collection of ObjectA's: //IsLogicBCorrect = oaGroup.Any( oa => oa.ObjectBCount == 3 ), } );），但这是我的猜测：

IQueryable

结果是一个匿名类型的Operation.Day，它会将“活动的” IsLogicACorrect与您对IsLogicXCorrect的逻辑进行映射。对于您的其他ObjecetA标志，如果可以使用aoGroup组GroupJoin来确定它们，只需将它们添加到ObjectB结果选择器中（如注释掉所示）属性）。如果这些标志需要它们自己的分组（例如，需要使用IsLogicBCorrect组来确定GroupJoin，则如上所述将附加调用添加到DbSet，但要使用它们各自的db.ObjectB和属性例如，如果您需要将IsLogicBCorrect用于var activeDaysIsLogicCorrectFlags = <existing logic from above> .GroupJoin( db.ObjectB, at => at.Day, ob => ob.Day, ( at, obGroup ) => new { // project all previous results at.Day, at.IsLogicACorrect, // new flag IsLogicBCorrecet = obGroup.Any( ob => ob.ObjectBCount == 3 ), } );：

const fs = require('fs');
const puppeteer = require('puppeteer');
function extractItems() {
  const extractedElements = document.querySelectorAll('.price');
  const items = [];
  for (let element of extractedElements) {
    items.push(element.innerText);
  }
  return items;
}
async function scrapeInfiniteScrollItems(
  page,
  extractItems,
  itemTargetCount,
  scrollDelay = 1000,
) {
  let items = [];
  try {
    let previousHeight;
    while (items.length < itemTargetCount) {
      items = await page.evaluate(extractItems);
      previousHeight = await page.evaluate('document.body.scrollHeight');
      await page.evaluate('window.scrollTo(0, document.body.scrollHeight)');
      await page.waitForFunction(`document.body.scrollHeight > ${previousHeight}`);
      await page.waitFor(scrollDelay);
    }
  } catch(e) { }
  return items;
}
(async () => {
  // Set up browser and page.
  const browser = await puppeteer.launch({
    headless: false,
    args: ['--no-sandbox', '--disable-setuid-sandbox'],
  });
  const page = await browser.newPage();
  page.setViewport({ width: 1280, height: 926 });
  // Navigate to the demo page.
  await page.goto('https://www.clubfactory.com/views/product.html?categoryId=53&subId=53&filter=%7B%22Price%22%3A%5B%7B%22beg%22%3A1.32%2C%22end%22%3A0%7D%5D%7D');
  // Scroll and extract items from the page.
  const items = await scrapeInfiniteScrollItems(page, extractItems, 4000);
  // Save extracted items to a file.
  fs.writeFileSync('./prices3.txt', items.join('\n') + '\n');
  // Close the browser.
  await browser.close();
})();

实体框架将多个查询组合为一个

1 个答案: