Why is Array.push(element) not working sometimes?

Question

I'm using node.js for a project, and I have this certain structure in my code which is causing problems. I have an array dateArr of sequential dates that contains 106 items. I have an array resultArr to hold resulting data. My code structure is like this:

function grabData(value, index, dateArr) {
  cassandra client execute query with value from dateArr {
    if (!err) {
      if (result has more than 0 rows) {
        process the query data
        push to resultArr
      }
      if (result is empty) {
        push empty set to resultArr
      }
    }
  }
}

dateArr.forEach(grabData);

I logged the size of resultArr after each iteration and it appears that on some iterations nothing is being pushed to resultArr. The code completes with only 66 items stored in resultArr when 106 items should be stored because the I/O structure between dateArr and resultArr is 1 to 1.

Answer 1

I logged the size of resultArr after each iteration When the grabData method gets called you start a query to somewhere, or someone named cassandra. As Felix Kling wrote, your notation seems to show an asynchronous function, that starts the request and returns. As the function is asynchronous, you don't know when the query is ready. That might even take very long, when the database is locked for a dump, or whatever. When you return from grabData "iteration" and check your resultArr, the resultArr will exactly be filled with each returned value. It might even be that the fifth iteration returns a query before the third, or fourth or any iteration before. So in you resultArr you sometimes have values of iteration n at some point m<n or o>n. As long as you (or we) don't know anything about how cassandra operates, you cannot say when a query gets answered. So when you check your result array, it returns the number of completed queries, not the number of iterations.

Answer 2

找到根本原因：使用node.js查询Cassandra时存在硬限制。我试图完全执行的查询太大了。将dateArr分成更小的块并使用那些较小的块查询解决了问题。