CUDA gridDim, blockDim and threadIdx

Question

This is a conceptual question. In CUDA, gridDim, blockDim and threadIdx can be 1D, 2D or 3D. I wonder, how are their 2D and 3D versions interpreted?

In more details, does CUDA think of multi-dimensional gridDim, blockDim and threadIdx just as a linear sequence, in the same way that C stores multi-dimensional array? If not, how should we interpret multi-dimensional gridDim, blockDim and threadIdx?

Thanks.

Edit 1. This question is not a duplicated one. I actually have come across the referred question. It asks about the order of execution of the GPU threads, not their layouts, as this one does.

Edit 2. Also, the answer to this question can be found at http://docs.nvidia.com/cuda/cuda-c-programming-guide/#thread-hierarchy. Thank you @talonmies, for the reference. To sum it up, multi-dimensional gridDim, blockDim and threadIdx is for convenience purposes. They can be interpreted just like a column major ordered multi-dimensional array.

Answer 1

直接从CUDA programming guide

报价

  

线程的索引及其线程ID在a中相互关联   直截了当的方式：对于一维块，它们是相同的;   对于大小的二维块（Dx，Dy），线程的线程ID   指数（x，y）是（x + y Dx）;对于三维块大小   （Dx，Dy，Dz），索引（x，y，z）的线程的线程ID是（x + y）   Dx + z Dx Dy）。

所以，是的，编程模型中的逻辑线程编号是顺序的，然后x维度变化最快，然后是y维度，然后是z维度。这适用于块内的线程编号和网格中的块编号。编号类似于column major有序多维数组，尽管实际的threadIdx和blockIdx变量本身只是反映内部线程和块标识字的结构，由调度程序分配给每个线程或块。

您应该注意threadIdx和blockIdx隐含的编号仅供程序员使用，并不意味着GPU上线程的执行顺序。

Answer 2

In more details, does CUDA think of multi-dimensional gridDim, blockDim and threadIdx just as a linear sequence, in the same way that C stores multi-dimensional array? Yes. All multidimensional arrays are linearized in C. They are linearized in row-major order--place all elements of the same row into consecutive locations, the rows are then placed one after another into the memory space. CUDA C also uses row-major layout. An example of 2D array layout: