Kernel - Parallelism

Once the first kernels from previous examples are working, the next step is to understand how alpaka maps logical work onto frames, blocks, threads, and warps. The important distinction is:

  • IdxRange describes the logical work that must be completed.

  • FrameSpec describes the available parallel structure for one launch.

  • makeIdxMap maps worker threads to valid indices of the logical work.

alpaka kernels can stay data-centric instead of being written around manual global-thread index formulas.

Frames, Blocks, Threads, and Warps

A good mental model is:

  1. Choose the frame extent from the tile shape you want in the data.

  2. Map thread blocks to cover the elements inside a tile/chunk.

  3. Use 1-dimensional warps only when there is a naturally one-dimensional inner direction.

The following kernel uses a small 2D image-style example to show how blocks, threads, and warps relate to one another in practice.

struct ImageTileHierarchyKernel
{
    ALPAKA_FN_ACC void operator()(
        onAcc::concepts::Acc auto const& acc,
        concepts::Vector auto const tileExtent,
        concepts::IDataSource auto const& input,
        concepts::IMdSpan auto mask,
        concepts::IMdSpan auto rowCounts,
        int threshold) const
    {
        auto const imageExtent = input.getExtents();

        for(auto tileStart :
            onAcc::makeIdxMap(acc, onAcc::worker::blocksInGrid, IdxRange{Vec{0u, 0u}, imageExtent, tileExtent}))
        {
            for(auto localIdx : onAcc::makeIdxMap(acc, onAcc::worker::threadsInBlock, IdxRange{tileExtent}))
            {
                auto globalIdx = tileStart + localIdx;
                if(globalIdx[0u] < imageExtent[0u] && globalIdx[1u] < imageExtent[1u])
                {
                    mask[globalIdx] = input[globalIdx] >= threshold ? 1u : 0u;
                }
            }

            for(auto warpRow :
                onAcc::makeIdxMap(acc, onAcc::worker::linearWarpsInBlock, onAcc::range::linearWarpsInBlock))
            {
                auto rowStart = tileStart + Vec{warpRow.x(), 0u};
                if(rowStart[0u] >= imageExtent[0u] || warpRow.x() >= tileExtent[0u])
                {
                    continue;
                }

                for(auto lane :
                    onAcc::makeIdxMap(acc, onAcc::worker::linearThreadsInWarp, onAcc::range::linearThreadsInWarp))
                {
                    auto globalIdx = rowStart + Vec{0u, lane.x()};
                    if(lane.x() < tileExtent[1u] && globalIdx[1u] < imageExtent[1u] && input[globalIdx] >= threshold)
                    {
                        onAcc::atomicAdd(acc, &rowCounts[Vec{rowStart[0u]}], 1u);
                    }
                }
            }
        }
    }
};

The structure is the important part:

  • onAcc::worker::blocksInGrid chooses tile offsets in the full 2D image.

  • onAcc::worker::threadsInBlock iterates the pixels inside one tile.

  • onAcc::worker::linearWarpsInBlock and linearThreadsInWarp reuse the same tile in a one-dimensional way.

Launching a Hierarchical Kernel

onHost::concepts::FrameSpec auto frameSpec = onHost::FrameSpec{divExZero(imageExtent, tileExtent), tileExtent};
queue.enqueue(
    frameSpec,
    KernelBundle{ImageTileHierarchyKernel{}, tileExtent, inputBuffer, maskBuffer, rowCountsBuffer, 5});

Chunked and Tiled Kernels

After the plain element-wise style, the next natural alpaka pattern is a chunked kernel. Here frames stop being just a launch shape and become reusable tiles of work.

struct ChunkedVectorAddKernel
{
    ALPAKA_FN_ACC void operator()(
        onAcc::concepts::Acc auto const& acc,
        auto const linearNumFrames,
        concepts::CVector auto const linearFrameExtent,
        concepts::IMdSpan auto out,
        concepts::IDataSource auto const& in0,
        concepts::IDataSource auto const& in1) const
    {
        for(auto linearFrameIdx : onAcc::makeIdxMap(acc, onAcc::worker::linearBlocksInGrid, IdxRange{linearNumFrames}))
        {
            auto tile = onAcc::declareSharedMdArray<int, uniqueId()>(acc, linearFrameExtent);

            for(auto linearFrameElem :
                onAcc::makeIdxMap(acc, onAcc::worker::linearThreadsInBlock, IdxRange{linearFrameExtent}))
            {
                auto globalIdx = linearFrameIdx * linearFrameExtent + linearFrameElem;
                tile[linearFrameElem] = in0[globalIdx];
            }

            onAcc::syncBlockThreads(acc);

            for(auto linearFrameElem :
                onAcc::makeIdxMap(acc, onAcc::worker::linearThreadsInBlock, IdxRange{linearFrameExtent}))
            {
                auto globalIdx = linearFrameIdx * linearFrameExtent + linearFrameElem;
                out[globalIdx] = tile[linearFrameElem] + in1[globalIdx];
            }

            onAcc::syncBlockThreads(acc);
        }
    }
};

There are a few moving parts in this pattern:

  • linearBlocksInGrid lets blocks iterate over frames.

  • linearThreadsInBlock lets threads iterate over elements inside one frame.

    constexpr auto frameExtent = CVec<uint32_t, 4u>{};
auto const totalElems = static_cast<uint32_t>(hostOut.size());
auto const frameElementCount = frameExtent.product();
REQUIRE(totalElems % frameElementCount == 0u);
auto numFrames = Vec{totalElems / frameElementCount};
onHost::concepts::FrameSpec auto frameSpec = onHost::FrameSpec{numFrames, frameExtent};

queue.enqueue(
    frameSpec,
    KernelBundle{ChunkedVectorAddKernel{}, numFrames.product(), frameExtent, outBuffer, in0Buffer, in1Buffer});

Practical Advice

  • Start with unnested makeIdxMap when the kernel is just “process every element once”.

  • Use hierarchy chunked kernels when there is real data reuse or tiled traversal.

  • Treat warps as one-dimensional helpers inside a block, not as a replacement for multidimensional mapping.

There are cases where explicit thread or block indices can be useful, for example:

  • Implementing a very specific CPU/GPU mapping.

  • Using an algorithm that must reason about exact block-local cooperation.

  • Porting low-level CUDA/HIP code step by step.

That is not the best starting point for most kernels. For portable code, prefer using FrameSpec with makeIdxMap. Once the algorithm is correct and tested, you can move to more specialized mappings if profiling shows that you need them.

Complete Source File

090_kernelParallelism.cpp
  1/* Copyright 2026 René Widera
  2 * SPDX-License-Identifier: ISC
  3 */
  4
  5#include "docsTest.hpp"
  6
  7#include <alpaka/alpaka.hpp>
  8
  9#include <catch2/catch_template_test_macros.hpp>
 10#include <catch2/catch_test_macros.hpp>
 11
 12#include <array>
 13
 14using namespace alpaka;
 15
 16struct ImageTileHierarchyKernel
 17{
 18    ALPAKA_FN_ACC void operator()(
 19        onAcc::concepts::Acc auto const& acc,
 20        concepts::Vector auto const tileExtent,
 21        concepts::IDataSource auto const& input,
 22        concepts::IMdSpan auto mask,
 23        concepts::IMdSpan auto rowCounts,
 24        int threshold) const
 25    {
 26        auto const imageExtent = input.getExtents();
 27
 28        for(auto tileStart :
 29            onAcc::makeIdxMap(acc, onAcc::worker::blocksInGrid, IdxRange{Vec{0u, 0u}, imageExtent, tileExtent}))
 30        {
 31            for(auto localIdx : onAcc::makeIdxMap(acc, onAcc::worker::threadsInBlock, IdxRange{tileExtent}))
 32            {
 33                auto globalIdx = tileStart + localIdx;
 34                if(globalIdx[0u] < imageExtent[0u] && globalIdx[1u] < imageExtent[1u])
 35                {
 36                    mask[globalIdx] = input[globalIdx] >= threshold ? 1u : 0u;
 37                }
 38            }
 39
 40            for(auto warpRow :
 41                onAcc::makeIdxMap(acc, onAcc::worker::linearWarpsInBlock, onAcc::range::linearWarpsInBlock))
 42            {
 43                auto rowStart = tileStart + Vec{warpRow.x(), 0u};
 44                if(rowStart[0u] >= imageExtent[0u] || warpRow.x() >= tileExtent[0u])
 45                {
 46                    continue;
 47                }
 48
 49                for(auto lane :
 50                    onAcc::makeIdxMap(acc, onAcc::worker::linearThreadsInWarp, onAcc::range::linearThreadsInWarp))
 51                {
 52                    auto globalIdx = rowStart + Vec{0u, lane.x()};
 53                    if(lane.x() < tileExtent[1u] && globalIdx[1u] < imageExtent[1u] && input[globalIdx] >= threshold)
 54                    {
 55                        onAcc::atomicAdd(acc, &rowCounts[Vec{rowStart[0u]}], 1u);
 56                    }
 57                }
 58            }
 59        }
 60    }
 61};
 62
 63
 64TEMPLATE_LIST_TEST_CASE("tutorial hierarchy blocks threads warps", "[docs]", docs::test::TestBackends)
 65{
 66    auto selector = onHost::makeDeviceSelector(TestType::makeDict());
 67    if(!selector.isAvailable())
 68        return;
 69    onHost::concepts::Device auto device = selector.makeDevice(0);
 70    onHost::Queue queue = device.makeQueue(queueKind::blocking);
 71
 72    auto const warpSize = device.getDeviceProperties().warpSize;
 73    auto const imageExtent = Vec{4u, 2u * warpSize};
 74    auto const tileExtent = Vec{1u, warpSize};
 75
 76    auto hostInput = onHost::allocHost<int>(imageExtent);
 77    auto hostMask = onHost::allocHost<uint32_t>(imageExtent);
 78    auto hostRowCounts = onHost::allocHost<uint32_t>(Vec{imageExtent[0u]});
 79
 80    for(auto idx : IdxRange{imageExtent})
 81    {
 82        if(idx[0u] == 0u)
 83        {
 84            hostInput[idx] = 10;
 85        }
 86        else if(idx[0u] == 1u)
 87        {
 88            hostInput[idx] = idx[1u] < warpSize ? 0 : 10;
 89        }
 90        else if(idx[0u] == 2u)
 91        {
 92            hostInput[idx] = (idx[1u] % 2u == 0u) ? 10 : 0;
 93        }
 94        else
 95        {
 96            hostInput[idx] = 0;
 97        }
 98    }
 99
100    auto inputBuffer = onHost::allocLike(device, hostInput);
101    auto maskBuffer = onHost::allocLike(device, hostMask);
102    auto rowCountsBuffer = onHost::allocLike(device, hostRowCounts);
103
104    onHost::memcpy(queue, inputBuffer, hostInput);
105    onHost::fill(queue, rowCountsBuffer, 0u);
106
107    onHost::concepts::FrameSpec auto frameSpec = onHost::FrameSpec{divExZero(imageExtent, tileExtent), tileExtent};
108    queue.enqueue(
109        frameSpec,
110        KernelBundle{ImageTileHierarchyKernel{}, tileExtent, inputBuffer, maskBuffer, rowCountsBuffer, 5});
111
112    onHost::memcpy(queue, hostMask, maskBuffer);
113    onHost::memcpy(queue, hostRowCounts, rowCountsBuffer);
114    onHost::wait(queue);
115
116    for(auto idx : IdxRange{imageExtent})
117    {
118        if(idx[0u] == 0u)
119        {
120            CHECK(hostMask[idx] == 1u);
121        }
122        else if(idx[0u] == 1u)
123        {
124            CHECK(hostMask[idx] == (idx[1u] < warpSize ? 0u : 1u));
125        }
126        else if(idx[0u] == 2u)
127        {
128            CHECK(hostMask[idx] == (idx[1u] % 2u == 0u ? 1u : 0u));
129        }
130        else
131        {
132            CHECK(hostMask[idx] == 0u);
133        }
134    }
135
136    CHECK(hostRowCounts[0u] == 2u * warpSize);
137    CHECK(hostRowCounts[1u] == warpSize);
138    CHECK(hostRowCounts[2u] == warpSize);
139    CHECK(hostRowCounts[3u] == 0u);
140}
141
142struct ChunkedVectorAddKernel
143{
144    ALPAKA_FN_ACC void operator()(
145        onAcc::concepts::Acc auto const& acc,
146        auto const linearNumFrames,
147        concepts::CVector auto const linearFrameExtent,
148        concepts::IMdSpan auto out,
149        concepts::IDataSource auto const& in0,
150        concepts::IDataSource auto const& in1) const
151    {
152        for(auto linearFrameIdx : onAcc::makeIdxMap(acc, onAcc::worker::linearBlocksInGrid, IdxRange{linearNumFrames}))
153        {
154            auto tile = onAcc::declareSharedMdArray<int, uniqueId()>(acc, linearFrameExtent);
155
156            for(auto linearFrameElem :
157                onAcc::makeIdxMap(acc, onAcc::worker::linearThreadsInBlock, IdxRange{linearFrameExtent}))
158            {
159                auto globalIdx = linearFrameIdx * linearFrameExtent + linearFrameElem;
160                tile[linearFrameElem] = in0[globalIdx];
161            }
162
163            onAcc::syncBlockThreads(acc);
164
165            for(auto linearFrameElem :
166                onAcc::makeIdxMap(acc, onAcc::worker::linearThreadsInBlock, IdxRange{linearFrameExtent}))
167            {
168                auto globalIdx = linearFrameIdx * linearFrameExtent + linearFrameElem;
169                out[globalIdx] = tile[linearFrameElem] + in1[globalIdx];
170            }
171
172            onAcc::syncBlockThreads(acc);
173        }
174    }
175};
176
177
178TEMPLATE_LIST_TEST_CASE("tutorial chunked frames kernel", "[docs]", docs::test::TestBackends)
179{
180    auto selector = onHost::makeDeviceSelector(TestType::makeDict());
181    if(!selector.isAvailable())
182        return;
183    onHost::concepts::Device auto device = selector.makeDevice(0);
184    onHost::Queue queue = device.makeQueue(queueKind::blocking);
185
186    std::array<int, 8u> hostIn0{0, 1, 2, 3, 4, 5, 6, 7};
187    std::array<int, 8u> hostIn1{10, 10, 10, 10, 10, 10, 10, 10};
188    std::array<int, 8u> hostOut{};
189
190    auto in0Buffer = onHost::allocLike(device, hostIn0);
191    auto in1Buffer = onHost::allocLike(device, hostIn1);
192    auto outBuffer = onHost::allocLike(device, hostIn0);
193
194    onHost::memcpy(queue, in0Buffer, hostIn0);
195    onHost::memcpy(queue, in1Buffer, hostIn1);
196
197    constexpr auto frameExtent = CVec<uint32_t, 4u>{};
198    auto const totalElems = static_cast<uint32_t>(hostOut.size());
199    auto const frameElementCount = frameExtent.product();
200    REQUIRE(totalElems % frameElementCount == 0u);
201    auto numFrames = Vec{totalElems / frameElementCount};
202    onHost::concepts::FrameSpec auto frameSpec = onHost::FrameSpec{numFrames, frameExtent};
203
204    queue.enqueue(
205        frameSpec,
206        KernelBundle{ChunkedVectorAddKernel{}, numFrames.product(), frameExtent, outBuffer, in0Buffer, in1Buffer});
207
208    onHost::memcpy(queue, hostOut, outBuffer);
209    onHost::wait(queue);
210
211    CHECK(hostOut[0] == 10);
212    CHECK(hostOut[1] == 11);
213    CHECK(hostOut[2] == 12);
214    CHECK(hostOut[3] == 13);
215    CHECK(hostOut[4] == 14);
216    CHECK(hostOut[5] == 15);
217    CHECK(hostOut[6] == 16);
218    CHECK(hostOut[7] == 17);
219}