Tangor

PoC of StarPU based Iyokan

CPU build

Tangor builds the CPU path by default. The HOGE/FPGA path remains opt-in via -DUSE_HOGE=ON.

git submodule update --init --recursive
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

Tangor uses thirdparties/cuFHEpp for the TFHEpp checkout at thirdparties/cuFHEpp/thirdparties/TFHEpp. It does not build cuFHEpp's CUDA library for the CPU path.

The default TFHE configuration is Block Binary keys with Subset Keys (USE_BLOCK_BINARY=ON, USE_SUBSET_KEY=ON). Regenerate secret/evaluation keys, encrypted packets, and snapshots after upgrading, since artifacts made with the former defaults are not compatible.

KVSP compatibility

With the default -DTANGOR_BUILD_KVSP_COMPAT=ON, the build produces build/bin/iyokan and build/bin/iyokan-packet in addition to Tangor's native executable. They implement the CLI and cereal packet/archive format expected by KVSP, including plain, tfhe, snapshots, key generation, and packet conversion. Point KVSP at them without changing the KVSP command line:

KVSP_IYOKAN_PATH="$PWD/build/bin/iyokan" \
KVSP_IYOKAN_PACKET_PATH="$PWD/build/bin/iyokan-packet" \
  /path/to/kvsp

The compatibility targets share the checked-out Iyokan frontend sources while linking to Tangor's selected TFHEpp build. This keeps packets, evaluation keys, and snapshots byte-compatible during the scheduler migration. Set -DTANGOR_BUILD_KVSP_COMPAT=OFF for a standalone Tangor-only build. If that checkout is not adjacent to Tangor, CMake fetches the pinned Iyokan frontend (including its submodules) by default; use -DTANGOR_FETCH_IYOKAN_COMPAT=OFF plus TANGOR_IYOKAN_COMPAT_SOURCE_DIR and TANGOR_IYOKAN_COMPAT_THIRDPARTY_DIR for an offline source mirror.

StarPU CPU/GPU execution

Tangor schedules each homomorphic gate as a StarPU task. With CUDA enabled, the same dependency graph contains TFHEpp CPU implementations and cuFHEpp CUDA implementations, so StarPU can place ready gates on both resource types. ROM/RAM CMUX operations are individual StarPU tasks too, but use TFHEpp CPU workers by default: the cuFHEpp FFT CMUX does not yet preserve enough noise margin for the depth-8 encrypted RAM update path. Developers can opt into that kernel with TANGOR_EXPERIMENTAL_CUDA_CMUX=1. No TFHE parameters are changed for runtime compatibility. CPU and CUDA builds both use the split Fourier archive layout, so keys and encrypted ROM/RAM packets are portable across Tangor, Iyokan, and KVSP.

Tangor accepts KVSP's Iyokan CMake cache variables, so KVSP can use Tangor as its source directory without renaming build flags:

git submodule update --init --recursive
cmake -S /path/to/Tangor -B build/Iyokan-avx2 \
  -DCMAKE_BUILD_TYPE=Release \
  -DIYOKAN_ENABLE_CUDA=ON \
  -DIYOKAN_MARCH=x86-64-v3 \
  -DUSE_AVX512=OFF
cmake --build build/Iyokan-avx2 --target iyokan iyokan-packet

To have KVSP build that compatibility target in its normal location:

make -C /path/to/kvsp ENABLE_CUDA=1 IYOKAN_SOURCE=/path/to/Tangor iyokan-avx2

Run with both CPU and GPU resources. For example, this uses 64 CPU workers and two GPUs:

/path/to/kvsp/build/bin/iyokan-avx2 tfhe --enable-gpu --cpu 64 --num-gpu 2 \
  --evalkey eval.key -c 224 -o result.enc --snapshot result.snapshot \
  --blueprint /path/to/kvsp/build/share/kvsp/alexandrite.toml -i fib.enc

--cpu selects the number of StarPU CPU workers, --num-gpu selects CUDA devices, and --enable-gpu enables the CUDA workers. Tangor defaults to the dmdas scheduler. STARPU_SCHED can override it, and STARPU_NWORKER_PER_CUDA controls the number of asynchronous workers per GPU (32 by default). CPU-only execution uses the same graph without CUDA workers.

StarPU diagnostics

Profiling is off by default. Build with -DTANGOR_STARPU_PROFILE=ON to enable StarPU worker and per-task reports plus an FxT trace by default. This option also builds the bundled StarPU with FxT support and requires the fxt development package (included in KVSP's Ubuntu 24.04 build image). Environment variables retain precedence, so reports can be redirected or disabled without rebuilding:

cmake -S /path/to/Tangor -B build/profile \
  -DIYOKAN_ENABLE_CUDA=ON -DTANGOR_STARPU_PROFILE=ON
STARPU_WORKER_STATS_FILE=starpu-workers.txt \
STARPU_FXT_PREFIX="$PWD" build/profile/bin/iyokan tfhe ...

The report distinguishes CPU and CUDA worker execution, sleeping, waiting, and scheduling time, and includes CPU↔GPU transfer statistics. Per-task records identify the gate/CMUX codelet and its queue delay. The FxT trace is written on shutdown (by default under /tmp) and can be converted with the starpu_fxt_tool installed by StarPU.

The evaluator cannot inspect encrypted termination itself. For KVSP's bundled fib(5) input, a plaintext emulator establishes that 224 cycles are required. Decrypt the resulting packet to confirm f0 = true and x10 = 5:

/path/to/kvsp/build/bin/kvsp dec --cpu alexandrite -k secret.key -i result.enc

StarPU gate scheduling is enabled by default. It can be stated explicitly for reproducible builds:

cmake -S /path/to/Tangor -B build/starpu \
  -DIYOKAN_ENABLE_CUDA=ON -DTANGOR_KVSP_STARPU_GATE_OFFLOAD=ON
STARPU_SCHED=dmdas build/starpu/bin/iyokan tfhe \
  --enable-gpu --cpu 64 --num-gpu 2 ...

To use a different cuFHEpp checkout:

cmake -S . -B build -DTANGOR_CUFHEPP_SOURCE_DIR=/path/to/cuFHEpp

To override only TFHEpp:

cmake -S . -B build -DTANGOR_TFHEPP_SOURCE_DIR=/path/to/TFHEpp