I played around with SIMD in Kotlin inside a multi-threaded Mandelbrot generator. My implementation was ~40% slower than the full scalar approach. What happened?

SIMD⌗

If you don’t know SIMD instructions, here’s a very short summary. Say you want to multiply 2 lists:

a = [1, 2, 3, 4]
b = [5, 6, 7, 8]

# Multiply each element
# 1*5, 2*6, 3*7, 4*8
res = []
for i in range(len(a)):
    res.append(a[i] * b[i])

With SIMD, you can pack the 4 values into a wide CPU register, and multiply with a single instruction:

import numpy as np

a = np.array([1, 2, 3, 4], dtype=np.float32)
b = np.array([5, 6, 7, 8], dtype=np.float32)

# SIMD here, the whole multiplication was done in a single instruction
result = a * b

The idea is to execute the same operation with variable operands. Pure data parallelisation.

In Kotlin⌗

I experimented with the Vector API inside a multi-threaded Mandelbrot generator. The perfect playground: the computation of each pixel is independent, and completely mathematical.

After spending (a long) time converting my scalar code to SIMD, I benchmarked it. Roughly 40% SLOWER 🤯. I dug around, and found this Netflix blog post. The short version is: if you are in Kotlin, you should write your SIMD operations in Java.

Why?

If you write this in Kotlin:

val zx2 = zx.mul(zy)

The bytecode will look roughly like this:

aload         7                    // load zx
aload         8                    // load zy  
checkcast     jdk/incubator/vector/Vector   // CAST zy to Vector
invokevirtual DoubleVector.mul     // zx.mul(zy)

The problem lies in checkcast. The Vector API relies on JIT intrinsics to emit CPU vector instructions. Without intrinsification, it falls back to scalar execution. Kotlin is stricter about variance/generics than Java, and inserts a checkcast instruction that prevents the JIT from applying those intrinsics. The JIT can no longer intrinsify each SIMD operation (sum, mul, blend, etc).

Compare it with the Java version:

DoubleVector zx2 = zx.mul(zy);

And the bytecode:

aload         7                    // load zx
aload         8                    // load zy
invokevirtual DoubleVector.mul     // intrinsified to vmulpd

Intrinsification means that the JIT compiler replaces the invokevirtual by a direct CPU instruction. This is what we want for SIMD.

The solution⌗

I rewrote only the pure SIMD part of the algorithm in Java and benchmarked it again… 4x faster than the scalar version 🤯🤯🤯.

The performance gain will vary depending on your CPU and the JIT’s vectorization capabilities, but that was just amazing. What a ride!