Heya ladberg:
* Were you doing direct light sampling (next event estimation)? If so that would explain it. It's the most naive path tracing you can imagine, so it's easier to understand.
* Or, it might be how shadertoy is vsync limited to 60 fps which means each pixel only gets 60 samples a second, when it could be getting ~64x as much before i see any slowdown.