Some benchmarks on my Dual X5690
This is using tpbench:
Code:
File='G:\tmp\data1' Length=41918956 element size=4. detected simd=sse3
tp_byte 3847.51 4094.03
tp_nibble 3606.50 3796.99
memcpy 3592.37
This is embedded in my application:
Code:
size = 40 MB, type = float, numIter = 20, calculating inter-quartile mean
copy (memcpy) = 0.021968 sec :: 1820 MB/s :: comp ratio LZ4 100.39%
blosc (shuffle) = 0.050594 sec :: 790 MB/s :: comp ratio LZ4 89.59%
turtt (tpenc) = 0.021133 sec :: 1892 MB/s :: comp ratio LZ4 89.60%
blosc (unshuff) = 0.053062 sec :: 753 MB/s
turtt (tpdec) = 0.019419 sec :: 2059 MB/s
And a larger file:
Code:
size = 1108 MB, type = double, numIter = 20, calculating inter-quartile mean
copy (memcpy) = 0.603125 sec :: 1837 MB/s :: comp ratio LZ4 100.39%
blosc (shuffle) = 1.882178 sec :: 589 MB/s :: comp ratio LZ4 87.93%
turtt (tpenc) = 0.664861 sec :: 1667 MB/s :: comp ratio LZ4 87.93%
blosc (unshuff) = 1.523911 sec :: 727 MB/s
turtt (tpdec) = 0.572066 sec :: 1937 MB/s
My application calls C++ from Matlab, and includes a malloc.
Either way, turbotranspose blows blosc out of the water for me. Many congratulations - this is excellent.