MPC v1.1
MPC is a GPUbased lossless compressor/decompressor written in CUDA for binary IEEE 754 32bit singleprecision (float) and 64bit doubleprecision (double) floatingpoint data. It outperforms many other lossless algorithms when compressing weights of DNNs, both in compression ratio and in speed.
Click on MPC_float_11.cu or MPC_double_11.cu to download the source code. A description of MPC is available here. Sample littleendian doubleprecision datasets are available here. Note that MPC is protected by this license and that by downloading MPC you agree to the terms and conditions set forth in this license.
Important notice: The provided code is not necessarily meant to be used as is. (While it should work correctly, it is slow due to PCI data transfers.) Rather, it is meant as an example of how to invoke the compression and decompression kernels directly from your own code.
The source code can be compiled as follows:
nvcc O3 arch=sm_30 MPC_float_11.cu o MPC_float
nvcc O3 arch=sm_30 MPC_double_11.cu o MPC_double
For GPUs that support higher compute capabilities, the sm_30 should be adjusted accordingly.
To compress the singleprecision file single.bin with a dimensionality of 1, enter:
./MPC_float single.bin 1
This generates a compressed file called single.bin.mpc . To decompress this file, enter:
./MPC_float single.bin.mpc
This, in turn, generates a decompressed file called single.bin.mpc.org .
Note that the input files have to be a multiple of 4 bytes long for singleprecision data and a multiple of 8 bytes long for doubleprecision data and should contain nothing but binary values. Only littleendian systems are currently supported.
Publications
A. Yang, H. Mukka, F. Hesaaraki, and M. Burtscher. "MPC: A Massively Parallel Compression Algorithm for Scientific Data." IEEE Cluster Conference. September 2015. [pdf] [pptx] [video]
Note that version 1.1 of the code is much faster and compresses slightly better than the version that is described in the above paper and presentation.
