MPC v1.2
MPC is a GPUbased lossless compressor/decompressor written in CUDA for binary IEEE 754 32bit singleprecision (float) and 64bit doubleprecision (double) floatingpoint data. It outperforms many other lossless compression algorithms, including when compressing weights of DNNs, both in compression ratio and in speed.
Click on MPC_float_12.cu or MPC_double_12.cu to download the source code. A description of MPC is available here. Sample littleendian doubleprecision datasets are available here. Sample littleendian singleprecision datasets are available here. Note that MPC is protected by the license included in the beginning of the code.
Important notice: The provided code is not necessarily meant to be used as is. (While it should work correctly, it is slow due to PCI data transfers.) Rather, it is meant as an example of how to invoke the compression and decompression kernels directly from your own code.
The source code can be compiled as follows:
nvcc O3 arch=sm_35 MPC_float_12.cu o MPC_float
nvcc O3 arch=sm_35 MPC_double_12.cu o MPC_double
For GPUs that support higher compute capabilities, the sm_35 should be adjusted accordingly.
To compress the singleprecision file single.bin with a dimensionality of 1, enter:
./MPC_float single.bin 1
This generates a compressed file called single.bin.mpc . To decompress this file, enter:
./MPC_float single.bin.mpc
This, in turn, generates a decompressed file called single.bin.mpc.org .
Note that the input files have to be a multiple of 4 bytes long for singleprecision data and a multiple of 8 bytes long for doubleprecision data and should contain nothing but binary values. Only littleendian systems are currently supported.
Publications
A. Yang, H. Mukka, F. Hesaaraki, and M. Burtscher. "MPC: A Massively Parallel Compression Algorithm for Scientific Data." IEEE Cluster Conference. September 2015. [pdf] [pptx] [video]
Note that version 1.2 of the code is much faster and compresses slightly better than the version that is described in the above paper and presentation.
This work has been supported in part by the National Science Foundation, Texas State University, and by equipment donations from Nvidia Corporation.
