Online demos

ZFP wavelet compression

OIV Xtras - ZFP Wavelet Compression

Open Inventor volume rendering (VolumeViz) provides a native file format called LDM.  The LDM format was designed over ten years ago and introduced the concept of storing volume data as multi-resolution tiles.  This innovation allows random access to all data, very fast display of initial (approximately correct) images, and “best possible” rendering on any size hardware. 

VolumeViz supports the concept of data compression in LDM using a plug-in architecture to allow enhancement and customization by our customers.  We currently support compression using the gzip, JPEG, and JPEG3D algorithms.  Gzip is currently the best option, but gzip only supports lossless compression and is not optimized for numeric data. Modern wavelet compression algorithms, designed for numeric data, can provide much higher compression ratios with “near-lossless” and “visually lossless” results.  JPEG is not very useful because it is lossy only, introduces artifacts, and only supports 8-bit data.  JPEG3D is not very useful because it does not support floating-point data, and decompression is very slow.

Introducing: Data compression for LDM using the “zfp” library

Zfp is an open-source data compression library developed at LLNL for HPC applications.  Zfp is designed to maximize both compression performance and compression ratio for numeric values.  In addition to lossless compression, zfp provides lossy but error-bounded compression.  Zfp lossy compression substantially reduces data size without introducing visible artifacts, as you see with JPEG algorithms.  Zfp is the compression algorithm used in the ZGY file format via the OpenZGY API.  There is an in-depth discussion here.  Zfp is also used in Equinor’s seismic-zfp format.  So we know that zfp is a good solution for seismic data.

Overall results:

  • Zfp is typically 4X faster than gzip for float data.
  • Zfp lossless compression ratios are similar to gzip.
  • Zfp visually lossless compression ratios are 2X to 3X better than gzip.
  • Exception: Zfp lossless compression is less good for 8-bit data.

Some specific results:

Notes: gzip compression level 3 is typical, and Zfp compression level 40 is a visually lossless PSNR.  All sizes in bytes.

 OriginalGzip level 3Zfp level 40
8-bit data
Engine
9,011,7123,720,192
58.7%
0.5 sec
3,016,192
66.5%
0.5 sec
16-bit data
DicomSample
398,461,440177,954,304
55.3%
11.2 sec
12,130,304
97.0%
6.6 sec
Float data
F3_entire (seismic)
1,853,888,000854,443,008
53.9%
71 sec
380,078,080
79.5%
15 sec
Float data
Kahu (seismic)
8,346,689,0246,657,977,344
20.2%
290.5 sec
1,417,525,248
83.0%
79.8 sec
16-bit data
CGG (seismic)
28,852,782,59220,715,173,888
27%
very long…
4,546,709,504
84.2%
780 sec

 

Usage:

Adding zfp compression support in a C++ application is very easy: Just include “ZfpDataCompressor.h” and call the method ZfpDataCompressor::initClass().

  • The zfp compressor is now registered with VolumeViz.
  • Compression
    • You can now use the LDMConverter (after rebuilding it) to create an LDM file with zfp compression using the command line argument “-c zfp”.
    • You can now use SoConverter to create an LDM file by calling the method SoConverterParameters::setCompressionName(“zfp”)
  • Decompression
    • VolumeViz will automatically select the zfp decompressor if the LDM file header specifies “zfp” as compression.

The zfp data compressor header and source files are part of the OIV Xtras project.  To get them please contact your Product Application Specialist (PAS) or account manager.