Over the past few decades, the amount of information available for analysis has increased significantly. Scientific instruments and related computation systems, such as the Linac Coherent Light Source [1], the Very Large Array Radio Telescope [2], and high-resolution climate modeling [3], have produced massive amounts of data, putting a huge burden on the existing storage system. Therefore, it is important to design efficient compression models that are able to reduce the data size for storage while maintaining the key information for analysis.

Data compression can be lossless and lossy. Lossless compression, whose reconstruction is exactly the same as the original data, suffers from a low compression ratio (around 2:1 [4]) in floating point data sets [5,6]. Meanwhile, lossy compression removes imperceptible details to achieve a much higher compression ratio. Despite the loss of information, the quality of the data reconstructed by lossy compression schemes is generally acceptable and usable [7]. The nature of lossy compression has led scientists and engineers to implement many compression algorithms and methods to substantially reduce the size of scientific data [8,9], whose size is often enormous (up to 32 exabytes [10]). Furthermore, recent studies by [11,12], and [13] showed that lossy compression reconstruction data can be used for post hoc analyses.

In recent years, significant attention from scientific and engineering communities has been directed towards the advancement of neural network models across various domains such as computer vision [14], natural language processing [15,16], and compression [17]. Among numerous types of deep learning architectures, Autoencoder (AE) has gained tremendous attention because of its capability to learn data representation. AE demonstrates proficiency in unsupervised learning of data representations for reconstruction purposes. Internally, the network contains a bottleneck layer, whose representation is much smaller than its inputs in terms of size. Consequently, AE finds primary application in dimensionality reduction and feature extraction tasks. Numerous AE adaptations have surfaced to enhance the fidelity of reconstructed data [18,19]. While AE has exhibited success in lossy compression of images and videos [20], its utilization in scientific data compression remains relatively under-explored. Despite the limited literature on scientific data compression, AE-based compression methodologies have demonstrated notable compression ratios, surpassing many traditional non-machine learning-based techniques [21].

Inspired by the studies by [21] and [22], our objective is to develop a machine learning approach to substantially reduce the storage footprint of scientific data, known for its substantial space requirements. Thus, we investigate the potential of employing a lossy AE for this task. Our objective is to attain high data reconstruction quality at an ultra-low bit rate, specifically aiming for values below 0.40. We present an AE-based model engineered to significantly reduce the size of the data while preserving the fidelity of the data. The primary contributions of this study are outlined below.

• Targeting a very low bit rate region, we implement our own architecture to significantly compress our simulation data from high-resolution HR-CESM1.3 data sets. Benchmark results are also presented to ensure the performance of our model.

• We incorporate masking layers and several preprocessing techniques to significantly improve the compression performance of our model.

The remainder of this paper is organized as follows. In Section 2, related work is discussed. In Section 3, we describe important concepts and techniques that are implemented in our proposed models. Section 4 provides available resources for this study. Section 5 describes compression experiments on the benchmark data and our large-scale simulation data. We evaluate and analyze our results in Section 6. Section 7 concludes our findings with directions for future work.

Traditional lossy compression for scientific data could be categorized into two types: prediction-based and transform-based. Transform compression (e.g. ZFP [23]) transformed the data before applying other techniques, e.g. embedded coding, to truncate the transformed data. Coefficients and bit-plans determined by the model were used to decompress the data. Increasing the number of coefficients and bit-planes improved the quality of reconstructed data but decreased the compression ratio.

On the other hand, prediction-based models, such as SZ [8,9], and FPZIP [24], predicted the target data using previously reconstructed data points [8,9]. Similar to transform-based models, the authors of [25] found that the fidelity of the reconstructed data degraded when a high compression ratio was required. Prediction-based models have been shown to have high reconstruction quality at a high compression ratio, which has led to more studies to improve the performance of this type of compression.

Recently, deep learning models have been leveraged to compress many types of data. Many AE-based models showed remarkable results in image and volumetric compression tasks.

Balle et al. [26] introduced an effective end-to-end AE-based model to compress images. The authors trained their models to optimize the performance of rate-distortion. To balance the trade-off between the quality of reconstructed data and compression ratio, both the losses for reconstruction and compression rate were minimized simultaneously. Since the quantization layer of their compression models prevented the gradients from flowing through the networks, independently and identically distributed uniform noise was used to replace the quantization layer during training. The added noise allowed backpropagation without significantly deteriorating the performance of the quantization layer when compressing images.

Models with two levels of quantization were also investigated in [27]. The second layer not only provided fine-grained quantization but also acted as a prior for the first quantization layer. In addition, arithmetic encoding [28,29] was implemented instead of variants of Huffman encoding [30]. Integer quantization, proposed by the authors of [31], was applied for quantization layers to eliminate the dependence on hardware-platform floating-point implementation, which varied from machine to machine, during compression.

Using the idea of two-level quantization, several studies have been conducted to improve the capabilities of neural networks to compress images. Minnen et al. [32] built an autoregressive model. The first quantization layer, which received input from the prior given by the second quantization and from the encoder, autoregressively processed data representation to produce high-quality images. Their neural networks were also among the first machine-learning models that outperformed the state-of-the-art BPG compression [33]. However, autoregression by its nature prevented neural networks from computing in parallel. The models created by [20] eliminated the autoregressive layer and replaced it with multiple splitting layers, allowing the decoder to fully learn different sets of channels in parallel. Furthermore, optimization for compression speed using neural networks was also addressed by [17], which suggested several methods to improve compression performance.

Compression on audio signals using AE-based neural networks has also experienced much progress. The work of [34] outperformed MP3 in both compression ratio and audio quality. Their models adopted the vector quantization techniques proposed by [35]. The authors not only optimized signal losses in the time domain, but also minimized reconstruction losses in the frequency domain. Furthermore, the coupling of AE and Generative Adversarial Networks (GAN) [36] was leveraged to achieve a high-quality compression model.

Neural networks have also been implemented to compress volumetric scene data. Kim et al. [34] replaced fine-grained compression layers in their tree-based models with neural networks, greatly enhancing the performance on volumetric representation. Coordinate networks by [37] not only focused on learning the scene representation but also provided great compression capability.

However, image and video compression models mainly reconstructed integer pixels (or voxels), which were only a subset of scientific data, where data types ranged from integer to floating-point. As a result, several studies have been conducted using neural networks to enhance scientific data compression. Glaws et al. [38] proposed an AE model, which was built upon 12 residual blocks of convolution layers. The authors incorporated three compression layers to reduce the data dimensions in their AE’s architecture. The model was trained to compress turbulence data at a fixed compression ratio of 64.

Liu et al. [22] introduced a seven-layer AE model to compress scientific data. The encoder consisted of three fully connected layers, each of which compressed input data by eight folds. Theoretically, the encoder could compress data by 512x ( $8^3$ ). Similar to the encoder, the decoder had three fully connected layers to decompress the compressed data. A bottleneck Between the encoder and decoder contained latent variables much smaller in size than the inputs. However, this work mainly focused on small-scale 1D data, whereas our models learned data representation in higher dimensions, particularly in 2D and 3D. Another limitation of this model was that it only utilized CPUS for compression, failing to fully utilize the parallel computing power offered by GPUs [39].

Recently, a compression method proposed by Liu et al. [21] achieved great results for 2D and 3D data. Their AE-SZ framework consisted of a Lorenzo prediction model and an AE model, each of which compressed the data independently. The compression results from both models were then compared for the framework to select the model for the data being compressed. The compression ratio of their proposed framework on many scientific data sets surpassed results from other hand-engineered models and AE-based models. However, instead of optimizing one particular model for each input, the framework employed two distinct models to compress the same data.

Slightly different from traditional deep learning models, physics-informed neural networks (PINNs) [40] have been successfully developed to extrapolate data and solve many scientific problems. Choi et al. [41] combined PINN and variational autoencoder (VAE) [18] to compress plasma simulation data. Unlike other types of neural networks, this PINN model optimized several physics constraints, such as mass, moment, and energy, along with the reconstruction loss, i.e., L2 distance. Similar to our work, the authors used integer quantized latent variables, which could be reliably transmitted between different hardware and software platforms as studied by [31].

AE-based structures are applied in various scientific investigations. The authors of [42] utilize AE to capture the latent representation of the input data. Following this, they employ long-short-term-memory (LSTM) models for forecasting sequential data across subsequent time intervals. This strategy reduces computational demands by minimizing the need for full dynamic simulations of drop interactions within microfluidics devices. Similarly, the authors of [43] present a methodology for predicting wildfire occurrences in future years based on input data from the previous year. Leveraging an AE model, input data undergo encoding via the AE’s encoder, mapping them to their latent representation. This latent representation is then fed into a recurrent network network (RNN) model to obtain a representation of the subsequent time step, subsequently which is decoded to derive predictions for that step. Both studies employ Latent Assimilation (LA) techniques to improve prediction accuracy. Additionally, the work of [44] adopts the concept of sequence prediction using RNN within the latent space generated by an AE model. Furthermore, they present Variational Generalized LA to refine the accuracy of sequential prediction within the RRN-AE pipeline. These investigations indicate the potential of utilizing the latent representation to forecast data across subsequent time intervals, presenting an intriguing avenue for our future exploration.

Our proposed model is built on three main components: an encoder network ($\mathbf{E}$), a quantizer ($\mathbf{Q}$), and a decoder network ($\mathbf{D}$). The encoder network encodes data inputs to output latent variables $z_e$ . The quantizer then processes $z_e$ to produce a quantized latent representation $z_q$ . Finally, the decoder network reconstructs data from the compressed representation $z_q$ to output $\widehat{x}$ . The whole model is trained in an end-to-end fashion to minimize a reconstruction loss and constraint losses imposed by the codebook of the quantizer. The model architecture is depicted in Figure 1.

The detailed implementation of the model is given in Table 1. Each stage (EncRes) of the Encoder is connected to an intermediate convolution layer. The intermediate layer acts as a bridge to map the number of channels to the desired vector dimension of the quantization layer. The output representation is then quantized using the corresponding codebook.

3.1. Encoder & Decoder Architecture

As mentioned above, the encoder is trained to extract data representation into latent spaces, whereas the decoder decodes the latent variables to reconstruct the given data. There are two most widely used reconstruction errors, mean-squared error (MSE) and multi-scale structural similarity (MS-SSIM). Depending on targeting criteria, either measure can be used to achieve desirable outcomes. Both measures have been shown to be effective metrics as they typically result in high-quality generated images [20,26,27].

The encoder network $\mathbf{E}$ is created hierarchically. The first level aggressively reduces the dimensions of the inputs and learns their data representation. The second level also performs slight dimension reduction. Data representation from the second level is quantized by its corresponding vector quantizer. These quantized values are then fed into the first-level vector quantizer. The second-level quantization acts as a prior to the first-level quantization. The additional information from the second-level quantization improves the capability of the first-level quantization, which leads to a better reconstruction quality. Although the second level creates slightly more bits during compression, the improvements in reconstruction quality significantly outweigh a slight decrease in compression ratio.

The network $\mathbf{E}$ comprises several 2D convolution layers and blocks of residual connections. The first two convolution layers map inputs to a higher number of channels using a kernel size of 4. It is followed by a couple of residual blocks, which consist of strided convolutions with a kernel size of 5. Components of a residual block are illustrated in Figure 2. The construction of residual blocks is inspired by the model proposed by [14], which achieves high performance for classification tasks without significantly increasing the size of the model. Moreover, residual blocks alleviate the vanishing gradient problem and enable the implementation of deeper models, which improves the expressiveness of our networks [45]. We use non-linear GELU functions as our activation functions [46]. A generalized divisive normalization (GDN) is used to normalize the outputs of the residual block and transform their distribution to be more Gaussian [47]. GDN has been shown to be effective for both image compression [48] and scientific data compression [21]. The encoder network can be simply represented as a mapping function, as shown in (1).

(1) $$z_e=\mathbf{E}\left(x\right),$$

where $\mathbf{E}$ is the encoder network.

The decoder network $\mathbf{D}$ is a mirror of the encoder network $\mathbf{E}$ . Transposed convolution layers are used to replace strided convolutions. At the beginning of each hierarchy, transposed convolutions alter the input of the decoder to acquire a suitable representation with C channels for the following residual blocks. In general, all blocks of the decoder loosely reverse the operations performed by the encoder. Network $\mathbf{D}$ maps the latent representation back to the original dimension, outputting reconstructed data. The decoder can also be considered to be a mapping function as shown in (2).

(2) $$\widehat{x}=\mathbf{D}\left(\mathrm{decoder}\_\mathrm{inputs}\right),$$

where $\mathbf{D}$ is the encoder network.

This paper uses existing, publicly available data from SDRBench (https://sdrbench.github.io/) for bench-marking the performance of our model. Regarding compression on our application data, the model compresses the High-Resolution Earth System Prediction (iHESP) data. The iHESP data have been deposited at https://ihesp.github.io/archive/ and are publicly available. All original code has been deposited on Github at https://github.com/hieutrungle/data-slim and is publicly available as of the date of publication. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Our proposed model proves to be effective in compressing floating-point scientific data, both on 2D benchmark data and our large-scale high-resolution data. It achieves a high compression ratio while preserving a high quality of reconstruction. The model outperforms other state-of-the-art models in some benchmark data sets, particularly 2D simulation data. However, there is room for further improvement. Other lossless compression schemes, such as arithmetic coding, which offers better compression performance, can be used to replace Huffman coding. The model can also be further improved by optimizing the rate loss term, potentially leading to a better compression ratio. Furthermore, the compression pipeline of the proposed model can be optimized to improve compression speed. Since scientific data compression using neural networks is still in its early stages, there is so much more potential improvement that can be achieved for future research along this line.

Conceptualization: Jian Tao and Hieu Le; Methodology: Hieu Le and Jian Tao; Investigation: Hieu Le and Jian Tao; Writing – Original Draft: Hieu Le and Jian Tao; Writing – Review & Editing: Hieu Le and Jian Tao; Funding Acquisition: Jian Tao; Resources: Jian Tao and Hieu Le; Supervision: Jian Tao.

The authors declare no competing interests.

This work is partially supported by the TAMIDS Career Initiation Fellow Program and NSF grants OAC-2112356, OAC-2019129, and OAC-1925764.

The authors would like to thank Dr. Chao Tian, Dr. Jaison Kurian, and Dr. Ping Chang from Texas A&M University for their suggestions and comments on this work. The authors gratefully acknowledge the helpful support provided by the School of Performance, Visualization and Fine Arts, Texas A&M High-Performance Research Computing (HPRC) and Texas A&M Institute of Data Science (TAMIDS). Portions of this research were conducted with the advanced computing resources provided by Texas A&M High Performance Research Computing.