• <tr id="yyy80"></tr>
  • <sup id="yyy80"></sup>
  • <tfoot id="yyy80"><noscript id="yyy80"></noscript></tfoot>
  • 99热精品在线国产_美女午夜性视频免费_国产精品国产高清国产av_av欧美777_自拍偷自拍亚洲精品老妇_亚洲熟女精品中文字幕_www日本黄色视频网_国产精品野战在线观看 ?

    Continuous-Time Prediction of Industrial Paste Thickener System With Differential ODE-Net

    2022-04-15 04:03:56ZhaolinYuanXiaoruiLiDiWuXiaojuanBanNaiQiWuHongNingDaiandHaoWang
    IEEE/CAA Journal of Automatica Sinica 2022年4期

    Zhaolin Yuan, Xiaorui Li, Di Wu, Xiaojuan Ban, Nai-Qi Wu,,Hong-Ning Dai,, and Hao Wang,

    Abstract—It is crucial to predict the outputs of a thickening system, including the underflow concentration (UC) and mud pressure, for optimal control of the process. The proliferation of industrial sensors and the availability of thickening-system data make this possible. However, the unique properties of thickening systems, such as the non-linearities, long-time delays, partially observed data, and continuous time evolution pose challenges on building data-driven predictive models. To address the above challenges, we establish an integrated, deep-learning, continuous time network structure that consists of a sequential encoder, a state decoder, and a derivative module to learn the deterministic state space model from thickening systems. Using a case study, we examine our methods with a tailing thickener manufactured by the FLSmidth installed with massive sensors and obtain extensive experimental results. The results demonstrate that the proposed continuous-time model with the sequential encoder achieves better prediction performances than the existing discrete-time models and reduces the negative effects from long time delays by extracting features from historical system trajectories. The proposed method also demonstrates outstanding performances for both short and long term prediction tasks with the two proposed derivative types.

    I. INTRODUCTION

    AS a core procedure in modern mineral separation, a thickening process produces a paste with high concentration for subsequent tailing storage or backfilling[1]–[3]. During this thickening process, an industrial paste thickener achieves solid–liquid separation based on gravity sedimentation. The purpose of the industrial paste thickener is to efficiently control the final underflow concentration (UC).Most closed-loop control systems manipulate the underflow slurry pump speed and flocculant pump speed as the inputs to stabilize the underflow concentration within its specified range during operation. Previous studies [4], [5] showed that model prediction control (MPC) can facilitate the control process of thickening systems owing to the advantages of its high robustness and applicability. Hence, the accurate prediction of thickening systems has received extensive attention for the analysis and control of thickeners [5]–[7].

    A complex industrial system such as the thickening system typically has the following key features:

    1) Non-Linear System Dynamics:Most industrial systems have extremely complex high-order dynamical equations that are not affine or linear systems.

    2) PartiaLly Observed Data:The information extracted from sensors or other available methods is incomplete. In particular, a number of unknown hidden variables exist in such systems.

    3) Influence of Long Delays:The system states are influenced by external inputs or internal states that occur over a long previous time.

    4) Continuous-Time (CT) Evolution:Because real industrial systems follow various physical laws, their time evolution can be expressed via CT differential equations.

    The above features of a thickening system create challenges for predictive control. There are a number of studies addressing these challenges. Data-driven methods are emerging as one of the most successful techniques for modeling complex processes [8]. Traditional data-based CT system prediction methods focus on fitting high-order differential equations based on sampled noisy data from real systems. However, they lack the ability to cope with partially observed and extremely complex system dynamics. Recent advances of deep neural networks (DNNs) have shown their strengths in addressing these issues owing to their strong feature representation abilities and scalable parameter structures, leading to the wide usage of DNNs in computer vision [9]–[11], natural language processing [12], [13], time series prediction [5], [14]–[17], and fault diagnosis [18].However, most DNN-based system-modeling methods are based on discrete time, disregarding the CT properties of a system. The lost prior information from physical insights undoubtedly leads to the deterioration of the model accuracy.

    Studies of prediction control can be categorized into two types. The first type provides short-term predictions for model-based control algorithms [5], such as model predictive control (MPC). The learned system provides deterministic prior knowledge of the dynamical systems, thereby approximating the infinite-horizon optimal control as a shortterm optimization problem. The second type is mainly based on simulations, which imitate the outputs of an unknown system under a long-term feed of inputs [19]. Compared with short-term predictions, simulations require higher robustness and stability to provide long-term predictions. However, there are few studies on designing predictive models for short and long term predictions to support subsequent applications, such as MPC and simulations.

    To address the above challenges, we propose a deep CT network composed of a sequential encoder, a state decoder,and a derivative module to learn the auto-regressive processes and influences from the system inputs based on real thickening data in an end-to-end manner. Specifically, the long-time system delay motivated us to utilize a sequential encoder to extract features from historical system trajectories.We designed the derivative module for the CT state space model based on a DNN. This module fits the non-linear CT evolution of the system and infers the non-observable information by introducing hidden states. Moreover, the problems of short and long term predictions are solved by feeding historical system trajectories and system inputs with arbitrary lengths to the model after incorporating the designed non-stationary and stationary systems into the trained model.The future system outputs are then predicted. The contributions of this paper are threefold:

    1) We propose a novel deep-learning-based CT predictive model for a paste thickener. The deep learning network consists of three components: a sequential encoder, a state decoder, and a derivative module.

    2) We design two kinds of derivative modules, named stationary and non-stationary systems, to handle the shortterm and long-term prediction tasks, respectively.

    3) We conduct extensive experiments on real industrial data collected from a real industrial copper mining process. The results demonstrate the outstanding performance of the proposed model in providing predictions for the thickener system with non-linear and time-delay properties. In addition,we conduct ablation studies to evaluate the effectiveness of each module in the proposed model.

    The rest of the paper is organized as follows. We briefly introduce the related work in Section II. We then present the problem formulation in Section III. We next present the CT deep sequential model in Section IV. Experimental results are shown in Section V. We then summarize the paper and discuss future research directions in Section VI.

    II. PRELIMINARIES AND RELATED WORK

    As a core device in a thickening system, a paste thickener is generally composed of a high sedimentation tank and a raking system. Fig. 1 depicts the general structure of a thickener and its key components. After being fed with flocculant and tailing slurry with a low concentration, underflow with a high concentration is discharged from the bottom of the thickener and is then used to produce paste in the subsequent procedures. The prediction of a thickening system refers to the estimation of the future system outputs, such as the underflow concentration and mud pressure based on the historical system trajectories and system inputs. The prediction of a thickening system is essentially similar to system identification [8]. For system identification, interpretable model structures are designed based on prior knowledge, and the parameters are determined by fitting real data. As one of the subsequent applications of identified models, the prediction forecasts the system outputs according to the inputs.

    Fig. 1. Crude slurry flow with a low concentration is fed into the mix tank accompanied by flocculant. Under the effect of the flocculant, particles agglomerate to larger clumps and concentrate at the bottom. The paste thickener continuously produces underflow with a high concentration and clear water in an overflow pipe located at the top of the thickener.

    A. Prediction of Thickening Systems

    The prediction methods for thickening systems can be categorized into two types: 1) gray-box thickening system simulations and 2) black-box thickener system predictions. In the gray-box simulations, the sedimentation process is mainly considered from a physical perspective [4], [20], [21]. Theorybased gray-box methods can be exactly explained and implemented effectively for specific systems. However, most of them are mainly built on many ideal hypotheses and suffer from the complexity of slurry particle dynamics and external unknown environment disturbances.

    In contrast, the black-box methods do not require prior assumptions or constraints to be given. A complete parameterized model with a high degree of freedom is defined to predict the system outputs and learn the optimal parameters from real data. Since the offline system trajectories from an industrial system are always available and adequate, blackbox-based methods, including latent factor model [22]–[24],imitation learning [25], and deep neural network [26] have been widely used in the current industrial systems [27]–[32].

    Moreover, Oulhiqet al. [33] used a black-box linear dynamic model with a deterministic time delay to identify an industrial thickener system using historical data. Such parameterized linear model lacks adequate expressivity to represent the non-linear properties in a thickening system.Most recently, random forest model is presented for modeling a paste thickening system based on a purely data-driven approach for modeling and evolutionary strategies [34].Because random forest model only fits the thickening system dynamic in single step, it ignores the time delay and the correlations between adjacent positions in sequential inputs and outputs. A bidirectional gated recurrent unit (BiGRU)with an encoder–decoder deep recurrent neural network is introduced to model thickening systems [5]. Yuanet al. [6]proposed a dual-attention recurrent neural network to model the spatial and temporal features of a thickening system,thereby improving the prediction accuracy of the underflow concentration. However, the above studies [5], [6] only focus on discrete-time system predictions rather than a CT thickening system.

    B. Prediction of Continuous-Time Systems

    The prediction of physical systems based on CT models directly from sampled data has the following advantages [8]:1) transparent physical insights into the system properties, 2)inherent data filtering, and 3) the capability of dealing with non-uniformly sampled data. For any numerical schemes for solving CT differential equations, sophisticated discretization methods have high accuracy but suffer from enormous time and memory costs. A recently developed advanced ordinary differential equation (ODE) solver [35] introduces the reversemode automatic differentiation of ODE solutions, thereby only requiring O(1) memory cost. Meanwhile, this method also allows the end-to-end training of ODEs within a large DNN. Moreover, Demeester [19] proposed a stationary CT state space model for predicting an input–output system when the observations from a system are unevenly sampled.Although it has successfully improved the accuracy and stability of long-term predictions by introducing a stationary system, it did not take advantage of a non-stationary system in the short-term prediction task.

    Compared with the existing CT models, our model considers both short-term and long-term predictions, thereby achieving outstanding performance.

    III. FORMULATION AND NOTATION OF PASTE THICKENING SYSTEM PREDICTION

    Fig. 2. Proposed model is composed of several components. The recurrent neural network (RNN) encoder outputs the initial hidden state h(t0) according to historical sequences from the system. The derivative module is embedded into the ordinary differential equation (ODE) solver to calculate the hidden state at an arbitrary time. A parallel interpolation mechanism is embedded into the derivative module, which interpolates discrete input sequences to a CT series. Finally,the state decoder module transforms the hidden state to the predicted system output.

    IV. CONTINUOUS-TIME DEEP SEQUENTIAL MODEL FOR THICKENING SYSTEM PREDICTIONS

    A. RNN for Encoding Historical Sequences

    industrial experience, we use the system delay as prior knowledge denoted byTd. If we assume a uniform sampling of the sensors for a sampling intervalTs,NyandNxcan be estimated according to the equationNy=Nx=N=Td/Ts.The influence of parameterNon the model accuracy is examined in Section V. In the thickening system, the correlations between the current system state and historical trajectories are mostly compressed in the short term. This property encourages us to use a simple and unidirectional RNN to encode the historical trajectories of a thickening system. The solved hidden state h(t0) from encoder involves all necessary information of historical trajectories and is represented as an initial state of solved ODE.

    B. ODE Solver for Modeling State Space

    We employ the parameterized CT state space model to represent the relations between the system inputs, hidden states, and outputs:

    Generally, an ODE solver with a lower error tolerance increases the frequency for calling differential functiond. It leads to more time consumption but results in a higher accuracy. This guideline is also tenable when we construct a neural ODE network to fit sequential datasets. The detailed comparisons of the time cost and accuracy are shown in Section V.

    It is worth investigating the definition of a suitable structure ofd. The most intuitive solution is to employ a basic neural network to estimate the derivative that is named nonstationary model.

    Non-Stationary:

    where MLP(·) denotes a multi-layer perceptron. The combination of a non-stationary system with an ODE solver has a strong similarity to residual connections, which have been widely used in other advanced deep networks [35].

    In the field of stochastic process analysis, non-stationary systems are a stochastic process with a mean and covariance that vary with respect to time [40]. Differencing [41] is an effective way to make non-stationary time series stationary by eliminating trend and seasonality. Generally, a thickening system has strong trends in the underflow concentration, mud pressure, and other core variables. The thickening system is an approximation of non-stationary systems, indicating that the differencing operation can improve the fitting accuracy. In(11), the derivative module intrinsically learns the first-order difference of the hidden states in the latent space. In contrast to the operation of differencing the system outputs directly, a model that differences the hidden states has an equivalent or stronger ability to represent a non-stationary system that is of first or even higher order. However, the non-stationary system(11) also suffers from a severe problem when handling longterm prediction tasks. To solve an ODE over long intervals,repetitive accumulation in a CT range can lead to a significant magnitude increase of the hidden states. Consequently, the estimation error will grow accordingly, resulting in the difficulties in achieving accurate system output from the decoder.

    Therefore, we devise another derivative module, namely,the stationary system, to handle the long-term prediction problem. In particular, we have

    Stationary:

    In a stationary system, we use a GRU to construct the derivative module because it has a strong ability to carry longtime information. In a non-stationary system, consecutive accumulations diffuse the hidden state in an unconstrained range. Thus, we employ the MLP to learn the first-order difference h˙(t) directly under h(t) and x(t).

    C. Parallel Spline Interpolation

    Note that the calculation of h˙(t) in (11) and (12) depends on the external input x(t), which may not exist in our dataset.External input sequences XMfin the training data are discrete,while the computation of the ODE needs x(t) in a CT range.Before each forward pass of the network, it is necessary to interpolate the external inputs to the continuous form. Deep networks are typically trained in mini-batches to take advantage of efficient data-parallel graphics processing unit(GPU) operations. Therefore, we implement a parallel spline interpolation mechanism on top of PyTorch, which is a wellknown deep learning framework.

    In our dataset, the external input data are evenly sampled,thereby simplifying the implementation of parallel interpolation. To simplify the explanation, we assume that the dimension of the external input is equal to 1.

    D. State Decoder

    The state decoder mechanism is essentially a fully connected network. We therefore have the following equation to represent the output,

    Compared with other state space models that only employ a single matrix for decoding, the nonlinear decoder is chosen because the accumulative form in (11) causes the range of the input h(t) to be non-deterministic. The activation function tanh(·) constrains the output of the decoder to a rational range.

    E. Model Training

    Since all of the operations of the ODE solver in our model are smooth and differentiable, we can train the complete model by the standard back-propagation algorithm with the loss function defined as follows:

    V. EXPERIMENTAL RESULTS

    This section presents experimental results for the proposed method on the dataset of real thickening systems. We mainly investigate three issues: RQ1: What are the advantages of employing a CT deep sequential network with a high-accuracy ODE solver for modeling a thickening system? RQ2: What are the pros and cons of using stationary and non-stationary systems in prediction tasks? RQ3: How do the different interpolation methods and sequential encoder affect the accuracy of the proposed CT model? We first describe the dataset, the hyper-parameters of the model, and the training and test configurations. We then present the detailed experimental results.

    A. Thickening System Dataset

    For our experiments, the dataset was collected from the paste thickener manufactured by the FLSmidth from the NFC Africa Mining PLC, Zambian Copperbelt Province. Fig. 4 illustrates two identical thickeners in our experiments. They are used to concentrate copper tailings to produce paste in the backfilling station. Both devices operate in the closed-loop mode with PID controllers.

    Some key technical parameters of the studied thickener are listed in Table I.

    The measured data are sampled evenly with two-minute intervals from May 2018 to February 2019. A short piece of original dataset is shown in Table II. The collected data come from seven monitoring sensors just as the defined y(k) and x(k)in Section III. After deleting the records corresponding to the time when the system was out of service, there are 24 673 pieces of data remaining.

    Fig. 3. Illustration of the process of building both the training, validation, and test datasets. An independent data tuple for training or testing is composed of four vector sequences. X[i:i+N] and Y[i:i+N] represent the historical trajectories in conditional range. X[i+N:i+N+M] and X[i+N:i+N+L] represent the inputs sequences, which have equivalent length with predicted sequences. Y[i+N:i+N+M] and Y[i+N:i+N+L] represent the real system outputs. The former is utilized to generate optimized loss in training and the latter is only used in testing and validation phase for evaluating the accuracy of prediction.

    Fig. 4. The figures illustrate two identical paste thickeners in our experimental mining station, including one primary and one alternate thickener. Both devices operate in closed-loop mode with proportionalintegral-derivative (PID) controllers.

    TABLE I SOME KEY TECHNICAL PARAMETERS OF THE THICKENER

    We employ the first 70% of the entire dataset to train the model. In the remaining 30% portion, the first 15% is used for validation to determine the best training epochs, and the other 15% is the test dataset for evaluating the model accuracy. By splitting and building the inputs-outputs sequential tuples according to Fig. 3, there are 17 131 tuples left for training and 3561, 3421, 3121 tuples left for testing and validation forL=60, 200, 500, respectively. All the datasets are normalized to standard normal distributions with a unified mean and variance before the training and test phases.

    B. Experimental Setup

    We use the mini-batch stochastic gradient descent (SGD)with the Adam optimizer [42] to train the models. The batch size is 512, and the learning rate is 0.001 with an exponential decay. The decay rate is 0.95, and the period for decay is 10 epochs. The size of the hidden state h(t) in ODE is 32. The RNN encoder module has a single hidden layer and the size is equal to 32 that is consistent with the size of hidden state h(t).The size of the hidden layer in state decoder is 64. In both the adaptive ODE solvers, the time for solving the ordinary differential equations increases if we reduce the tolerance of approximate error. For balancing the time cost and accuracy,we set the relative tolerance to 1E–4 and absolute tolerance to 1E–5 in all of experiments.

    During the training procedure, the length of the historical sequences denoted byNis 80, and the length of predicted outputs denoted byMis 60. The best-performing model in the validation dataset is chosen for further evaluation with the test dataset. The training and test phases were performed on a single Nvidia V100 GPU. The implementation uses the PyTorch framework. We define the CT range as0 ≤t≤Mδtfor given discrete integral indices [0,1,...,M]. The time interval δtof the adjacent data points is set to 0.1.Accordingly, the normalized factor μtin (12) is also set to 0.1.When we use the Euler approximation to solve an ODE equation in a stationary system, the predicted hidden state in the next time step is equal to the output of the GRU cell corresponding to the discrete-time system:

    C. Results and Discussion

    1) Main Results

    We investigate the influence of the types of ODE solversand system types. We select four ODE solvers: Euler, Mid-Point, fourth-order Runge-Kutta (RK4), Dormand-Prince(Dopri5) [35], and 3-order Bogacki-Shampine (Bosh) [43].We investigate the performances of those ODE solvers in both non-stationary and stationary systems. To make a trade-off between the model accuracy and time consumption, we set the relative tolerance to 1E–4 and the absolute tolerance to 1E–5.Moreover, we also consider the discrete-time deep sequential model for the state space (DT-State-Space), the attentionbased Seq2Seq model (Attention-Seq2Seq) [5], and Transformer [44] for comparison. The DT-State-Space [45]model employs a parameterized per-time-series linear state space model based on a recurrent neural network (RNN) to forecast the probabilistic time series. The sizes of state space and RNN hidden layer are set to 16 and 32, respectively. The hyperparameters setting of Transformer and Attention-Seq2Seq are kept with the original literature.

    TABLE II A TABULAR EXAMPLE OF PASTE THICKENING SYSTEM DATASET

    We conduct three groups of experiments to investigate the RRSEs, MSEs, and time consumption of models with prediction lengths ofL=60, 200, and 500.

    a) Comparison of proposed and other baseline models:

    We first examine the performance of the Attention-Seq2Seq model, the DT-State-Space model, and Transformer, which are defined in discrete-time settings in Table III. Although they perform competitively, better than the proposed models with the Euler ODE solver, they perform worse than the models with high-order ODE solvers, especially for long-term predictions. The results also indicate that employing a CT model is consistent with the features of the CT evolution in thickening systems, thereby improving the prediction accuracy.

    b) Comparison of different ODE solvers:

    We analyze the comparisons of different ODE solvers respectively from stationary system and non-stationary system. When the derivative module is defined as a nonstationary system and we only focus on short-term prediction withL=60, we find that the Euler method achieves relatively higher RRSE and MSE values (i.e., poorer prediction performances) than the other four ODE solvers, though it has a much lower time consumption than the other solvers. As the simplest method for solving ODEs, the Euler method evaluates the derivative network only once between two adjacent time points. Meanwhile, the Mid-Point and RK4 methods have higher prediction accuracies than the Euler method, since they evaluate the derivative network two and four times, respectively, between two adjacent time points.Moreover, the Dopri5 and Bosh methods achieve better accuracies, though they have higher time consumptions.Dopri5 performs slightly better than Bosh. As adaptive methods in the Runge-Kutta family, the Dopri5 and Bosh methods ensure that the output is within a given tolerance of the true solution. Their time consumptions for solving an ODE equation increase as the accuracy tolerance is decreased.

    Strangely, with the increase in the prediction length, we find that the accuracies of non-stationary models crash gradually and the degradation of Euler is slightly lower than the others.The reason of this inconsistent phenomenon is that nonstationary system brings accumulative errors in long-term predictions. High-order ODE solvers evaluate the derivative module more times recursively, which brings more accumulative errors. Not only do the high-order solvers not improved the accuracies of non-stationary system in long-term predictions, they made the accuracies worse.

    When the derivative module is switched to a stationary system. It is worth mentioning that the time consumption for the two adaptive methods, Bosh and Dopri5, to solve an ODE equation significantly increases. We do not list the accuracies of the Dopri5 and Bosh for the stationary systems in Table III because the extremely slow speed makes the method ineffective for practical applications. According to the comparison of the ODE solvers, the high-order ODE solver,such as RK4, results in lower fitting errors than the low-order methods while requiring more time to evaluate the ODE equations intensively.

    c) Comparison of stationary models and non-stationary models:

    For comparing the distinctions between stationary models and non-stationary models more intuitively, we further visualize the prediction performance of the non-stationary and stationary systems with different ODE solvers. Fig. 5 depicts the predicted sequences of the non-stationary and stationary systems with different ODE solvers for the short-term prediction task withL=60. The results show that the nonstationary models outperform the stationary models in shortterm prediction tasks. The estimated sequences from the nonstationary models are slightly closer to the real system output than those from the stationary models. The learning process of a non-stationary system is essentially equivalent to differencing the hidden state and employing the MLP network to learn the relatively stationary first-order difference.Furthermore, the non-stationary models can predict the system outputs smoothly because the non-stationary structure limitsthe hidden states to only changing in a continuous and slow manner. This constraint is consistent with the properties of a slow thickening system that shrinks the searching space of the model parameters to prevent overfitting.

    TABLE III ROOT RELATIVE SQUARED ERROR (RRSE), MEAN SQUARED ERROR (MSE), AND TIME CONSUMPTION OF PREDICTED UNDERFLOW CONCENTRATION

    Fig. 5. In the short-term prediction task with L=60, the non-stationary models output more stable and accurate sequences than the stationary models.

    Fig. 6 presents the experimental results of a long-term prediction task withL=200 (similar results withL=500 can be found in Table III). The tabular results in Table III demonstrate that the RRSE and MSE of non-stationary models are much higher than those of the stationary ones in long-term prediction task, which is consistent with the graphical results.Compared to the excellent results of non-stationary models in Fig. 5, Fig. 6(a) shows that the prediction accuracy for the non-stationary system decays significantly, and the predicted outputs deviate from the true outputs gradually with the increase in the prediction length. However, the predicted results of stationary models are stabilized and closed to the true system outputs, which confirms the excellent accuracies of stationary models in the long-term prediction problem. The structure of non-stationary ODE leads to the hidden state in progressive evolution that is unconstrained and gradually expanding. Although we embed a tanh function for the decoder network to restrict the final prediction of the underflow concentration and pressure to a rational range, it is impossible for the decoder module to learn an effective mapping function from an extremely large hidden state space to the system output space. Similarly, Fig. 6 also demonstrates that high-order ODE solvers, such as the 4th-order Runge-Kutta, still perform slightly better than the low-order solvers,such as Euler, in long-term prediction.

    We conduct five other groups of experiments with different values of the prediction length to evaluate the prediction performance (i.e., the MSE) of the underflow concentration and ground-truth for both stationary and non-stationary systems. The results in Fig. 7 show that the non-stationary system performs better than the stationary system in the shortterm prediction task (e.g.,L<100), although the stationary system outperforms the non-stationary system in long-term prediction tasks. For example, whenLexceeds 120, the errors from the non-stationary system increase with the predicted length, while the stationary system significantly stabilizes the accumulative errors in the long-term prediction.

    Fig. 6. In the long-term prediction task with L=200, Fig. 6(a) illustrates that non-stationary models only performed well in the early time horizon. In the late horizon, the predicted sequences deviated from the true system outputs significantly compared with those of the stationary models, which are shown in Figs. 6(b) and 6(c). The results also indicate that the models with high-order ODE solvers performed better than the models with low-order ODE solvers in long-term prediction tasks.

    Fig. 7. Predicted length L affected the accuracy (log10 MSE ± 2σ, computed across five runs) of predicted underflow concentration for both stationary and non-stationary systems.

    2) Experiments for Evaluating Interpolation Order

    We next investigate the effect of the interpolation method on the prediction accuracy. We test four spline interpolation methods with different orders and compare the prediction accuracies on test datasets. The results in Table IV demonstrate that the higher-order interpolations slightly outperform the lower-order ones. This proves that the system input of the thickening system is a non-linear complex process, and that the information from external inputs is essential for predicting the outputs of the system. Higherorder spline interpolations exploit more correlational features from adjacent inputs and interpolate the empty area with a better accuracy than the lower-order interpolations.

    3) Ablation Experiments for Studying System Time Delays and Improvements From Sequential Encoder

    Finally, we investigate the significance of introducing the sequential encoder to confront the system time delay. We investigate the influence ofNon the model accuracy.Specially, whenNis set to 1, the sequential encoder is replaced by a neural network with one hidden layer that encodes the system output x(k?1) in a single time step to the initial hidden state h(t0). WhenNis set to 0, the initial state h(t0)is a learnable or zero vector [19] that had no relationship with historical system trajectories. We examine the different choices ofNin experiments withL=60, 200, and 500,respectively. In the experiments withL=60, the derivative module is set to be a non-stationary system with an MLP cell.We change it to a stationary system with a GRU cell whenL=200and 500. The ODE solver is the fourth-order Runge-Kutta solver for all of the models.

    The results shown in Table V demonstrate that the introduction of the sequential encoder to extract features from the historical sequence leads to better performance than those in the cases withN=1 or 0. The intuitive explanation is that the predicted output sequences have strong statistical correlations with historical system trajectories. The optimal length of the encoded sequence is approximatelyN=80,which is consistent with our prior experience of 2–3-h time delays in thickening systems. When the length of input sequence exceeds the optimal value, the accuracy slightly decreases.

    Intuitively, the short-term prediction task benefits more from historical system trajectories than the long-term prediction task. When the length of the predicted sequence increases, the advantage brought by the sequential encoder also decreases. In the task withL=500, the profit of employing sequential encoder decreases obviously.

    VI. CONCLUSIONS

    This paper focuses on the prediction of the outputs of a thickening system based on deep neural sequence models. We introduce a CT network composed of a sequential encoder, a state decoder, and a derivative module, with internal computation processes including interpolation and a differential ordinary differential equation solver, to describe the complex dynamics of a thickening system. Experiments on datasets from real thickening systems demonstrate that the introduction of the sequential encoder and parallel cubic spline interpolation play a crucial role in our model architecture. We conducted extensive experiments to evaluate the proposed models for both stationary and non-stationary systems with different ODE solvers. The results show that the non-stationary system outperforms the stationary system for short-term prediction tasks. However, the non-stationarymodel suffers from the accumulation of errors from the incremental calculation, thereby leading to inferior results in long-term prediction tasks. This demonstrates that the model with the non-stationary system is more suitable for being embedded in a model-based feedback controller (e.g., MPC controller) while the stationary system avoids this problem and performs better in long-term prediction tasks. Therefore,the model with the stationary system is a better choice when a stable and robust identified system is required to predict longterm sequences (e.g., simulations or controller testing).

    TABLE IV ACCURACY COMPARISONS WITH DIFFERENT ORDERS OF INTERPOLATIONS

    TABLE V ACCURACY COMPARISONS WITH DIFFERENT METHODS FOR GENERATING THE INITIAL HIDDEN STATE h(t0)

    In the industrial data processing field, it is a common requirement to process unevenly spaced data. Although the dataset employed in this paper is sampled evenly, we can extend our method to deal with uneven data naturally by adjusting the time intervals. This extension deserves further experimental verification in future work. Another promising research direction is to extend the method to probabilistic generative models and perturbed time-varying models [46] for determining the unknown sampling noise and uncertainty in thickening systems. Moreover, it is worth investigating our method for other dynamical industrial systems.

    极品少妇高潮喷水抽搐| 自线自在国产av| 夜夜爽夜夜爽视频| 亚洲经典国产精华液单| 亚洲精品自拍成人| 一级片'在线观看视频| 啦啦啦中文免费视频观看日本| 在现免费观看毛片| 精品人妻一区二区三区麻豆| 老司机影院毛片| 少妇人妻一区二区三区视频| av天堂久久9| 99久国产av精品国产电影| 水蜜桃什么品种好| 秋霞在线观看毛片| 免费观看的影片在线观看| 久久 成人 亚洲| 亚洲精品久久午夜乱码| freevideosex欧美| 欧美老熟妇乱子伦牲交| 国产视频内射| 97在线视频观看| 日韩av在线免费看完整版不卡| 亚洲精品一区蜜桃| 成人无遮挡网站| 久久影院123| 2018国产大陆天天弄谢| 久久人人爽av亚洲精品天堂| 男男h啪啪无遮挡| 国产精品久久久久久av不卡| 午夜福利影视在线免费观看| 国产老妇伦熟女老妇高清| 国产精品久久久久久精品电影小说| 亚洲国产精品成人久久小说| 晚上一个人看的免费电影| 女的被弄到高潮叫床怎么办| 国产日韩欧美亚洲二区| av天堂中文字幕网| 日韩三级伦理在线观看| 麻豆成人午夜福利视频| 国产成人午夜福利电影在线观看| 一区二区三区精品91| 激情五月婷婷亚洲| 在线观看www视频免费| 日韩在线高清观看一区二区三区| 久久99蜜桃精品久久| 国产日韩一区二区三区精品不卡 | 久久久久精品性色| 午夜免费男女啪啪视频观看| 2022亚洲国产成人精品| 天美传媒精品一区二区| 欧美成人午夜免费资源| 精品国产一区二区三区久久久樱花| 99热国产这里只有精品6| 91在线精品国自产拍蜜月| 另类精品久久| 三级经典国产精品| 黑人高潮一二区| 性色avwww在线观看| 精品亚洲乱码少妇综合久久| 插逼视频在线观看| xxx大片免费视频| 欧美 亚洲 国产 日韩一| 精品亚洲成国产av| 永久免费av网站大全| 啦啦啦在线观看免费高清www| 亚洲综合精品二区| 免费人成在线观看视频色| 大话2 男鬼变身卡| 国产日韩一区二区三区精品不卡 | 蜜桃久久精品国产亚洲av| 久久久久国产网址| 日韩大片免费观看网站| 熟女人妻精品中文字幕| 国产伦在线观看视频一区| 国产 一区精品| 视频中文字幕在线观看| 特大巨黑吊av在线直播| 欧美区成人在线视频| 久久久久久伊人网av| 国产在视频线精品| 成人综合一区亚洲| 国产亚洲5aaaaa淫片| 男女边吃奶边做爰视频| a级毛色黄片| 国产成人一区二区在线| 亚洲电影在线观看av| 成人亚洲精品一区在线观看| 午夜免费男女啪啪视频观看| 日韩一区二区三区影片| 我要看黄色一级片免费的| 日本av手机在线免费观看| 亚洲欧洲日产国产| 亚洲精品日韩av片在线观看| 国产精品国产三级专区第一集| 精品国产露脸久久av麻豆| 99国产精品免费福利视频| 人妻 亚洲 视频| 中文在线观看免费www的网站| 九九爱精品视频在线观看| 亚洲国产欧美日韩在线播放 | 国产片特级美女逼逼视频| 久久韩国三级中文字幕| 午夜免费鲁丝| av又黄又爽大尺度在线免费看| 免费观看a级毛片全部| 午夜福利在线观看免费完整高清在| 男女边摸边吃奶| 久久99精品国语久久久| 日日撸夜夜添| 国产日韩一区二区三区精品不卡 | 亚洲精品亚洲一区二区| 三级国产精品欧美在线观看| 欧美亚洲 丝袜 人妻 在线| 国产一区二区在线观看日韩| 久久精品国产自在天天线| 日韩强制内射视频| 亚洲情色 制服丝袜| 少妇裸体淫交视频免费看高清| 黄色毛片三级朝国网站 | 欧美高清成人免费视频www| 欧美+日韩+精品| 久久国产精品大桥未久av | 免费黄色在线免费观看| 特大巨黑吊av在线直播| 亚洲三级黄色毛片| 成人亚洲精品一区在线观看| 国产有黄有色有爽视频| 汤姆久久久久久久影院中文字幕| 亚洲国产精品专区欧美| 午夜久久久在线观看| 六月丁香七月| 日韩 亚洲 欧美在线| 亚洲图色成人| 免费看不卡的av| 国产精品久久久久久久电影| 午夜av观看不卡| 欧美精品国产亚洲| 高清不卡的av网站| 日韩 亚洲 欧美在线| 丝袜喷水一区| 亚洲欧美日韩东京热| 亚洲精品亚洲一区二区| 免费不卡的大黄色大毛片视频在线观看| 精品少妇内射三级| 久久久久久久大尺度免费视频| 99re6热这里在线精品视频| 蜜臀久久99精品久久宅男| 少妇的逼好多水| 午夜老司机福利剧场| 婷婷色av中文字幕| 一个人免费看片子| 国产伦在线观看视频一区| 欧美精品一区二区大全| 国产精品秋霞免费鲁丝片| 国产亚洲最大av| 18禁在线无遮挡免费观看视频| 国产美女午夜福利| 涩涩av久久男人的天堂| 国产精品99久久久久久久久| 美女大奶头黄色视频| 九九久久精品国产亚洲av麻豆| 国产男人的电影天堂91| 久久韩国三级中文字幕| 伊人久久国产一区二区| 男人舔奶头视频| 久久国产亚洲av麻豆专区| 亚洲av成人精品一区久久| 亚洲av国产av综合av卡| 丰满少妇做爰视频| 国产成人一区二区在线| xxx大片免费视频| a级毛色黄片| 亚洲经典国产精华液单| 91久久精品国产一区二区三区| 婷婷色av中文字幕| 在线看a的网站| 国产精品一区二区在线不卡| 国产视频内射| 91精品一卡2卡3卡4卡| 亚洲av欧美aⅴ国产| 七月丁香在线播放| 在现免费观看毛片| 亚洲av成人精品一区久久| 日本免费在线观看一区| 观看av在线不卡| 一级片'在线观看视频| 精品卡一卡二卡四卡免费| 韩国高清视频一区二区三区| 久久婷婷青草| 在线天堂最新版资源| 亚洲天堂av无毛| 欧美少妇被猛烈插入视频| 亚洲精品日本国产第一区| 观看免费一级毛片| 久久国产精品大桥未久av | 久久精品国产亚洲网站| 免费人成在线观看视频色| 国内少妇人妻偷人精品xxx网站| 深夜a级毛片| 国产免费视频播放在线视频| 午夜免费男女啪啪视频观看| 亚洲精品乱久久久久久| 女的被弄到高潮叫床怎么办| 日韩av在线免费看完整版不卡| 日日摸夜夜添夜夜爱| 黄色欧美视频在线观看| 日韩制服骚丝袜av| 成人二区视频| 亚洲欧美精品自产自拍| 最近中文字幕高清免费大全6| 丝袜喷水一区| 久久精品久久久久久噜噜老黄| 久久久久精品性色| 国产精品.久久久| 国产亚洲午夜精品一区二区久久| 中文字幕人妻熟人妻熟丝袜美| 亚洲精品久久久久久婷婷小说| 六月丁香七月| 精品人妻熟女毛片av久久网站| 菩萨蛮人人尽说江南好唐韦庄| 曰老女人黄片| 欧美激情极品国产一区二区三区 | 精华霜和精华液先用哪个| 中文字幕精品免费在线观看视频 | 欧美一级a爱片免费观看看| 午夜av观看不卡| 日本欧美国产在线视频| av福利片在线观看| 亚洲色图综合在线观看| 亚洲人成网站在线观看播放| 日韩三级伦理在线观看| 久久精品国产a三级三级三级| 男女啪啪激烈高潮av片| 2021少妇久久久久久久久久久| 日韩成人伦理影院| 2018国产大陆天天弄谢| 久久精品国产鲁丝片午夜精品| 久久久午夜欧美精品| 午夜91福利影院| 亚洲欧美成人精品一区二区| 中文精品一卡2卡3卡4更新| 国产无遮挡羞羞视频在线观看| 亚洲第一区二区三区不卡| 午夜免费鲁丝| kizo精华| 一边亲一边摸免费视频| 中文在线观看免费www的网站| 欧美日韩在线观看h| 国产亚洲精品久久久com| 国产精品99久久久久久久久| 男女国产视频网站| 精品视频人人做人人爽| 亚洲av.av天堂| 韩国高清视频一区二区三区| 人妻夜夜爽99麻豆av| 亚洲色图综合在线观看| 亚洲国产精品999| 日韩中文字幕视频在线看片| 亚洲国产精品国产精品| 少妇被粗大猛烈的视频| 国产日韩欧美视频二区| 国产在线免费精品| 女人精品久久久久毛片| 男人和女人高潮做爰伦理| 国产成人精品一,二区| 我的老师免费观看完整版| 黑人高潮一二区| 成年人午夜在线观看视频| 精品久久久噜噜| 成人毛片a级毛片在线播放| 国产精品欧美亚洲77777| 日韩 亚洲 欧美在线| 久久久欧美国产精品| 2022亚洲国产成人精品| 91精品伊人久久大香线蕉| 自线自在国产av| 人体艺术视频欧美日本| 国产精品免费大片| 男人狂女人下面高潮的视频| 波野结衣二区三区在线| 亚洲av福利一区| 水蜜桃什么品种好| 免费大片黄手机在线观看| 日韩不卡一区二区三区视频在线| 老司机影院成人| 午夜视频国产福利| 亚洲欧美成人综合另类久久久| 人妻人人澡人人爽人人| 久久久a久久爽久久v久久| 欧美激情国产日韩精品一区| 精华霜和精华液先用哪个| 国产免费视频播放在线视频| 久久精品国产鲁丝片午夜精品| 你懂的网址亚洲精品在线观看| 色哟哟·www| 亚洲国产欧美日韩在线播放 | 免费观看无遮挡的男女| 男人爽女人下面视频在线观看| 中文在线观看免费www的网站| 男女免费视频国产| 国产亚洲最大av| 久久精品国产a三级三级三级| 欧美精品一区二区大全| 丰满迷人的少妇在线观看| 99久久人妻综合| 免费观看av网站的网址| 我要看日韩黄色一级片| 色视频www国产| h日本视频在线播放| 日本猛色少妇xxxxx猛交久久| 国产精品久久久久久精品电影小说| 国产成人一区二区在线| 精华霜和精华液先用哪个| 国内精品宾馆在线| 少妇的逼水好多| 久久99蜜桃精品久久| 大片免费播放器 马上看| 午夜免费观看性视频| 国产黄频视频在线观看| 嫩草影院入口| 80岁老熟妇乱子伦牲交| 一区二区三区精品91| 精品久久久精品久久久| av专区在线播放| 亚洲熟女精品中文字幕| 国产精品久久久久久精品电影小说| 欧美+日韩+精品| 国产精品人妻久久久影院| 国产美女午夜福利| 国产探花极品一区二区| 亚洲熟女精品中文字幕| 精品国产国语对白av| tube8黄色片| 国产极品天堂在线| 美女xxoo啪啪120秒动态图| 91精品国产九色| 伦理电影大哥的女人| 亚洲av福利一区| 人妻系列 视频| 内射极品少妇av片p| av免费在线看不卡| 国产片特级美女逼逼视频| 免费少妇av软件| 自拍偷自拍亚洲精品老妇| 国产av国产精品国产| 国产91av在线免费观看| 成人免费观看视频高清| 国产欧美日韩一区二区三区在线 | 最近中文字幕高清免费大全6| 国产视频内射| 日本-黄色视频高清免费观看| 久久影院123| 十八禁高潮呻吟视频 | 秋霞在线观看毛片| 中文字幕人妻丝袜制服| 日韩成人av中文字幕在线观看| 在线观看人妻少妇| 精品久久久久久久久亚洲| 热99国产精品久久久久久7| 久久久久久久久久久丰满| 国产精品.久久久| 亚洲熟女精品中文字幕| 噜噜噜噜噜久久久久久91| 丝袜脚勾引网站| 老女人水多毛片| 国产成人精品福利久久| 亚洲精品aⅴ在线观看| 美女国产视频在线观看| 人人澡人人妻人| 久久久久精品性色| 国产淫语在线视频| 性高湖久久久久久久久免费观看| 日本av免费视频播放| 天堂8中文在线网| 国产精品一二三区在线看| 国产高清不卡午夜福利| 免费久久久久久久精品成人欧美视频 | 激情五月婷婷亚洲| 亚洲av不卡在线观看| 在线观看av片永久免费下载| 在线观看美女被高潮喷水网站| 乱码一卡2卡4卡精品| 永久网站在线| 一级毛片电影观看| 丝瓜视频免费看黄片| 久久免费观看电影| 国产综合精华液| 精品人妻一区二区三区麻豆| 亚洲激情五月婷婷啪啪| 国产亚洲精品久久久com| 少妇高潮的动态图| 欧美国产精品一级二级三级 | av在线观看视频网站免费| 国产高清国产精品国产三级| 中文字幕人妻熟人妻熟丝袜美| 久久99蜜桃精品久久| 我要看日韩黄色一级片| 新久久久久国产一级毛片| 国产精品成人在线| 日韩免费高清中文字幕av| 熟女av电影| 国产欧美另类精品又又久久亚洲欧美| 国产乱来视频区| 免费观看av网站的网址| 亚洲欧美日韩卡通动漫| 男女无遮挡免费网站观看| 又大又黄又爽视频免费| 国产精品久久久久久精品电影小说| 亚洲,欧美,日韩| 国产精品免费大片| 黑人巨大精品欧美一区二区蜜桃 | 久久精品国产亚洲av天美| 久久热精品热| 国产精品久久久久久av不卡| 少妇被粗大的猛进出69影院 | 欧美xxxx性猛交bbbb| 国产av码专区亚洲av| 人人妻人人添人人爽欧美一区卜| 久久婷婷青草| 亚洲综合色惰| 丝袜在线中文字幕| 亚洲精品视频女| 久久国内精品自在自线图片| 免费看光身美女| 啦啦啦在线观看免费高清www| 精品亚洲乱码少妇综合久久| 欧美日韩精品成人综合77777| 妹子高潮喷水视频| av网站免费在线观看视频| 国产精品熟女久久久久浪| 成人毛片60女人毛片免费| 久久久久国产精品人妻一区二区| 深夜a级毛片| 蜜桃久久精品国产亚洲av| 菩萨蛮人人尽说江南好唐韦庄| 丝袜喷水一区| a级毛片免费高清观看在线播放| 日韩成人伦理影院| 在线观看免费高清a一片| 国产有黄有色有爽视频| 最近2019中文字幕mv第一页| 少妇人妻一区二区三区视频| 丝瓜视频免费看黄片| 免费不卡的大黄色大毛片视频在线观看| 青春草视频在线免费观看| 激情五月婷婷亚洲| 免费大片黄手机在线观看| 国模一区二区三区四区视频| 成人毛片a级毛片在线播放| 人妻系列 视频| 美女主播在线视频| 丝袜喷水一区| 国产视频首页在线观看| 人妻一区二区av| 丰满乱子伦码专区| 青春草视频在线免费观看| 桃花免费在线播放| 91精品国产九色| 最新的欧美精品一区二区| 五月天丁香电影| 成人亚洲欧美一区二区av| 嫩草影院新地址| 午夜福利,免费看| 国产男女内射视频| 亚洲精品中文字幕在线视频 | 精品一区二区免费观看| 国产精品伦人一区二区| 人妻系列 视频| 日韩,欧美,国产一区二区三区| 少妇人妻久久综合中文| 国产深夜福利视频在线观看| 国产精品福利在线免费观看| 国产中年淑女户外野战色| 99视频精品全部免费 在线| 观看美女的网站| 欧美日韩亚洲高清精品| 国产极品天堂在线| 少妇 在线观看| 中文天堂在线官网| 一级毛片久久久久久久久女| 国产又色又爽无遮挡免| 多毛熟女@视频| 国产欧美日韩精品一区二区| 午夜av观看不卡| 精品久久久久久电影网| 丝袜在线中文字幕| 亚洲熟女精品中文字幕| 在线观看www视频免费| 亚洲真实伦在线观看| 中文乱码字字幕精品一区二区三区| 你懂的网址亚洲精品在线观看| 美女cb高潮喷水在线观看| 久久女婷五月综合色啪小说| 久久韩国三级中文字幕| 国产精品人妻久久久久久| 国产精品免费大片| 人人妻人人看人人澡| 有码 亚洲区| 新久久久久国产一级毛片| 七月丁香在线播放| 伊人久久国产一区二区| 亚洲国产av新网站| 亚洲国产精品国产精品| 国产 一区精品| 男人狂女人下面高潮的视频| 欧美日韩视频精品一区| 99热6这里只有精品| 亚洲av国产av综合av卡| 下体分泌物呈黄色| av网站免费在线观看视频| 最近的中文字幕免费完整| 在线观看av片永久免费下载| 国产亚洲精品久久久com| 欧美日韩综合久久久久久| 99九九在线精品视频 | 日韩伦理黄色片| 欧美日韩亚洲高清精品| 69精品国产乱码久久久| 国产一级毛片在线| 国产精品成人在线| 国产精品久久久久久久久免| av网站免费在线观看视频| 国产精品一区www在线观看| 边亲边吃奶的免费视频| 狂野欧美激情性bbbbbb| 国产女主播在线喷水免费视频网站| 一本一本综合久久| 国产成人精品福利久久| 欧美区成人在线视频| 一级毛片我不卡| 99视频精品全部免费 在线| 亚洲婷婷狠狠爱综合网| 夫妻性生交免费视频一级片| 免费观看的影片在线观看| 成人毛片60女人毛片免费| 亚洲国产色片| 伊人亚洲综合成人网| 丝袜脚勾引网站| 日产精品乱码卡一卡2卡三| 久久精品国产亚洲网站| 免费人妻精品一区二区三区视频| 国产精品国产三级国产专区5o| av网站免费在线观看视频| 51国产日韩欧美| 免费高清在线观看视频在线观看| 青春草国产在线视频| 久久99蜜桃精品久久| 香蕉精品网在线| 久久久久精品久久久久真实原创| 97在线人人人人妻| 国产一区亚洲一区在线观看| av在线老鸭窝| xxx大片免费视频| 午夜免费观看性视频| 噜噜噜噜噜久久久久久91| 午夜福利,免费看| 日日爽夜夜爽网站| 亚洲成色77777| 一区二区三区乱码不卡18| 欧美xxⅹ黑人| 人妻少妇偷人精品九色| 午夜日本视频在线| 超碰97精品在线观看| 免费播放大片免费观看视频在线观看| 亚洲精品一区蜜桃| 免费不卡的大黄色大毛片视频在线观看| 国产日韩欧美在线精品| 久久人人爽人人爽人人片va| 欧美97在线视频| 成人18禁高潮啪啪吃奶动态图 | 九草在线视频观看| 欧美xxxx性猛交bbbb| 美女国产视频在线观看| 欧美激情国产日韩精品一区| 麻豆成人午夜福利视频| 亚洲国产av新网站| 午夜福利,免费看| 五月天丁香电影| 人人妻人人澡人人看| kizo精华| 99久国产av精品国产电影| 26uuu在线亚洲综合色| 国产精品国产av在线观看| 午夜91福利影院| 黄片无遮挡物在线观看| 精品久久久噜噜| 91精品伊人久久大香线蕉| 亚洲真实伦在线观看| 精品久久久噜噜| 国产成人a∨麻豆精品| 亚洲av国产av综合av卡| av在线老鸭窝| 久久 成人 亚洲| 蜜臀久久99精品久久宅男| 国产色爽女视频免费观看| 久久6这里有精品| 久久免费观看电影| 观看免费一级毛片| 中文字幕av电影在线播放| 这个男人来自地球电影免费观看 | 天天操日日干夜夜撸| 久久精品国产鲁丝片午夜精品| 在线观看美女被高潮喷水网站| 91久久精品电影网| 乱人伦中国视频| 黄色怎么调成土黄色| 天天躁夜夜躁狠狠久久av| 久久久a久久爽久久v久久| 91久久精品国产一区二区成人| 男女啪啪激烈高潮av片| 亚洲国产毛片av蜜桃av| 久久6这里有精品| 久久99精品国语久久久| 欧美一级a爱片免费观看看|