• <tr id="yyy80"></tr>
  • <sup id="yyy80"></sup>
  • <tfoot id="yyy80"><noscript id="yyy80"></noscript></tfoot>
  • 99热精品在线国产_美女午夜性视频免费_国产精品国产高清国产av_av欧美777_自拍偷自拍亚洲精品老妇_亚洲熟女精品中文字幕_www日本黄色视频网_国产精品野战在线观看 ?

    Continuous-Time Prediction of Industrial Paste Thickener System With Differential ODE-Net

    2022-04-15 04:03:56ZhaolinYuanXiaoruiLiDiWuXiaojuanBanNaiQiWuHongNingDaiandHaoWang
    IEEE/CAA Journal of Automatica Sinica 2022年4期

    Zhaolin Yuan, Xiaorui Li, Di Wu, Xiaojuan Ban, Nai-Qi Wu,,Hong-Ning Dai,, and Hao Wang,

    Abstract—It is crucial to predict the outputs of a thickening system, including the underflow concentration (UC) and mud pressure, for optimal control of the process. The proliferation of industrial sensors and the availability of thickening-system data make this possible. However, the unique properties of thickening systems, such as the non-linearities, long-time delays, partially observed data, and continuous time evolution pose challenges on building data-driven predictive models. To address the above challenges, we establish an integrated, deep-learning, continuous time network structure that consists of a sequential encoder, a state decoder, and a derivative module to learn the deterministic state space model from thickening systems. Using a case study, we examine our methods with a tailing thickener manufactured by the FLSmidth installed with massive sensors and obtain extensive experimental results. The results demonstrate that the proposed continuous-time model with the sequential encoder achieves better prediction performances than the existing discrete-time models and reduces the negative effects from long time delays by extracting features from historical system trajectories. The proposed method also demonstrates outstanding performances for both short and long term prediction tasks with the two proposed derivative types.

    I. INTRODUCTION

    AS a core procedure in modern mineral separation, a thickening process produces a paste with high concentration for subsequent tailing storage or backfilling[1]–[3]. During this thickening process, an industrial paste thickener achieves solid–liquid separation based on gravity sedimentation. The purpose of the industrial paste thickener is to efficiently control the final underflow concentration (UC).Most closed-loop control systems manipulate the underflow slurry pump speed and flocculant pump speed as the inputs to stabilize the underflow concentration within its specified range during operation. Previous studies [4], [5] showed that model prediction control (MPC) can facilitate the control process of thickening systems owing to the advantages of its high robustness and applicability. Hence, the accurate prediction of thickening systems has received extensive attention for the analysis and control of thickeners [5]–[7].

    A complex industrial system such as the thickening system typically has the following key features:

    1) Non-Linear System Dynamics:Most industrial systems have extremely complex high-order dynamical equations that are not affine or linear systems.

    2) PartiaLly Observed Data:The information extracted from sensors or other available methods is incomplete. In particular, a number of unknown hidden variables exist in such systems.

    3) Influence of Long Delays:The system states are influenced by external inputs or internal states that occur over a long previous time.

    4) Continuous-Time (CT) Evolution:Because real industrial systems follow various physical laws, their time evolution can be expressed via CT differential equations.

    The above features of a thickening system create challenges for predictive control. There are a number of studies addressing these challenges. Data-driven methods are emerging as one of the most successful techniques for modeling complex processes [8]. Traditional data-based CT system prediction methods focus on fitting high-order differential equations based on sampled noisy data from real systems. However, they lack the ability to cope with partially observed and extremely complex system dynamics. Recent advances of deep neural networks (DNNs) have shown their strengths in addressing these issues owing to their strong feature representation abilities and scalable parameter structures, leading to the wide usage of DNNs in computer vision [9]–[11], natural language processing [12], [13], time series prediction [5], [14]–[17], and fault diagnosis [18].However, most DNN-based system-modeling methods are based on discrete time, disregarding the CT properties of a system. The lost prior information from physical insights undoubtedly leads to the deterioration of the model accuracy.

    Studies of prediction control can be categorized into two types. The first type provides short-term predictions for model-based control algorithms [5], such as model predictive control (MPC). The learned system provides deterministic prior knowledge of the dynamical systems, thereby approximating the infinite-horizon optimal control as a shortterm optimization problem. The second type is mainly based on simulations, which imitate the outputs of an unknown system under a long-term feed of inputs [19]. Compared with short-term predictions, simulations require higher robustness and stability to provide long-term predictions. However, there are few studies on designing predictive models for short and long term predictions to support subsequent applications, such as MPC and simulations.

    To address the above challenges, we propose a deep CT network composed of a sequential encoder, a state decoder,and a derivative module to learn the auto-regressive processes and influences from the system inputs based on real thickening data in an end-to-end manner. Specifically, the long-time system delay motivated us to utilize a sequential encoder to extract features from historical system trajectories.We designed the derivative module for the CT state space model based on a DNN. This module fits the non-linear CT evolution of the system and infers the non-observable information by introducing hidden states. Moreover, the problems of short and long term predictions are solved by feeding historical system trajectories and system inputs with arbitrary lengths to the model after incorporating the designed non-stationary and stationary systems into the trained model.The future system outputs are then predicted. The contributions of this paper are threefold:

    1) We propose a novel deep-learning-based CT predictive model for a paste thickener. The deep learning network consists of three components: a sequential encoder, a state decoder, and a derivative module.

    2) We design two kinds of derivative modules, named stationary and non-stationary systems, to handle the shortterm and long-term prediction tasks, respectively.

    3) We conduct extensive experiments on real industrial data collected from a real industrial copper mining process. The results demonstrate the outstanding performance of the proposed model in providing predictions for the thickener system with non-linear and time-delay properties. In addition,we conduct ablation studies to evaluate the effectiveness of each module in the proposed model.

    The rest of the paper is organized as follows. We briefly introduce the related work in Section II. We then present the problem formulation in Section III. We next present the CT deep sequential model in Section IV. Experimental results are shown in Section V. We then summarize the paper and discuss future research directions in Section VI.

    II. PRELIMINARIES AND RELATED WORK

    As a core device in a thickening system, a paste thickener is generally composed of a high sedimentation tank and a raking system. Fig. 1 depicts the general structure of a thickener and its key components. After being fed with flocculant and tailing slurry with a low concentration, underflow with a high concentration is discharged from the bottom of the thickener and is then used to produce paste in the subsequent procedures. The prediction of a thickening system refers to the estimation of the future system outputs, such as the underflow concentration and mud pressure based on the historical system trajectories and system inputs. The prediction of a thickening system is essentially similar to system identification [8]. For system identification, interpretable model structures are designed based on prior knowledge, and the parameters are determined by fitting real data. As one of the subsequent applications of identified models, the prediction forecasts the system outputs according to the inputs.

    Fig. 1. Crude slurry flow with a low concentration is fed into the mix tank accompanied by flocculant. Under the effect of the flocculant, particles agglomerate to larger clumps and concentrate at the bottom. The paste thickener continuously produces underflow with a high concentration and clear water in an overflow pipe located at the top of the thickener.

    A. Prediction of Thickening Systems

    The prediction methods for thickening systems can be categorized into two types: 1) gray-box thickening system simulations and 2) black-box thickener system predictions. In the gray-box simulations, the sedimentation process is mainly considered from a physical perspective [4], [20], [21]. Theorybased gray-box methods can be exactly explained and implemented effectively for specific systems. However, most of them are mainly built on many ideal hypotheses and suffer from the complexity of slurry particle dynamics and external unknown environment disturbances.

    In contrast, the black-box methods do not require prior assumptions or constraints to be given. A complete parameterized model with a high degree of freedom is defined to predict the system outputs and learn the optimal parameters from real data. Since the offline system trajectories from an industrial system are always available and adequate, blackbox-based methods, including latent factor model [22]–[24],imitation learning [25], and deep neural network [26] have been widely used in the current industrial systems [27]–[32].

    Moreover, Oulhiqet al. [33] used a black-box linear dynamic model with a deterministic time delay to identify an industrial thickener system using historical data. Such parameterized linear model lacks adequate expressivity to represent the non-linear properties in a thickening system.Most recently, random forest model is presented for modeling a paste thickening system based on a purely data-driven approach for modeling and evolutionary strategies [34].Because random forest model only fits the thickening system dynamic in single step, it ignores the time delay and the correlations between adjacent positions in sequential inputs and outputs. A bidirectional gated recurrent unit (BiGRU)with an encoder–decoder deep recurrent neural network is introduced to model thickening systems [5]. Yuanet al. [6]proposed a dual-attention recurrent neural network to model the spatial and temporal features of a thickening system,thereby improving the prediction accuracy of the underflow concentration. However, the above studies [5], [6] only focus on discrete-time system predictions rather than a CT thickening system.

    B. Prediction of Continuous-Time Systems

    The prediction of physical systems based on CT models directly from sampled data has the following advantages [8]:1) transparent physical insights into the system properties, 2)inherent data filtering, and 3) the capability of dealing with non-uniformly sampled data. For any numerical schemes for solving CT differential equations, sophisticated discretization methods have high accuracy but suffer from enormous time and memory costs. A recently developed advanced ordinary differential equation (ODE) solver [35] introduces the reversemode automatic differentiation of ODE solutions, thereby only requiring O(1) memory cost. Meanwhile, this method also allows the end-to-end training of ODEs within a large DNN. Moreover, Demeester [19] proposed a stationary CT state space model for predicting an input–output system when the observations from a system are unevenly sampled.Although it has successfully improved the accuracy and stability of long-term predictions by introducing a stationary system, it did not take advantage of a non-stationary system in the short-term prediction task.

    Compared with the existing CT models, our model considers both short-term and long-term predictions, thereby achieving outstanding performance.

    III. FORMULATION AND NOTATION OF PASTE THICKENING SYSTEM PREDICTION

    Fig. 2. Proposed model is composed of several components. The recurrent neural network (RNN) encoder outputs the initial hidden state h(t0) according to historical sequences from the system. The derivative module is embedded into the ordinary differential equation (ODE) solver to calculate the hidden state at an arbitrary time. A parallel interpolation mechanism is embedded into the derivative module, which interpolates discrete input sequences to a CT series. Finally,the state decoder module transforms the hidden state to the predicted system output.

    IV. CONTINUOUS-TIME DEEP SEQUENTIAL MODEL FOR THICKENING SYSTEM PREDICTIONS

    A. RNN for Encoding Historical Sequences

    industrial experience, we use the system delay as prior knowledge denoted byTd. If we assume a uniform sampling of the sensors for a sampling intervalTs,NyandNxcan be estimated according to the equationNy=Nx=N=Td/Ts.The influence of parameterNon the model accuracy is examined in Section V. In the thickening system, the correlations between the current system state and historical trajectories are mostly compressed in the short term. This property encourages us to use a simple and unidirectional RNN to encode the historical trajectories of a thickening system. The solved hidden state h(t0) from encoder involves all necessary information of historical trajectories and is represented as an initial state of solved ODE.

    B. ODE Solver for Modeling State Space

    We employ the parameterized CT state space model to represent the relations between the system inputs, hidden states, and outputs:

    Generally, an ODE solver with a lower error tolerance increases the frequency for calling differential functiond. It leads to more time consumption but results in a higher accuracy. This guideline is also tenable when we construct a neural ODE network to fit sequential datasets. The detailed comparisons of the time cost and accuracy are shown in Section V.

    It is worth investigating the definition of a suitable structure ofd. The most intuitive solution is to employ a basic neural network to estimate the derivative that is named nonstationary model.

    Non-Stationary:

    where MLP(·) denotes a multi-layer perceptron. The combination of a non-stationary system with an ODE solver has a strong similarity to residual connections, which have been widely used in other advanced deep networks [35].

    In the field of stochastic process analysis, non-stationary systems are a stochastic process with a mean and covariance that vary with respect to time [40]. Differencing [41] is an effective way to make non-stationary time series stationary by eliminating trend and seasonality. Generally, a thickening system has strong trends in the underflow concentration, mud pressure, and other core variables. The thickening system is an approximation of non-stationary systems, indicating that the differencing operation can improve the fitting accuracy. In(11), the derivative module intrinsically learns the first-order difference of the hidden states in the latent space. In contrast to the operation of differencing the system outputs directly, a model that differences the hidden states has an equivalent or stronger ability to represent a non-stationary system that is of first or even higher order. However, the non-stationary system(11) also suffers from a severe problem when handling longterm prediction tasks. To solve an ODE over long intervals,repetitive accumulation in a CT range can lead to a significant magnitude increase of the hidden states. Consequently, the estimation error will grow accordingly, resulting in the difficulties in achieving accurate system output from the decoder.

    Therefore, we devise another derivative module, namely,the stationary system, to handle the long-term prediction problem. In particular, we have

    Stationary:

    In a stationary system, we use a GRU to construct the derivative module because it has a strong ability to carry longtime information. In a non-stationary system, consecutive accumulations diffuse the hidden state in an unconstrained range. Thus, we employ the MLP to learn the first-order difference h˙(t) directly under h(t) and x(t).

    C. Parallel Spline Interpolation

    Note that the calculation of h˙(t) in (11) and (12) depends on the external input x(t), which may not exist in our dataset.External input sequences XMfin the training data are discrete,while the computation of the ODE needs x(t) in a CT range.Before each forward pass of the network, it is necessary to interpolate the external inputs to the continuous form. Deep networks are typically trained in mini-batches to take advantage of efficient data-parallel graphics processing unit(GPU) operations. Therefore, we implement a parallel spline interpolation mechanism on top of PyTorch, which is a wellknown deep learning framework.

    In our dataset, the external input data are evenly sampled,thereby simplifying the implementation of parallel interpolation. To simplify the explanation, we assume that the dimension of the external input is equal to 1.

    D. State Decoder

    The state decoder mechanism is essentially a fully connected network. We therefore have the following equation to represent the output,

    Compared with other state space models that only employ a single matrix for decoding, the nonlinear decoder is chosen because the accumulative form in (11) causes the range of the input h(t) to be non-deterministic. The activation function tanh(·) constrains the output of the decoder to a rational range.

    E. Model Training

    Since all of the operations of the ODE solver in our model are smooth and differentiable, we can train the complete model by the standard back-propagation algorithm with the loss function defined as follows:

    V. EXPERIMENTAL RESULTS

    This section presents experimental results for the proposed method on the dataset of real thickening systems. We mainly investigate three issues: RQ1: What are the advantages of employing a CT deep sequential network with a high-accuracy ODE solver for modeling a thickening system? RQ2: What are the pros and cons of using stationary and non-stationary systems in prediction tasks? RQ3: How do the different interpolation methods and sequential encoder affect the accuracy of the proposed CT model? We first describe the dataset, the hyper-parameters of the model, and the training and test configurations. We then present the detailed experimental results.

    A. Thickening System Dataset

    For our experiments, the dataset was collected from the paste thickener manufactured by the FLSmidth from the NFC Africa Mining PLC, Zambian Copperbelt Province. Fig. 4 illustrates two identical thickeners in our experiments. They are used to concentrate copper tailings to produce paste in the backfilling station. Both devices operate in the closed-loop mode with PID controllers.

    Some key technical parameters of the studied thickener are listed in Table I.

    The measured data are sampled evenly with two-minute intervals from May 2018 to February 2019. A short piece of original dataset is shown in Table II. The collected data come from seven monitoring sensors just as the defined y(k) and x(k)in Section III. After deleting the records corresponding to the time when the system was out of service, there are 24 673 pieces of data remaining.

    Fig. 3. Illustration of the process of building both the training, validation, and test datasets. An independent data tuple for training or testing is composed of four vector sequences. X[i:i+N] and Y[i:i+N] represent the historical trajectories in conditional range. X[i+N:i+N+M] and X[i+N:i+N+L] represent the inputs sequences, which have equivalent length with predicted sequences. Y[i+N:i+N+M] and Y[i+N:i+N+L] represent the real system outputs. The former is utilized to generate optimized loss in training and the latter is only used in testing and validation phase for evaluating the accuracy of prediction.

    Fig. 4. The figures illustrate two identical paste thickeners in our experimental mining station, including one primary and one alternate thickener. Both devices operate in closed-loop mode with proportionalintegral-derivative (PID) controllers.

    TABLE I SOME KEY TECHNICAL PARAMETERS OF THE THICKENER

    We employ the first 70% of the entire dataset to train the model. In the remaining 30% portion, the first 15% is used for validation to determine the best training epochs, and the other 15% is the test dataset for evaluating the model accuracy. By splitting and building the inputs-outputs sequential tuples according to Fig. 3, there are 17 131 tuples left for training and 3561, 3421, 3121 tuples left for testing and validation forL=60, 200, 500, respectively. All the datasets are normalized to standard normal distributions with a unified mean and variance before the training and test phases.

    B. Experimental Setup

    We use the mini-batch stochastic gradient descent (SGD)with the Adam optimizer [42] to train the models. The batch size is 512, and the learning rate is 0.001 with an exponential decay. The decay rate is 0.95, and the period for decay is 10 epochs. The size of the hidden state h(t) in ODE is 32. The RNN encoder module has a single hidden layer and the size is equal to 32 that is consistent with the size of hidden state h(t).The size of the hidden layer in state decoder is 64. In both the adaptive ODE solvers, the time for solving the ordinary differential equations increases if we reduce the tolerance of approximate error. For balancing the time cost and accuracy,we set the relative tolerance to 1E–4 and absolute tolerance to 1E–5 in all of experiments.

    During the training procedure, the length of the historical sequences denoted byNis 80, and the length of predicted outputs denoted byMis 60. The best-performing model in the validation dataset is chosen for further evaluation with the test dataset. The training and test phases were performed on a single Nvidia V100 GPU. The implementation uses the PyTorch framework. We define the CT range as0 ≤t≤Mδtfor given discrete integral indices [0,1,...,M]. The time interval δtof the adjacent data points is set to 0.1.Accordingly, the normalized factor μtin (12) is also set to 0.1.When we use the Euler approximation to solve an ODE equation in a stationary system, the predicted hidden state in the next time step is equal to the output of the GRU cell corresponding to the discrete-time system:

    C. Results and Discussion

    1) Main Results

    We investigate the influence of the types of ODE solversand system types. We select four ODE solvers: Euler, Mid-Point, fourth-order Runge-Kutta (RK4), Dormand-Prince(Dopri5) [35], and 3-order Bogacki-Shampine (Bosh) [43].We investigate the performances of those ODE solvers in both non-stationary and stationary systems. To make a trade-off between the model accuracy and time consumption, we set the relative tolerance to 1E–4 and the absolute tolerance to 1E–5.Moreover, we also consider the discrete-time deep sequential model for the state space (DT-State-Space), the attentionbased Seq2Seq model (Attention-Seq2Seq) [5], and Transformer [44] for comparison. The DT-State-Space [45]model employs a parameterized per-time-series linear state space model based on a recurrent neural network (RNN) to forecast the probabilistic time series. The sizes of state space and RNN hidden layer are set to 16 and 32, respectively. The hyperparameters setting of Transformer and Attention-Seq2Seq are kept with the original literature.

    TABLE II A TABULAR EXAMPLE OF PASTE THICKENING SYSTEM DATASET

    We conduct three groups of experiments to investigate the RRSEs, MSEs, and time consumption of models with prediction lengths ofL=60, 200, and 500.

    a) Comparison of proposed and other baseline models:

    We first examine the performance of the Attention-Seq2Seq model, the DT-State-Space model, and Transformer, which are defined in discrete-time settings in Table III. Although they perform competitively, better than the proposed models with the Euler ODE solver, they perform worse than the models with high-order ODE solvers, especially for long-term predictions. The results also indicate that employing a CT model is consistent with the features of the CT evolution in thickening systems, thereby improving the prediction accuracy.

    b) Comparison of different ODE solvers:

    We analyze the comparisons of different ODE solvers respectively from stationary system and non-stationary system. When the derivative module is defined as a nonstationary system and we only focus on short-term prediction withL=60, we find that the Euler method achieves relatively higher RRSE and MSE values (i.e., poorer prediction performances) than the other four ODE solvers, though it has a much lower time consumption than the other solvers. As the simplest method for solving ODEs, the Euler method evaluates the derivative network only once between two adjacent time points. Meanwhile, the Mid-Point and RK4 methods have higher prediction accuracies than the Euler method, since they evaluate the derivative network two and four times, respectively, between two adjacent time points.Moreover, the Dopri5 and Bosh methods achieve better accuracies, though they have higher time consumptions.Dopri5 performs slightly better than Bosh. As adaptive methods in the Runge-Kutta family, the Dopri5 and Bosh methods ensure that the output is within a given tolerance of the true solution. Their time consumptions for solving an ODE equation increase as the accuracy tolerance is decreased.

    Strangely, with the increase in the prediction length, we find that the accuracies of non-stationary models crash gradually and the degradation of Euler is slightly lower than the others.The reason of this inconsistent phenomenon is that nonstationary system brings accumulative errors in long-term predictions. High-order ODE solvers evaluate the derivative module more times recursively, which brings more accumulative errors. Not only do the high-order solvers not improved the accuracies of non-stationary system in long-term predictions, they made the accuracies worse.

    When the derivative module is switched to a stationary system. It is worth mentioning that the time consumption for the two adaptive methods, Bosh and Dopri5, to solve an ODE equation significantly increases. We do not list the accuracies of the Dopri5 and Bosh for the stationary systems in Table III because the extremely slow speed makes the method ineffective for practical applications. According to the comparison of the ODE solvers, the high-order ODE solver,such as RK4, results in lower fitting errors than the low-order methods while requiring more time to evaluate the ODE equations intensively.

    c) Comparison of stationary models and non-stationary models:

    For comparing the distinctions between stationary models and non-stationary models more intuitively, we further visualize the prediction performance of the non-stationary and stationary systems with different ODE solvers. Fig. 5 depicts the predicted sequences of the non-stationary and stationary systems with different ODE solvers for the short-term prediction task withL=60. The results show that the nonstationary models outperform the stationary models in shortterm prediction tasks. The estimated sequences from the nonstationary models are slightly closer to the real system output than those from the stationary models. The learning process of a non-stationary system is essentially equivalent to differencing the hidden state and employing the MLP network to learn the relatively stationary first-order difference.Furthermore, the non-stationary models can predict the system outputs smoothly because the non-stationary structure limitsthe hidden states to only changing in a continuous and slow manner. This constraint is consistent with the properties of a slow thickening system that shrinks the searching space of the model parameters to prevent overfitting.

    TABLE III ROOT RELATIVE SQUARED ERROR (RRSE), MEAN SQUARED ERROR (MSE), AND TIME CONSUMPTION OF PREDICTED UNDERFLOW CONCENTRATION

    Fig. 5. In the short-term prediction task with L=60, the non-stationary models output more stable and accurate sequences than the stationary models.

    Fig. 6 presents the experimental results of a long-term prediction task withL=200 (similar results withL=500 can be found in Table III). The tabular results in Table III demonstrate that the RRSE and MSE of non-stationary models are much higher than those of the stationary ones in long-term prediction task, which is consistent with the graphical results.Compared to the excellent results of non-stationary models in Fig. 5, Fig. 6(a) shows that the prediction accuracy for the non-stationary system decays significantly, and the predicted outputs deviate from the true outputs gradually with the increase in the prediction length. However, the predicted results of stationary models are stabilized and closed to the true system outputs, which confirms the excellent accuracies of stationary models in the long-term prediction problem. The structure of non-stationary ODE leads to the hidden state in progressive evolution that is unconstrained and gradually expanding. Although we embed a tanh function for the decoder network to restrict the final prediction of the underflow concentration and pressure to a rational range, it is impossible for the decoder module to learn an effective mapping function from an extremely large hidden state space to the system output space. Similarly, Fig. 6 also demonstrates that high-order ODE solvers, such as the 4th-order Runge-Kutta, still perform slightly better than the low-order solvers,such as Euler, in long-term prediction.

    We conduct five other groups of experiments with different values of the prediction length to evaluate the prediction performance (i.e., the MSE) of the underflow concentration and ground-truth for both stationary and non-stationary systems. The results in Fig. 7 show that the non-stationary system performs better than the stationary system in the shortterm prediction task (e.g.,L<100), although the stationary system outperforms the non-stationary system in long-term prediction tasks. For example, whenLexceeds 120, the errors from the non-stationary system increase with the predicted length, while the stationary system significantly stabilizes the accumulative errors in the long-term prediction.

    Fig. 6. In the long-term prediction task with L=200, Fig. 6(a) illustrates that non-stationary models only performed well in the early time horizon. In the late horizon, the predicted sequences deviated from the true system outputs significantly compared with those of the stationary models, which are shown in Figs. 6(b) and 6(c). The results also indicate that the models with high-order ODE solvers performed better than the models with low-order ODE solvers in long-term prediction tasks.

    Fig. 7. Predicted length L affected the accuracy (log10 MSE ± 2σ, computed across five runs) of predicted underflow concentration for both stationary and non-stationary systems.

    2) Experiments for Evaluating Interpolation Order

    We next investigate the effect of the interpolation method on the prediction accuracy. We test four spline interpolation methods with different orders and compare the prediction accuracies on test datasets. The results in Table IV demonstrate that the higher-order interpolations slightly outperform the lower-order ones. This proves that the system input of the thickening system is a non-linear complex process, and that the information from external inputs is essential for predicting the outputs of the system. Higherorder spline interpolations exploit more correlational features from adjacent inputs and interpolate the empty area with a better accuracy than the lower-order interpolations.

    3) Ablation Experiments for Studying System Time Delays and Improvements From Sequential Encoder

    Finally, we investigate the significance of introducing the sequential encoder to confront the system time delay. We investigate the influence ofNon the model accuracy.Specially, whenNis set to 1, the sequential encoder is replaced by a neural network with one hidden layer that encodes the system output x(k?1) in a single time step to the initial hidden state h(t0). WhenNis set to 0, the initial state h(t0)is a learnable or zero vector [19] that had no relationship with historical system trajectories. We examine the different choices ofNin experiments withL=60, 200, and 500,respectively. In the experiments withL=60, the derivative module is set to be a non-stationary system with an MLP cell.We change it to a stationary system with a GRU cell whenL=200and 500. The ODE solver is the fourth-order Runge-Kutta solver for all of the models.

    The results shown in Table V demonstrate that the introduction of the sequential encoder to extract features from the historical sequence leads to better performance than those in the cases withN=1 or 0. The intuitive explanation is that the predicted output sequences have strong statistical correlations with historical system trajectories. The optimal length of the encoded sequence is approximatelyN=80,which is consistent with our prior experience of 2–3-h time delays in thickening systems. When the length of input sequence exceeds the optimal value, the accuracy slightly decreases.

    Intuitively, the short-term prediction task benefits more from historical system trajectories than the long-term prediction task. When the length of the predicted sequence increases, the advantage brought by the sequential encoder also decreases. In the task withL=500, the profit of employing sequential encoder decreases obviously.

    VI. CONCLUSIONS

    This paper focuses on the prediction of the outputs of a thickening system based on deep neural sequence models. We introduce a CT network composed of a sequential encoder, a state decoder, and a derivative module, with internal computation processes including interpolation and a differential ordinary differential equation solver, to describe the complex dynamics of a thickening system. Experiments on datasets from real thickening systems demonstrate that the introduction of the sequential encoder and parallel cubic spline interpolation play a crucial role in our model architecture. We conducted extensive experiments to evaluate the proposed models for both stationary and non-stationary systems with different ODE solvers. The results show that the non-stationary system outperforms the stationary system for short-term prediction tasks. However, the non-stationarymodel suffers from the accumulation of errors from the incremental calculation, thereby leading to inferior results in long-term prediction tasks. This demonstrates that the model with the non-stationary system is more suitable for being embedded in a model-based feedback controller (e.g., MPC controller) while the stationary system avoids this problem and performs better in long-term prediction tasks. Therefore,the model with the stationary system is a better choice when a stable and robust identified system is required to predict longterm sequences (e.g., simulations or controller testing).

    TABLE IV ACCURACY COMPARISONS WITH DIFFERENT ORDERS OF INTERPOLATIONS

    TABLE V ACCURACY COMPARISONS WITH DIFFERENT METHODS FOR GENERATING THE INITIAL HIDDEN STATE h(t0)

    In the industrial data processing field, it is a common requirement to process unevenly spaced data. Although the dataset employed in this paper is sampled evenly, we can extend our method to deal with uneven data naturally by adjusting the time intervals. This extension deserves further experimental verification in future work. Another promising research direction is to extend the method to probabilistic generative models and perturbed time-varying models [46] for determining the unknown sampling noise and uncertainty in thickening systems. Moreover, it is worth investigating our method for other dynamical industrial systems.

    亚洲图色成人| 亚洲av日韩在线播放| 日韩中字成人| 91精品国产国语对白视频| 熟女人妻精品中文字幕| 欧美一级a爱片免费观看看| 伦理电影大哥的女人| 国产精品成人在线| 天美传媒精品一区二区| 亚洲精品中文字幕在线视频 | 亚洲av不卡在线观看| 亚洲精华国产精华液的使用体验| 在线天堂最新版资源| 成人黄色视频免费在线看| 亚洲精品国产成人久久av| 亚洲精品日本国产第一区| 国产日韩欧美亚洲二区| 人体艺术视频欧美日本| 国产精品秋霞免费鲁丝片| 国产黄片视频在线免费观看| 久热久热在线精品观看| 人妻制服诱惑在线中文字幕| 99热国产这里只有精品6| 日韩成人av中文字幕在线观看| 免费黄色在线免费观看| 一级毛片aaaaaa免费看小| 欧美xxⅹ黑人| 一级a做视频免费观看| av不卡在线播放| 亚洲精品一区蜜桃| 一个人看的www免费观看视频| 五月伊人婷婷丁香| 国产成人a∨麻豆精品| 亚洲精品aⅴ在线观看| 日韩制服骚丝袜av| 日韩免费高清中文字幕av| 亚洲图色成人| 人人妻人人澡人人爽人人夜夜| 日韩伦理黄色片| 久久青草综合色| 欧美3d第一页| 五月开心婷婷网| 欧美精品一区二区免费开放| 亚洲色图综合在线观看| 午夜精品国产一区二区电影| 国产一区二区三区综合在线观看 | 亚洲av在线观看美女高潮| 成人高潮视频无遮挡免费网站| 亚洲欧美精品专区久久| 搡老乐熟女国产| 狂野欧美白嫩少妇大欣赏| 80岁老熟妇乱子伦牲交| 中文欧美无线码| 青春草亚洲视频在线观看| 少妇人妻精品综合一区二区| 人妻一区二区av| 国产精品福利在线免费观看| 久久精品久久精品一区二区三区| 亚洲人成网站在线播| 成人高潮视频无遮挡免费网站| av国产免费在线观看| 人人妻人人添人人爽欧美一区卜 | 在线观看av片永久免费下载| 成人18禁高潮啪啪吃奶动态图 | 国产精品精品国产色婷婷| 秋霞在线观看毛片| 日韩,欧美,国产一区二区三区| 国产一区二区三区av在线| 国产黄片视频在线免费观看| 老师上课跳d突然被开到最大视频| 国产男女超爽视频在线观看| 一本—道久久a久久精品蜜桃钙片| 亚洲综合精品二区| 欧美一级a爱片免费观看看| av在线蜜桃| 久久人人爽人人爽人人片va| 麻豆精品久久久久久蜜桃| 熟女av电影| 亚洲欧美精品专区久久| 六月丁香七月| tube8黄色片| 欧美三级亚洲精品| 久久久精品94久久精品| 成人二区视频| 亚洲色图av天堂| 久久99热这里只有精品18| 丰满少妇做爰视频| 精品视频人人做人人爽| 国产精品成人在线| 丰满少妇做爰视频| 少妇的逼水好多| 日韩欧美 国产精品| 国产精品免费大片| 国产精品久久久久久精品古装| 久久精品国产亚洲av涩爱| av免费观看日本| 久久6这里有精品| 亚洲人成网站在线播| 久久久久久久久久人人人人人人| 久久久久久久久大av| 日韩免费高清中文字幕av| 国产成人精品一,二区| 成人毛片60女人毛片免费| 有码 亚洲区| 亚洲欧美日韩另类电影网站 | 亚洲国产欧美人成| 99热6这里只有精品| 在线播放无遮挡| av播播在线观看一区| 日本av手机在线免费观看| 午夜精品国产一区二区电影| 国产免费一级a男人的天堂| 日韩av免费高清视频| 亚洲欧美日韩另类电影网站 | a级毛片免费高清观看在线播放| 亚洲精品第二区| 国产成人一区二区在线| 3wmmmm亚洲av在线观看| 日韩免费高清中文字幕av| 男女无遮挡免费网站观看| 国内少妇人妻偷人精品xxx网站| 我要看日韩黄色一级片| av黄色大香蕉| 国产成人精品久久久久久| 国产男女内射视频| 建设人人有责人人尽责人人享有的 | 在线观看免费视频网站a站| 久久精品人妻少妇| 午夜福利视频精品| 在线观看人妻少妇| 亚洲av欧美aⅴ国产| 久久影院123| 干丝袜人妻中文字幕| 少妇高潮的动态图| 国产亚洲欧美精品永久| 国产精品久久久久久久久免| 国产成人精品久久久久久| 狂野欧美激情性xxxx在线观看| 免费看av在线观看网站| 高清午夜精品一区二区三区| 久久影院123| 亚洲美女搞黄在线观看| 国产乱人视频| 久久韩国三级中文字幕| a级一级毛片免费在线观看| 欧美亚洲 丝袜 人妻 在线| 亚洲色图综合在线观看| 亚洲丝袜综合中文字幕| 少妇丰满av| 一本久久精品| 人人妻人人看人人澡| 日韩中字成人| 国产 精品1| 国产毛片在线视频| 99久久精品热视频| 免费在线观看成人毛片| 欧美日韩综合久久久久久| 又粗又硬又长又爽又黄的视频| 日韩一区二区三区影片| 成人亚洲欧美一区二区av| 国产男人的电影天堂91| 国产精品久久久久久精品古装| av天堂中文字幕网| 久久ye,这里只有精品| 如何舔出高潮| 一级毛片黄色毛片免费观看视频| 激情 狠狠 欧美| 欧美日韩亚洲高清精品| 成人毛片a级毛片在线播放| 黄色配什么色好看| 亚洲内射少妇av| 一级av片app| 插逼视频在线观看| 五月玫瑰六月丁香| 亚洲人成网站在线播| 我的老师免费观看完整版| 青春草亚洲视频在线观看| 欧美成人午夜免费资源| 国产黄色免费在线视频| 国产精品久久久久成人av| 日日摸夜夜添夜夜添av毛片| 中国三级夫妇交换| 18禁动态无遮挡网站| 黄色一级大片看看| 欧美少妇被猛烈插入视频| 欧美极品一区二区三区四区| 久久久午夜欧美精品| 99久久综合免费| 男女无遮挡免费网站观看| 亚洲精品日韩av片在线观看| 中文乱码字字幕精品一区二区三区| 中国美白少妇内射xxxbb| 国产伦精品一区二区三区视频9| 免费观看的影片在线观看| 久久久欧美国产精品| 草草在线视频免费看| av女优亚洲男人天堂| 日韩,欧美,国产一区二区三区| 久久久久视频综合| 亚洲美女搞黄在线观看| 一区二区三区精品91| 精品久久久精品久久久| 内射极品少妇av片p| 国产成人免费观看mmmm| 日本黄色片子视频| 1000部很黄的大片| 美女内射精品一级片tv| 中国美白少妇内射xxxbb| 亚洲av男天堂| 如何舔出高潮| 久久久午夜欧美精品| 一区二区三区免费毛片| 精品少妇黑人巨大在线播放| 夫妻午夜视频| 最近最新中文字幕大全电影3| 在线观看免费日韩欧美大片 | av国产精品久久久久影院| 精品久久久噜噜| 国产一区二区三区av在线| 国产亚洲精品久久久com| 内射极品少妇av片p| 亚洲自偷自拍三级| 18禁动态无遮挡网站| 成人毛片a级毛片在线播放| 91精品国产国语对白视频| 国产av一区二区精品久久 | 国产69精品久久久久777片| 欧美日韩综合久久久久久| 18禁在线播放成人免费| 人妻 亚洲 视频| 久久99热这里只有精品18| 99热全是精品| 男的添女的下面高潮视频| 天天躁日日操中文字幕| 91狼人影院| 亚洲国产欧美在线一区| 亚洲精品乱码久久久v下载方式| 亚洲精品视频女| 一区二区三区免费毛片| 免费大片黄手机在线观看| 国产精品免费大片| 亚洲av成人精品一区久久| 婷婷色麻豆天堂久久| 性色avwww在线观看| 中国美白少妇内射xxxbb| 日韩三级伦理在线观看| 精品亚洲乱码少妇综合久久| 精华霜和精华液先用哪个| 精品一区二区三区视频在线| 国产熟女欧美一区二区| 91精品国产国语对白视频| 蜜桃在线观看..| 国产精品国产三级国产av玫瑰| 国产一级毛片在线| 亚洲精品久久久久久婷婷小说| 插阴视频在线观看视频| 国产精品.久久久| 亚洲精品国产色婷婷电影| 丝袜喷水一区| 国内揄拍国产精品人妻在线| 亚洲av综合色区一区| 身体一侧抽搐| 日韩免费高清中文字幕av| 日本黄色日本黄色录像| 深爱激情五月婷婷| 国产淫语在线视频| 国产欧美另类精品又又久久亚洲欧美| 高清在线视频一区二区三区| 最新中文字幕久久久久| 免费看日本二区| 午夜福利高清视频| 国产精品熟女久久久久浪| 亚洲成色77777| 欧美一区二区亚洲| 国产精品国产av在线观看| 十分钟在线观看高清视频www | 中文字幕人妻熟人妻熟丝袜美| 亚洲欧美精品专区久久| 国产精品国产三级国产专区5o| 免费观看av网站的网址| 一区二区三区免费毛片| 国产黄片美女视频| 偷拍熟女少妇极品色| 久久婷婷青草| 精品午夜福利在线看| 亚洲人成网站在线播| 只有这里有精品99| 51国产日韩欧美| 国产熟女欧美一区二区| 欧美日韩国产mv在线观看视频 | 大香蕉97超碰在线| 九九在线视频观看精品| av卡一久久| 久久av网站| 国产亚洲91精品色在线| 极品少妇高潮喷水抽搐| 国产精品秋霞免费鲁丝片| 亚洲成人一二三区av| 亚洲激情五月婷婷啪啪| 日韩亚洲欧美综合| 免费观看在线日韩| 能在线免费看毛片的网站| www.av在线官网国产| 激情 狠狠 欧美| 精品一品国产午夜福利视频| 中文精品一卡2卡3卡4更新| av在线app专区| 精品久久久久久久久亚洲| 国产中年淑女户外野战色| 亚洲欧美精品自产自拍| 欧美成人一区二区免费高清观看| av天堂中文字幕网| videossex国产| 亚洲欧美日韩东京热| 国产精品一区二区在线不卡| 少妇的逼水好多| 2018国产大陆天天弄谢| 成年女人在线观看亚洲视频| 大片电影免费在线观看免费| 黄色欧美视频在线观看| 久久久国产一区二区| 丝袜喷水一区| 日韩中文字幕视频在线看片 | 一级a做视频免费观看| 99热这里只有是精品50| 精品午夜福利在线看| 乱码一卡2卡4卡精品| 欧美bdsm另类| 纯流量卡能插随身wifi吗| 在线观看免费日韩欧美大片 | 日韩亚洲欧美综合| 午夜福利在线观看免费完整高清在| 午夜激情久久久久久久| 国产久久久一区二区三区| 街头女战士在线观看网站| 欧美成人a在线观看| 女性生殖器流出的白浆| av国产精品久久久久影院| 国产成人一区二区在线| 综合色丁香网| 久久精品国产亚洲av涩爱| 欧美一区二区亚洲| 99国产精品免费福利视频| 大片电影免费在线观看免费| 一级毛片aaaaaa免费看小| 777米奇影视久久| 亚洲国产毛片av蜜桃av| 各种免费的搞黄视频| 在线观看人妻少妇| 亚洲国产欧美在线一区| 联通29元200g的流量卡| 日韩av免费高清视频| 中文字幕久久专区| 国产精品免费大片| 久久毛片免费看一区二区三区| 夫妻午夜视频| 亚洲精品一二三| 国产欧美另类精品又又久久亚洲欧美| 日本欧美视频一区| 亚洲内射少妇av| 一级毛片aaaaaa免费看小| 一边亲一边摸免费视频| 久久精品熟女亚洲av麻豆精品| 精品午夜福利在线看| 成人美女网站在线观看视频| 大片电影免费在线观看免费| av视频免费观看在线观看| 一级毛片电影观看| 国产伦理片在线播放av一区| 国产综合精华液| 亚洲无线观看免费| 一区二区三区四区激情视频| 看十八女毛片水多多多| 一区二区三区乱码不卡18| 99视频精品全部免费 在线| 一区二区三区乱码不卡18| 国产有黄有色有爽视频| 大香蕉97超碰在线| 久久热精品热| 建设人人有责人人尽责人人享有的 | 国产精品三级大全| 18+在线观看网站| 亚洲精品国产成人久久av| 久久久欧美国产精品| 国产精品一区二区在线不卡| 日韩成人伦理影院| 在线观看美女被高潮喷水网站| 99re6热这里在线精品视频| 欧美一区二区亚洲| 大香蕉久久网| 能在线免费看毛片的网站| 一级毛片黄色毛片免费观看视频| 亚洲国产色片| 男人舔奶头视频| 午夜激情福利司机影院| 少妇人妻一区二区三区视频| 欧美成人午夜免费资源| 各种免费的搞黄视频| 日本欧美视频一区| 男女国产视频网站| 一个人看视频在线观看www免费| 视频中文字幕在线观看| 亚洲色图av天堂| 丰满少妇做爰视频| 国产女主播在线喷水免费视频网站| 国产 精品1| 亚洲精品久久久久久婷婷小说| 美女高潮的动态| 亚洲欧美日韩无卡精品| 欧美成人午夜免费资源| 日韩三级伦理在线观看| 久久久精品免费免费高清| a级毛片免费高清观看在线播放| 亚洲精华国产精华液的使用体验| 嫩草影院新地址| 亚洲色图av天堂| 精品久久久精品久久久| 中文天堂在线官网| 国产69精品久久久久777片| 日本午夜av视频| 久久精品国产亚洲av天美| 日韩制服骚丝袜av| 成人综合一区亚洲| 激情 狠狠 欧美| 精品少妇黑人巨大在线播放| 在线观看免费高清a一片| 欧美区成人在线视频| 国产精品久久久久成人av| 中文天堂在线官网| 亚洲av综合色区一区| 成人国产麻豆网| 尤物成人国产欧美一区二区三区| 中文精品一卡2卡3卡4更新| 欧美变态另类bdsm刘玥| 天美传媒精品一区二区| 亚洲成人av在线免费| 在现免费观看毛片| 欧美亚洲 丝袜 人妻 在线| 久热久热在线精品观看| 七月丁香在线播放| 成人午夜精彩视频在线观看| 亚洲精品久久午夜乱码| 色网站视频免费| 亚洲精品日本国产第一区| 亚洲精品日韩在线中文字幕| 中文字幕制服av| 亚洲av男天堂| 国产在线男女| 亚洲,一卡二卡三卡| 人人妻人人澡人人爽人人夜夜| 99久久精品国产国产毛片| 久久久久久九九精品二区国产| 建设人人有责人人尽责人人享有的 | 久久99热这里只频精品6学生| 欧美激情极品国产一区二区三区 | 中文字幕av成人在线电影| 久久久久性生活片| 精品人妻视频免费看| 国产爽快片一区二区三区| 亚洲一级一片aⅴ在线观看| 午夜免费观看性视频| 搡老乐熟女国产| www.av在线官网国产| 亚洲va在线va天堂va国产| 欧美精品一区二区大全| 国产精品伦人一区二区| 色吧在线观看| 成年女人在线观看亚洲视频| 日韩成人av中文字幕在线观看| .国产精品久久| 久久久久网色| 欧美国产精品一级二级三级 | 深夜a级毛片| 亚洲av在线观看美女高潮| 在线观看三级黄色| 国产伦理片在线播放av一区| videossex国产| 国产高潮美女av| 国产中年淑女户外野战色| 最黄视频免费看| 国产高清三级在线| 国产视频首页在线观看| 国产成人精品福利久久| 精品一区二区三区视频在线| 欧美日韩视频精品一区| av在线老鸭窝| 偷拍熟女少妇极品色| 波野结衣二区三区在线| 国产精品久久久久久精品电影小说 | 久久国内精品自在自线图片| 免费人妻精品一区二区三区视频| 欧美高清性xxxxhd video| 亚洲自偷自拍三级| 伦精品一区二区三区| 99久久综合免费| 欧美精品人与动牲交sv欧美| 欧美 日韩 精品 国产| 97超视频在线观看视频| 亚洲国产精品999| 美女福利国产在线 | 国语对白做爰xxxⅹ性视频网站| 日韩av不卡免费在线播放| 纯流量卡能插随身wifi吗| 国产男人的电影天堂91| 伊人久久精品亚洲午夜| 在现免费观看毛片| 国产 一区精品| 久久久a久久爽久久v久久| 国产视频首页在线观看| .国产精品久久| 最近2019中文字幕mv第一页| 精品国产乱码久久久久久小说| 国产一区有黄有色的免费视频| 99九九线精品视频在线观看视频| 九九在线视频观看精品| 三级国产精品片| 久久久久久久久大av| 18+在线观看网站| 精品久久久久久久末码| 亚洲激情五月婷婷啪啪| 久久精品国产a三级三级三级| 一级a做视频免费观看| 国产欧美亚洲国产| 老司机影院成人| 国产精品无大码| 亚洲人成网站高清观看| 色视频在线一区二区三区| 久久人妻熟女aⅴ| 婷婷色av中文字幕| 国产男人的电影天堂91| 成人高潮视频无遮挡免费网站| 女性被躁到高潮视频| 观看av在线不卡| 男女无遮挡免费网站观看| 久久6这里有精品| 交换朋友夫妻互换小说| 成年av动漫网址| 亚洲内射少妇av| 亚洲中文av在线| 国产片特级美女逼逼视频| 黄片wwwwww| 日韩电影二区| 男男h啪啪无遮挡| 人妻夜夜爽99麻豆av| 啦啦啦啦在线视频资源| 亚州av有码| 精品人妻视频免费看| 国产成人a区在线观看| 校园人妻丝袜中文字幕| 亚洲欧美精品自产自拍| 国产精品蜜桃在线观看| 亚洲内射少妇av| 女性生殖器流出的白浆| 亚洲欧美日韩另类电影网站 | 国产精品国产三级国产专区5o| 久久久久久伊人网av| 亚洲精华国产精华液的使用体验| 亚洲精品国产av成人精品| 自拍偷自拍亚洲精品老妇| 乱码一卡2卡4卡精品| 超碰av人人做人人爽久久| 亚洲成人av在线免费| 久久99热6这里只有精品| 中国三级夫妇交换| 国产探花极品一区二区| 国产精品久久久久久精品电影小说 | 一级黄片播放器| 色婷婷av一区二区三区视频| 亚洲无线观看免费| 一本一本综合久久| 99热国产这里只有精品6| 赤兔流量卡办理| 欧美成人一区二区免费高清观看| 在线观看免费日韩欧美大片 | 久久精品国产鲁丝片午夜精品| 国产精品一区二区在线不卡| 免费观看在线日韩| 久久久久久久大尺度免费视频| 国国产精品蜜臀av免费| 人体艺术视频欧美日本| 亚洲精品中文字幕在线视频 | 啦啦啦中文免费视频观看日本| 午夜福利高清视频| 极品少妇高潮喷水抽搐| 综合色丁香网| 久久久久久九九精品二区国产| 亚洲av在线观看美女高潮| 一级二级三级毛片免费看| 亚洲欧美精品自产自拍| 99热这里只有精品一区| 激情 狠狠 欧美| 寂寞人妻少妇视频99o| 免费黄网站久久成人精品| 久久ye,这里只有精品| 成年美女黄网站色视频大全免费 | 麻豆精品久久久久久蜜桃| 五月玫瑰六月丁香| 网址你懂的国产日韩在线| 天堂俺去俺来也www色官网| 啦啦啦在线观看免费高清www| 久久久久久久久久久免费av| 99热这里只有是精品在线观看| 国产成人精品婷婷| av福利片在线观看| 国产精品伦人一区二区| 国产精品欧美亚洲77777| 色网站视频免费| 香蕉精品网在线| 国国产精品蜜臀av免费| 欧美日韩一区二区视频在线观看视频在线| 欧美 日韩 精品 国产| 狂野欧美激情性xxxx在线观看| 亚洲欧美一区二区三区黑人 | 国产高清不卡午夜福利| 免费看日本二区|