Data-Driven Learning Control Algorithms for Unachievable Tracking Problems

Zeyi Zhang , Hao Jiang , Dong Shen ,,, and Samer S.Saab ,,

Abstract—For unachievable tracking problems, where the system output cannot precisely track a given reference, achieving the best possible approximation for the reference trajectory becomes the objective.This study aims to investigate solutions using the Ptype learning control scheme.Initially, we demonstrate the necessity of gradient information for achieving the best approximation.Subsequently, we propose an input-output-driven learning gain design to handle the imprecise gradients of a class of uncertain systems.However, it is discovered that the desired performance may not be attainable when faced with incomplete information.To address this issue, an extended iterative learning control scheme is introduced.In this scheme, the tracking errors are modified through output data sampling, which incorporates lowmemory footprints and offers flexibility in learning gain design.The input sequence is shown to converge towards the desired input, resulting in an output that is closest to the given reference in the least square sense.Numerical simulations are provided to validate the theoretical findings.

I.INTRODUCTION

ITERATIVE learning control (ILC) is a powerful approach for addressing tracking problems in repetitive processes [1],[2].It leverages historical information obtained during repeated iterations to improve performance in subsequent iterations [3], similar to how humans learn through experience.In ILC, an iteration refers to a system operating over a finite time interval, and input and output data from previous iterations are used to compute the input for the current iteration [4], [5].Among various ILC schemes, the P-type learning control algorithm has been widely studied and applied due to its simple structure, where the tracking error is linearly mapped to the input space using a learning gain matrix [6].However,designing the learning gain matrix requires knowledge of the system, which can be challenging when the system model is unknown or uncertain and the available data is incomplete.

Many studies have highlighted the challenges posed by unknown/uncertain system models and incomplete data in the context of ILC [7]–[12].Robust and data-driven control methods have been developed to address these issues.Incomplete data scenarios often involve data dropouts [13], [14], communication delays [15], [16], and varying operation lengths [17],[18].In such scenarios, the unit does not receive all the information, but instead, it obtains input and output fragments at certain time instants and dimensions.The incompleteness is typically modeled using a 0-1 distribution to indicate whether the data is obtained.

However, the P-type learning control algorithm can achieve high-precision tracking performance despite the challenges posed by unknown/uncertain models and incomplete data[19].This is predicated on the assumption that the reference trajectory is realizable [20]–[22], meaning that there exists an input that can produce an output equal to the reference trajectory over the entire operation interval.Achieving perfect tracking, in this case, requires zero initial tracking error at each iteration [23], although precise initialization is often difficult to guarantee in practice.Various techniques have been developed to relax this condition [24], [25].

However, what happens when ILC encounters an unrealizable reference trajectory? An unrealizable reference trajectory refers to a situation where the tracking errors cannot be simultaneously reduced to zero for all time instants, regardless of the input used.This problem frequently arises in underactuated systems, where the dimension of the inputs is less than that of the output [26].In such cases, neither ILC nor other control methods can ensure that the system output matches the desired reference.This is known as the unachievable tracking problem, which has received limited attention in previous ILC studies.

One approach to address the unachievable tracking problem is to minimize the accumulated error index over the entire operation interval.In an earlier work [27], the authors proposed a P-type ILC method to minimize the Euclidean norm of tracking errors, leveraging the system model to achieve the best possible tracking performance.However, the information necessary to achieve optimal performance remains unspecified.A preliminary work [28] addressed the unachievable tracking problem under random data dropouts; however, it required system information, and the learning control scheme was limited to specific inputs.Thus, a more rigorous formulation is needed to tackle the unachievable tracking problem.

For the achievable tracking problem, ILC can ensure zeroerror tracking performance over the entire time interval, and most existing ILC studies have focused on this problem.However, for the unachievable tracking problem, it is inherently impossible to achieve perfect zero-error tracking performance for a given reference over the entire time interval, which is jointly determined by the system model and the desired reference.Hence, it is not merely a technical control issue but rather an inherent property.In this case, the goal is to approximate the desired reference as closely as possible in a certain sense.Motivated by this idea, we use an optimization objective to model the best achievable tracking performance,describe the necessary information to solve the optimization objective, and propose data-driven ILC algorithms for practical applications.In summary, our work aims to address the following issues related to the unachievable tracking problem:

1) Can the widely used P-type learning control scheme solve the optimization objective, and what information is necessary for the design?

2) How can a data-driven P-type learning control scheme be established to handle uncertain/unknown system models?

3) What is the essential effect of incomplete tracking data,and how can this difficulty be overcome?

To tackle these problems, this study makes the following contributions:

1) We characterize the concept of the unachievable tracking problem and demonstrate the necessity of the true gradient information in achieving the control objective using the conventional P-type ILC (Proposition 2 and Claim 1).

2) To handle uncertain/unknown system models, we design a new learning gain matrix using an input and output (I/O)sampling strategy (Algorithm 1).This strategy extracts sufficient gradient information from the available input and output data and achieves the optimization objective in the mean square sense without relying on the system matrix (Theorem 1).

3) Incomplete input and output data can affect the learning process and lead to drift in the gradient information (Proposition 3).To overcome this challenge, we propose an extended ILC scheme that combines the sampling strategy with an error compensation strategy (Algorithm 2).The extended scheme incorporates error correction and achieves convergence of the input error to zero in the mean square sense, thereby achieving the optimization objective (Theorem 2).

Given the inherent difficulties introduced by the unachievable tracking problem, this paper presents two novel datadriven schemes of the P-type ILC that aim to achieve an optimization objective.In comparison to [27], we clarify the necessity of gradient information and highlight the impact of incomplete data on the convergence of the unachievable problem, which does not occur when the reference is realizable.Previous methods such as [9], [10], [29], [30] identify a dynamic system model, whereas our proposed sampling scheme constructs the learning gain matrix directly from samples, eliminating the need for system identification.The compensation method in [11] accelerates robust ILC by compensating for the input, and [12] presents a real-time data-based compensation method to address external disturbances.In contrast, our second scheme aims to compensate for the tracking error of the unrealizable reference.Compared to the earlier attempt in [28], this study defines the unachievable tracking problem, reveals the necessity of gradient information for learning, and proposes novel data-driven learning control algorithms.Furthermore, the algorithm in [28] can be seen as a special case of Algorithm 2 presented in this study.Overall,the existing ILC literature rarely addresses the unachievable tracking problem, and this work represents the first comprehensive investigation of this topic.

Organization: Section II presents the problem formulation.Section III demonstrates the necessity of the gradient.Section IV proposes an ILC scheme with a sampling strategy.Section V elaborates the effect of incomplete data, which then is overcome in Section VI by proposing an extended ILC scheme.In Section VII, numerical simulations verify the theoretical analysis.Section VIII concludes the paper.

Notations: E[·] denotes the mathematical expectation of a random variable.Range(M) and Null(M) denote the column space and null spaces of a matrixM, respectively.S⊥is the complementary orthogonal space ofS.//·// and //·//Fdenote the 2-norm and the Frobenius norm of a matrix, respectively.

II.PROBLEM FORMULATION

Consider the discrete time-varying linear system,

Remark 1: Assumption 1 is necessary to ensure the uniqueness of the input solution based on the minimum performance index.Relaxing the requirement of the full column rank does not affect the essence of the analysis but leads to more complex derivations.Additionally, Assumption 1 does not consider the case of full row rank because the latter implies that any given reference can be perfectly tracked.

Assumption 2: The initial statexk(0) is reset to an unknown invariant valuex0for all iterations.

Remark 2: To emphasize our main contributions, Assumption 2 is used as an initialization condition.This condition is more practical than the commonly used identical initialization condition, where the initial state is required to be reset to the desired statexr(0) satisfyingyr(0)=C0xr(0).The latter condition ensures that the desired reference is achievable, as defined below.Assumption 2 only requires the initial state to be reset to a fixed point for each iteration, which is easy to implement in various applications.

Generally, learning tracking problems are classified into two categories: achievable and unachievable tracking problems.

1) Theachievable tracking problemindicates that the desired referenceyr(t) can be precisely tracked by the system output, i.e., aur(t) (unnecessarily unique) exists such that

where the initial statexr(0) satisfiesyr(0)=C0xr(0).Here, the desired referenceyr(t) is calledrealizable.The tracking problem is widely investigated in the existing literature, where the existence and uniqueness ofur(t) are usually required as an assumption.The convergence of the output sequence to the desired reference can be verified by showing that the input sequence converges tour(t) for all time instants.

2) Theunachievable tracking problemrefers to the case that the desired referenceyr(t) is unrealizable, i.e., no input exists such that (2) is satisfied.In this case, the system cannot achieve zero-error tracking simultaneously for all time instants.Instead, the control objective becomes to approximate the desired reference as closely as possible in a certain sense.

For the unachievable tracking problem, this study considers the tracking performance index,

wherey(t) is output generated by the system (1).Our objective is to iteratively generate an input sequence converging to a limit that minimizes the index J by designing suitable ILC algorithms without knowing the system information.

Remark 3: The tracking performance at the initial time instant, i.e., //yk(0)-yr(0)//2, is not included in the index (3)for the following reasons: First, the output at the initial time instantyk(0) is solely determined by the initial statexk(0) and cannot be improved by any input signal due to the system’s relative degree being one, based on Assumption 1.For systems with a higher relative degree, the performance index could be modified accordingly, but the subsequent derivations remain valid with slight modifications.Therefore,adding this term does not affect finding the minimum index.If the initial state is available, the tracking reference for the remaining time instants can be adjusted accordingly, such that the performance index adequately determines the learning control problem.Several initial state learning mechanisms proposed in [31] can be combined with the learning control.However, investigating these mechanisms is beyond the scope of this study.

We employ the lifting formulation of the system dynamics[1], integrate the time domain dynamics into super-vectors,and highlight the iteration dynamics, thus making the derivations uncomplicated.The super-vectors are defined as follows:

Then, the system matrixGis given by

Consequently, the tracking performance index (3) becomes

Here,YandUare vectors with the same dimensions asYkandUk, respectively.From Assumption 1,Gis of full column rank; therefore, minimizing (5) leads to the unique desired input, denoted by.Theoretically, the desired input is

Replacing the inputUkin (4) with the desired inputUd, we obtain thebest achievable referenceYd, given byYd=GUd+Mx0.Hence, the control objective can be summarized as findingUdthrough iterative learning of the input using input/output data.Moreover, the vectorYd-Mx0∈Range(G)corresponds to the unique vector closest toYr-Mx0in the least square sense.Consequently,Yr-Ydis orthogonal to Range(G), and the minimum performance index(5) is obtained as follows:

For the sake of brevity, we denoteYr=Yr-Mx0and=Yd-Mx0as themodified referenceandmodified bestachievable reference, respectively.

Next, we introduce the conventional P-type ILC algorithm.Given an arbitrary initial inputU0, the update law is defined as follows:

Remark 4: The conventional P-type ILC algorithm, known for its simplicity, has been widely utilized in the literature.The learning gain matrixLplays a crucial role in determining the update direction during the learning process.Furthermore,the block diagonal structure of the matrixLallows for independent implementation of the algorithm (8) at each time instant.

To describe the control objective for the unachievable tracking problem, we formulate an optimization problem.In the subsequent sections, we will elaborate on how the P-type ILC algorithm achieves the control objective by addressing the following aspects:

1) We demonstrate the necessity of precise gradient for the algorithm (8) to solve the problem (3) (see Section III).

2) We propose a data-driven sampling strategy to obtain an alternative gradient and analyze the convergence of the associated ILC algorithm (see Section IV).

3) We delve into the profound impact of incomplete data on the P-type ILC (see Section V).

4) We design an extended ILC scheme to mitigate the effects of incomplete data (see Section VI).

III.NECESSITY OF GRADIENT FOR BEST TRACKING PERFORMANCE

Overall, Claim 1 demonstrates the necessity of the gradient in determiningLto minimize the unlearned part JLand achieve the best approximation.However, the system matrixGis often unknown or uncertain in many applications.Consequently, the associated learning gain matrixLmay be incorrect, defective, or unavailable.To address this problem, the next section presents a sampling strategy.

IV.ILC WITH SAMPLING STRATEGY

This section addresses learning control for the deterministic system (4) without any data incompleteness.We use priorsampled I/O data to acquire system information and resolve the unachievable tracking problem without system information.In particular, these data are used to form the randomized learning gain matrix, avoiding the direct use of the system matrix; thus, this mechanism is data-driven.

Definition 1(Sample): A sample refers to a pair of I/O data{Uk,Yk}generated by the system (4).

Algorithm 1 ILC With a Random Sampling Strategy 1: Determine and division.U0k=0{Xi,Zi}i∈I D={I1,...,Iτ}2: Initialize arbitrary ,.k ≤K 3: while do Is ps ZIs XIs 4: Select with probability , construct and.Uk+1=Uk-αXIsZTIsEk 5:.6:.7: end while Uk k=k+1 8: return

Taking mathematical expectation on this equality, we have

V.GRADIENT DRIFT BY INCOMPLETE DATA

This section shows that algorithms in the form (8) cannot achieve the best tracking performance in incomplete data scenarios even if a precise system matrixGis employed.

To this end, we first present a general model of the incomplete data environment.In this study, we use ΓkEkto represent the available error information for the input updating.Data incompleteness is modeled by a random matrix Γk∈RqN×qNmultiplying the corresponding data vector, where each entry of Γkis subject to 0-1 distribution: the entry being equal to 1 if the associated information is available and 0 if the associated information is lost.Therefore, Γkis not directly available to the controller but is indicated by the obtained data.For further analysis, we assume that Γkis independent of the iterationkand Γ ?E[Γk] is nonsingular.The following incomplete data scenarios are covered by this model.

1)Random Data Dropouts: The outputs are randomly lost during the information exchange through an unreliable communication channel.We compensate for the output with the desired reference signal if the output is lost.Then, a variable γk(t)is employed to denote the transmission effect on the output at time instantt, where γk(t)=1 and γk(t)=0 correspond to successful transmission and data dropout, respectively.Finally, Γk=diag{γk(1),...,γk(N)}?Iqis defined to denote the virtual error for the learning control algorithm by ΓkEk.

2)Randomly Varying Lengths: The operation process for each iteration randomly ends before the desired lengthN.The iteration length is modeled byNk, randomly varying in the candidate set {N,N+1,...,N}.In this scenario, we also use γk(t)to denote the occurrence of the output at time instantt.Differing from the previous scenario, the random variables γk(t) are not mutually dependent, resulting in Γk=diag{INk,0N-Nk}?Iq.

Remark 10: Bernoulli random variables are most popular in modeling various randomness such as data dropouts, varying trial lengths, and communication delays [33].These are covered in our setting where Γkis independent regarding the iteration number.More complex settings such as the Markov chain and random sequence model can be considered; however,these models are omitted in this study to highlight the main novelty.

We provide a proposition to show the convergence of the Ptype ILC algorithm (8) employing the precise gradientL=GTunder incomplete data.In Section III,L=GTguarantees that limk→∞Uk=Udif the received data are complete.However,this conclusion may fail to hold if the received tracking error information is incomplete.

Proposition 3: Consider the system (1).Let Assumptions 1 and 2 hold,L=GT, andαbe smaller than the inverse of the maximum eigenvalue ofGTG.Applying the P-type learning control algorithm (8) using the incomplete data ΓkEk, the update process becomes

Then,Ukconverges to the drifted inputUdriftin the expectation sense, whereUdriftis specified in the proof.

Proof: Taking expectation on both sides of (11), the following expression is obtained:

Next, we provide an example to show that the best tracking performance (7) cannot be achieved by the P-type learning control algorithm (8) without a precise gradient.

Example 1: First, we consider the system (4) and algorithm(8) withx0=0, whereGandLare given by

In the learning control algorithm (8), making learning gain matrixLprovide a gradient requires precise system matrices and statistical information of incomplete data.This is very difficult in real world applications.Thus, in the following section, we offer a new idea to avoid providing gradients ofL.

VI.EXTENDED ILC SCHEME

While considering the unachievable tracking problem under incomplete data environments, we observe a random gradient drift.As there is no input ensuring the simultaneous perfect tracking at all time instants, it is required that the control direction (reflected by the learning gain matrix) must be the gradient direction, either implicitly or explicitly, to achieve the best tracking performance.Note that this requirement is unnecessary for the achievable tracking problem (see Remarks 5 and 11).Therefore, we aim to resolve the unachievable tracking problem under incomplete data environments from a novel perspective significantly differing from the sampling strategy in Section IV.In particular, we refine the tracking error by adding a correction term defined by the sampling data such that the modified reference signals for updating are asymptotically achievable as the iteration number increases.Then, any conventional P-type learning control scheme can be used to solve the unachievable tracking problem using the refined tracking errors.The integrated framework is called the extended ILC scheme.

Algorithm 2 Extended ILC Algorithm 1: Determine and division.U0z0=Yr-Y?1k=0 2: Initialize arbitrary , ,.{Xi,Zi}i∈I D={I1,...,Iτ}3: while do Is ps ZIs k ≤K 4: Select with probability , construct.zk+1=zk-ZIs ZTIs 5:.Uk+1=Uk-αL(ΓkEk+Γkzk+1)//ZIs //2Fzk 6:.k=k+1 7:.8: end while Uk 9: return

The extended computationzkis to actively learn the unachievable partYr-Yd.In this way, the modified referenceYr-zkbecomes realizable asymptotically (see Lemma 3), and we use the modified reference instead of the original reference for input updating.The following lemma shows the convergence ofzk, whose proof is put in the Appendix.

Lemma 3: The sequenceYr-Yd-zkconverges monotonically to zero in the mean square sense.

Consequently, we have the following convergence result for Algorithm 2, whose proof is put in the Appendix.

Theorem 2: Consider the system (1).Let Assumptions 1 and 2 hold,αbe sufficiently small, and all eigenvalues ofLΓGbe positive real numbers.Then, for any unachievable referenceYr, the input sequenceUkgenerated by Algorithm 2 converges to the desiredUddefined by (6) in the mean square sense.

The proof of this corollary is put in the Appendix.

We emphasize that the central idea of Algorithm 2 is to introduce an active reference refinement mechanism such that the modified reference is asymptotically achievable.Then,any learning control method for the achievable tracking problem can be integrated into the proposed scheme to resolve the unachievable tracking problem.The primary advantage of this scheme is to relax the requirement for precise gradient information, which might be unavailable due to various conditions.Particularly, the sampling mechanism given in Algorithm 1 can be embedded into the input update step of Algorithm 2.In this case, the entire framework is completely data-driven.The convergence is summarized in the following corollary, whose proof is omitted for brevity.

Corollary 2: In Step 6 of Algorithm 2, if the learning gain matrixLis replaced with, the convergence results of Theorem 2 still hold.

Table I summarizes the information required for the algorithms.Here, PILC, SILC, and EILC represent the P-type ILC,ILC with sampling strategy, and extended ILC, respectively.

TABLE I CONVERGENCE REQUIREMENTS OF THE ALGORITHMS

VII.ILLUSTRATIVE SIMULATIONS

In this section, we validate the theoretical results through numerical simulations conducted on a toy example and a chemical batch reactor system.We evaluate the performance of gradient-based ILC (GILC), PILC with block diagonal gain, SILC, and EILC under both complete and incomplete data environments.GILC, which utilizes the precise gradient,serves as the benchmark for comparison.The relationships between the simulation outcomes and theoretical results are summarized in Table II.

TABLE II CORRESPONDENCE BETWEEN FIGURES AND THEORETICAL RESULTS

In practical implementations, ILC is often deployed within a networked control structure, where information and data are transmitted through communication networks.However, network congestion and limited bandwidth can lead to data packet loss during transmission [13].Therefore, data dropout is a typical scenario in incomplete data environments, which is adopted in this section to illustrate the challenges posed by incomplete data.

A. Numerical Case

We consider the system represented by (At,Bt,Ct),

Let the reference trajectory be

The following settings are considered: the time interval is set to [0,10]; the time instants calculated in (3) are 1 ,2,...,10;and the initial input and state are zero vectors.

To assess the performance of each scheme, all results are normalized by the error at the first iteration, ensuring that all lines in the corresponding figures start from 10.

Fig.1 shows the decrease in the input error //δk//2for the four schemes: GILC, PILC, SILC, and EILC.The y-axis scale is logarithmic.Based on the 30th iteration as the dividing point, we analyze the performance of the different schemes.For the first stage (k<30), GILC, which utilizes full system information, achieves the fastest convergence and is considered the benchmark (solid line).PILC (dotted line) and EILC(dashed line) utilize only the control and measurement matrix knowledge, making them slower than GILC.SILC (dasheddotted line) is the slowest among the four schemes, as its search space is at most ten-dimensional according to Proposition 1, whereas the other schemes have a 20-dimensional search space.Additionally, EILC exhibits slower convergence than PILC because it uses a revised referenceYr-zk,which can be far from the best achievable reference in the early stage.For the second stage, when the learning gaindoes not provide the precise gradient, PILC does not converge to the desired input, even in the case of complete data.This behavior is explained by Proposition 2 and Claim 1.It can be observed that GILC, EILC, and SILC show zero convergence tendencies, demonstrating their effectiveness, with SILC being particularly guaranteed by Theorem 1.

Fig.1.Input error profiles for GILC, PILC, SILC, and EILC, where GILC is labeled as a benchmark.

Furthermore, Fig.2 depicts the decrease in the tracking error, providing further support for the observations described above.In the subplot, we observe that GILC, EILC, and SILC achieve the best approximation performance, while PILC significantly deviates from the other three methods in terms of approximation performance.

2)Incomplete Data Environments: We discard PILC as it is invalid even under complete data and maintain the benchmark performance under complete data for comparison.Following [13], we model the random data dropout using γk(t).Here, γk(t) is generated according to the Bernoulli distribution with E[γk(t)]=γ(t), where γ(t) is predetermined and unknown to any algorithm.The affected tracking error γk(t)(yk(t)-yr(t))is used in the schemes for updating.Other settings remain the same as in the previous subsection.

Fig.2.Tracking error profiles for GILC, PILC, SILC, and EILC.

Fig.3 depicts the input error decrease in GILC (dotted line),SILC, and EILC using incomplete data.As indicated by our analysis in Proposition 3, GILC and SILC lose their efficacy due to gradient drift.In particular, GILC and SILC cannot handle random missing data and consequently exhibit oscillatory behavior without achieving convergence.Only EILC demonstrates a continuous decreasing trend, confirming Theorem 2.As shown in Fig.4, while EILC reaches the benchmark performance, GILC and SILC fail to converge to the benchmark and exhibit oscillatory behavior.This highlights the effectiveness of the EILC scheme.

Fig.3.Input error profiles for GILC, SILC, and EILC.

B. Chemical Batch Reactor Simulation

ILC research has considered a nonlinear chemical batch reactor, which exhibits a second-order exothermic reactionA→B[30], [36].In this reactor, the temperature of the cooling jacket is directly manipulated, while the objective is to track a reference temperature trajectory.The dynamics of the reactor can be described by the following continuous equations:

Fig.4.Tracking error profiles for GILC, SILC, and EILC.

whereT,CA, andTjrepresent the reaction temperature, concentration of reactantA, and temperature of the coolant stream, respectively.The parameters used in the model are as follows [36]:

We define the state variables asxk(t)=[T,CA]Tand the input asuk(t)=Tj.Additionally, we introduce the following notations:

With these definitions, we can rewrite (15) as a state-space model,

where the initial state is for all iterations.It is worth noting that the system (16) has two outputs and one input.There may exist reference trajectories that are not achievable, making the ILC tracking problem infeasible.Let the reference trajectory be defined as follows:

For the system described by (15), a linear model-based design approach can be employed [36].Although the algorithms in this study are designed for linear systems, it is important to acknowledge the existence of nonlinear systems such as (15).Consequently, we discretize the system (16)using inputsand obtain the corresponding outputswhere the time interval is from 0 to 80 min with a sampling time of 1 min.We then preprocess this data using Algorithm 1:Subsequently,we construct the learning gainLfor P-type ILC by employing the least squares principle,

The sampling data required for Algorithms 1 and 2 are constructed from the available data, and additional operations are not necessary for subsequent iterations.Each iteration involves a random selection of 10 samples.Notably, we do not present an input error analysis due to the inability to analytically solve for the desired input trajectory in the presence of system nonlinearity.Therefore, our performance analysis focuses on tracking errors.

1)Complete Data Environment: In the complete data environment, PILC, SILC, and EILC are applied to the reactor tracking problem using the same learning gain matrixL.The step size is set to α=1, and the initial input isu0(t)=26 fort=0,...,80.The actual tracking error profiles of PILC, SILC,and EILC are shown in Fig.5.

Fig.5.Actual tracking error profiles for PILC, SILC, and EILC.

Fig.5 demonstrates that the actual tracking error decreases as the iterations progress, indicating that all three methods improve their performance over time.They-axis values in the figure are normalized by the first iteration error.SILC(dashed-dotted line) shows slower convergence compared to PILC (solid line) and EILC (dashed line).PILC and EILC exhibit similar performance levels since they use the same learning gain matrix.However, from the subplot, it can be observed that EILC yields a more steady convergence.

2)Incomplete Data Environment: Incomplete data environments are simulated by introducing data dropouts, where the tracking error at each time instant is multiplied by a Bernoulli random variable.PILC, SILC, and EILC are tested on the nonlinear system (16) using the same learning gain matrixLfor PILC and EILC.Due to the presence of data dropouts,which can reduce the convergence rate, we perform 150 iterations to observe the convergence trends.

Fig.6 shows the actual tracking errors of PILC, SILC, and EILC under data dropouts.It can be observed that the performance comparison of the algorithms is similar to that of the complete data case.However, the random dropouts induce larger oscillations, making it difficult to conclude that EILC performs significantly better than PILC.This indicates the need for improved application methods for nonlinear systems,as the sampling data of nonlinear systems may not fully reflect the global system properties.

Fig.6.Actual tracking errors profiles of PILC, SILC, and EILC.

In Fig.7, the 1st, 20th, 40th, 60th, and 80th iterations of the reaction temperature generated by EILC under incomplete data are presented.Since temperature control is the main objective, only the reaction temperature is plotted (dotted red line).The figure shows that the reference trajectory (solid black line) is gradually tracked.Due to the fixed initial state of 25 instead of the reference value 22.2, the reaction temperature in the first few minutes shows a tendency to approach the reference but does not perfectly reach it.Overall, the output achieves satisfactory tracking performance within 20 iterations.This indicates that EILC can provide benefits for a class of nonlinear systems.

Fig.7.Reaction temperature at the 1st, 20th, 40th, 60th, and 80th iterations for EILC under incomplete data.

VIII.CONCLUSIONS

This study has successfully formulated and characterized the unachievable tracking problem, which arises when the reference cannot be precisely achieved at all time instants.In response to this problem, we have provided and analyzed different ILC solutions based on specific requirements.Notably,we extensively clarified the necessity of precise gradients in the conventional P-type ILC.Moreover, when the system information is unknown, preventing the establishment of gradients, we identified the capabilities of the P-type ILC for linear time-varying systems and devised a data-driven approach.Additionally, to address systems operating under incomplete data environments, we proposed an extended ILC scheme to mitigate the gradient drift effect.Through investigations, we demonstrated that the proposed learning control algorithms can achieve the best approximation performance of the unrealizable reference in a mean square sense.For future research, it is imperative to explore the integration of offline and online data to enhance these methods against disturbances and noises.

APPENDIX

Taking the Euclidean norm and conditional expectation gives

Based on the above, we claim that

On the other hand, the largest eigenvalue of the left-hand side of (19) is not greater than one because this term is equal to

in which the sum of the eigenvalues of a symmetric matrix is equal to its trace.

Subsequently, taking conditional expectation gives

wherec5is a positive constant.Retaking expectation gives

As a result,we fix appropriateαand ? in place, such that(1+?)c0=1-ρ1<1.Then, we can apply Lemma 4 on

Note that

is convergent because of ρ2<1 by Lemma 3.Thus,→0by Lemma 4 holds.

Proof of Corollary 1: First, we redefinewhich is still convergent to zero in the mean square sense.Next, the expression below is produced using the Cauchy-Schwartz and Young’s inequalities:

and then,

We observed that the two terms on the right-hand side of the above inequality tend to zero becauseis bounded.Then, the rest of the proof can be completed similar to Theorem 2.