On RNN-Based k-WTA Models With Time-Dependent Inputs

Mei Liu and Mingsheng Shang

Dear editor,

This letter identifies two weaknesses of state-of-the-artk-winnerstake-all (k-WTA) models based on recurrent neural networks (RNNs)when considering time-dependent inputs, i.e., the lagging error and the infeasibility in finite-time convergence based on the Lipschitz continuity. Specifically, in the case of time-dependent inputs, theoretical analyses and simulations are conducted to illustrate that the lagging error is inevitable for the dual network model based on RNN.Then, a newk-WTA model aided with RNN is constructed in this letter with the ability of eliminating the lagging error. Theoretical analyses demonstrate that the finite-time convergence of the existingk-WTA models based on the Lipschitz continuity with time-dependent inputs cannot be achieved. Besides, this letter offers a feasible solution to performk-WTA operations with desired convergent speed efficiently and precisely.

Introduction: Deemed as a competitive neural network, winnertakes-all (WTA) networks gain widespread applications in various fields. As a generalized form,k-WTA is widely applied to the modelling of a competitive frame withkdenoting the number of winners[1]. For example, thek-WTA network is leveraged to explore the evolution law of individuals in social systems [2]. A dynamic thresholdingk-WTA model owing relatively simple structure and faster convergent speed is presented in [3]. Moreover, an approach to complete task object tracking of multiple robots using thek-WTA strategy is designed in [4].

Recurrent neural RNN algorithms achieve great success for optimal online solutions over the recent years. For example, an RNN model with the aid of the saddle-point theorem is utilized to dispose of the non-convex optimization problem, which is applied to the identification problem of genetic regulatory networks [5]. A deep RNN model is constructed to predict the residual life of the roller by exploiting a comprehensive health indicator [6]. A gesture prediction model based on RNN is presented and verified on social robots [7].In a nutshell, RNN-based models demonstrate the high advantages in processing optimization problems.

In view of the feasibility that thek-WTA operation can be transformed into a constrained optimization problem, many RNN algorithms are investigated for online solutions ofk-WTA. A series ofk-WTA models aided with the dual neural network is studied to deal with thek-WTA operations [8], [9]. The above RNN-basedk-WTA schemes demonstrate their strengths with different investigation focuses. However, the lagging error is inevitable in handlingk-WTA operations with time-dependent inputs for the existing RNN-basedk-WTA models, which leads to an unsatisfactory performance. Qiet al.[10] introduce a robustk-WTA method with time-dependent inputs,which overcomes the time-lagging error for disposingk-WTA problems, while it has higher computational complexity than most of existingk-WTA models. This letter constructs a simplified RNNbasedk-WTA model which is capable of eliminating the lagging error. Furthermore, theoretical analyses and simulative results are provided to verify the validity of the constructed RNN-basedk-WTA model. It is worth noting that, the existing models, including the model constructed in this letter, are not feasible in terms of finitetime convergence based on the Lipschitz continuity with time-dependent inputs.

For the above two activation functions, it is derived similarly that they do not satisfy the requirement of the Lipschitz continuity. Thus,implementing the models equipped with these types of activation functions with finite-time convergence is extremely difficult or even impossible in practice. ■

Feasible replacement to the finite-time convergence: The absolute zero error, i.e., a completely exact solution is non-existent on account of the environmental disturbance and hardware errors, and the most desirable result is to obtain an optimal solution with the expected precision in the preset time.

Experiments: A group of four sine signals as inputs of thek-WTA system are employed to demonstrate the results of the dual-networkbasedk-WTA model (3) and the RNN-basedk-WTA model (7) with different γ. In simulations, the sinusoidal functions are set asx(t)=-sin(2π(t+0.8(i-1)))(i=1,2,3,4); δ=0.001 and the selected winnersk=2. Evidently, in Figs. 1(a) and 1(b), the dual-network-basedk-WTA model makes a wrong selection for the winners.Compared with Fig. 1(a), when γ=1, results in Fig. 2(a) are consistent with the demonstration of Theorem 3 that the RNN-basedk-WTA model is exponentially convergent. According to Figs. 1(c) and 1(d), the dual-network-basedk-WTA model (3) shows the accurate selection of time-dependent inputs by increasing the coefficient γ.However, there are noticeable prickles and vibrations that can generate unacceptable errors and even select wrong winners. Significantly,as demonstrated in Fig. 2, with different γ, the RNN-basedk-WTA model (7) is capable of accurately and effectively selecting the winners in the time-dependentk-WTA system.

Fig. 1. Inputs and outputs of dual-network-based k-WTA model (3) with different γ . (a) γ =1; (b) γ =10; (c) γ =100; (d) γ =1000.

Fig. 2. Inputs and outputs of RNN-based k-WTA model (7) with different γ.(a) γ =1; (b) γ =10; (c) γ =100; (d) γ =1000.

Conclusions: This letter has analyzed the limitations of the existing RNN-based models for executing thek-WTA operations with time-dependent inputs considered. Then, theoretical analyses and simulative results have provided that the dual-network-basedk-WTA model (3) is inefficient in eliminating the lagging error for dealing with thek-WTA operations with time-dependent inputs. In addition,the infeasibility of the RNN-basedk-WTA models on the finite-time convergence based on the Lipschitz continuity has been verified through theoretical analyses, and feasible alternatives are provided.This letter has raised a new RNN-basedk-WTA model (7) that makes up the deficiency of the dual-network-basedk-WTA model. Theoretical analyses have shown that the feasibility of the new RNN-basedk-WTA model when handling thek-WTA operations considering the time-dependent inputs.

Acknowledgments: This work was supported by the National Natural Science Foundation of China (62072429) and the Key Cooperation Project of Chongqing Municipal Education Commission(HZ2021017, HZ2021008).