Neural-Network-Based Nonlinear Model Predictive Tracking Controlof a Pneumatic Muscle Actuator-Driven Exoskeleton

Yu Cao, and Jian Huang,

Abstract—Pneumatic muscle actuators(PMAs)are compliant and suitable for robotic devices that have been shown to be effective in assisting patients with neurologic injuries,such as strokes,spinal cord injuries,etc., to accomplish rehabilitation tasks. However, because PMAs have nonlinearities,hysteresis,and uncertainties,etc.,complex mechanisms are rarely involved in the study of PMA-driven robotic systems. In this paper, weuse nonlinear model predictive control(NMPC)and an extension of the echo state network called an echo state Gaussian process(ESGP) to design a tracking controller for a PMA-driven lower limb exoskeleton. The dynam ics of the system include the PMA actuation and mechanism of the leg orthoses; thus, the system is represented by two nonlinear uncertain subsystems.To facilitate the design of the controller, joint angles of leg orthoses are forecasted based on the universal approximation ability of the ESGP. A gradient descent algorithm is employed to solve the optim ization problem and generate the control signal.The stability of the closed-loop system isguaranteed when the ESGP is capable of approximating system dynam ics.Simulations and experiments are conducted to verify the approximation ability of the ESGP and achieve gait pattern training w ith four healthy subjects.

I.In t roduction

THERE is an increasing interest in using various kinds of exoskeleton devices to deal w ith practical problems,such as rehabilitation exoskeletons[1],[2],cooperative exoskeletons[3],[4],and powered exoskeletons[5],[6],etc.Such robots directly interact w ith the human body so that the system requiresnecessary compliance to ensure human safety.Pneumatic muscle actuators(PMAs)are compliant and very promising in the field of robot actuation,ow ing to their advantages including a high-force-to-weight ratio,no mechanical parts,and low cost[7].These make PMAs particularly suitable for the exoskeleton class of robotic devices.As early as 2006,Costaet al.[8]designed a 10-DOF PMA-driven exoskeleton for human walking,in which each DOFwas actuated by an antagonistic configuration of PMAs.By 2010,Choiet al.[9]established the dynamics of an antagonistic configuration of PMAs and proposed sliding mode control for position tracking of joint angles. Hussainet al.[10]developed a gait training paradigm by using a PMAdriven lower limb exoskeleton and provided interactive robotic gait training w ith adaptive impedance control in 2013.At present,most PMA-driven exoskeletons are dependent on antagonistic configurations, because a single PMA can only provide a unidirectional force[11],[12]. A lthough there exist different lower limb PMA-driven exoskeletons,complex mechanismsare rarely involved.The main reason is the PMAs nonlinearities,hysteresis,time-varying parameters and unmodeled uncertainties which cause difficulties in position and force control.Most researches focus on position control,and various model-based and model-free strategies are proposed,including sliding mode control[13], nonlinear model predictive control [14],active model-based control [15],proportional-integral-derivative(PID) based control[16],etc.However,model-based strategies have to deal w ith system uncertainties whilemodel-free strategies,like traditional PID and Mamdani fuzzy control,lack theoretical foundation for ensuring system stability when no theoretical model is involved.

Model predictive control(MPC) has been w idely recognized from academ ia and industry due to its conceptual simplicity and ability to effectively handle complex systems w ith hard control constraints and multiple inputs and outputs[17]–[19].This strategy turns a control problem into a quadratic programm ing problem which can be solved by various methods,such as gradient-based optim ization[20],neural networks[21],[22],etc.Hence,it has been an important approach to control complex mechano-electronic systems.However,the MPC requires an accuratemodel that might be hard to obtain for nonlinear systems.Fortunately,different kinds of approximators,such as neural networks and fuzzy approximation, provide an alternate way to estimate unmodeled dynam ics,due to their universal approximation ability[23]–[26].Therefore,neural-networks-based nonlinear model predictive control(NN-NMPC)is gradually known.W ith this approach,only the system’s input and output data are required to realize the neural network approximation.Chenget al.[27]applied nonlinear predictive controlw ith two cascaded feedforward neural networks to approximate the dynam ics of Piezoelectric Actuators.Panetal.[28]presented model predictive control of unknown nonlinear dynamical systems based on two recurrent neural networks. Note that different topologies of neural networks result in varying modeling accuracy which may bring about serious problems to NMPC.Traditional feedforward neural networks,like multi-layer perceptrons(M LP), provide a staticmap between the inputs and outputs thatmay not be suitable for nonlinear dynamic systems. Recursive neural networks (RNNs) [29], [30]m ight be more suitable for dynamic learning because they recursively input information in the order where they are connected.The echo state network(ESN),a special RNN,is famous for reservoir computing which is in the form of sparse connections of hidden neurons[31].In 2011,Chatziset al.[32]fused the ESN and Gaussian process(GP)into the echo state Gaussian process(ESGP)which inherits the ESN’s reservoir computing and combines the merits of GP.As a result,it can solve the ESN’s ill-posed problem and strengthen generalization capabilities[33]. Although the ESGP had been proposed for years,few application studies were presented in robotic systems.

In this paper,we propose a neural-network-based nonlinear model predictive control strategy for a PMA-driven lower limb exoskeleton system.The system is comprised of a treadm ill, a bodyweight support system, and a few mechanisms.Among them,the mechanisms contain two-leg orthoses,each of which contains two rotary joints,and each joint isdriven by an antagonistic configuration of PMAs.Meanwhile,the PMAs are independently installed in an external frame.Actuation torques are not generated on the joints, thus force transm issions via several connecting bars are required.The practical problem mainly contains the follow ing two aspects.One is the identification problem of two cascaded nonlinear subsystems,including the PMA actuation and multi-linkagemechanisms.The other is slow response characteristics of PMA actuation.Considering the factors above,we focus on the precise tracking control of the exoskeleton.The proposed approach turns out to be a data-driven strategy so that only the sensory data of the system is necessary.The main contribution of this paper lies in:1)the proposed controller for the PMA-driven exoskeleton;2)the stability analysis of the closed-loop system;and 3) the experimental studies for validating the control strategy.

The rest of this paper is organized as follows.Section II describes the dynamics of the exoskeleton system,and the controller designed is presented in Section III together w ith the technique for addressing the modeling and optim izing problems.Section IV presents the simulations that verify the approximation ability of the ESGP.Real-world experiments are conducted to demonstrate the effectiveness of the proposed method in Section V.Finally, Section VI summarizes the conclusion of this paper.To enhance readability,the proof of system stability is presented in Appendix.

II.Problem Formu lat ion

This paper focuses on tracking control of the leg orthoses driven by antagonistic configurations of PMAs.Two groups of antagonistic configurationsof PMAs provide torquesand,which drive the hip and knee joints through connecting rods.Therefore,we describe the system from two aspects of actuation and mechanism.

A. Actuation

One actuation unit drives one rotary joint.This is an antagonistic configuration of PMAs that is comprised of two PMAs and a pulley.One PMA is inflated while the other one is deflated,and the PMAs generate forces of different magnitude in the same direction thereby the torque is produced.Each unit drives a joint so that two-leg orthoses require four groupsof antagonistic configurationsof PMAs.

The basicmodelof PMAs is called the three-element model[34],asshown:

B. Mechanism

Fig.1.Theantagonistic configuration of PMAs.

Fig.2.Themechanism of lower limb exoskeleton.

III.Con t rol ler Design

A. Echo State Gaussian Process

B. Nonlinear Model Predictive Control

The idea of this controller mainly contains two aspects.One is the approximation process.The other one is to solve an optimization problem.The control scheme is shown in Fig.3.In the former aspect, the ESGP approximates the dynam icsof the PMA-driven exoskeleton and forecasts future behaviors.Then,together w ith the desired trajectories,the predicted tracking errors are fed into the model predictive controller which solves theoptimization problem.

C.Optimization Method

For convenience, we first define follow ing equations:

Then,(29)and (30)can be rew ritten as

IV.Simu la tion Studies

In this section,the approximation ability of the ESGP is verified w ith a simplified model of the exoskeleton(10)and(11)which isa 2-DOF roboticmanipulator [38]

The parameters,including antagonistic configurations of PMAsand the mechanisms,areshown in Table I.

TABLE I M odel Pa rameter s

Another two neural networks, the ESN,and the MLP,are used to compare the results of ESGP.To make a fair comparison,these three approximators have the same topology,including an input layer,a hidden layer,and an output layer.The parameter training methods of these approximators can be referred to[39].Also,the number of neurons in the hidden layers of three neural networks is also the same, which is set as 25.The prediction horizon and the control horizon areand.

The results are shown in Figs.4–7.It is seen that although the input frequency varies w ith time,all the networks can approximate the dynamics of the exoskeleton and predict output angles of the hip and knee joints.The three networks show the same characteristics that the prediction errors become larger as the input frequency increases.Compared w ith the ESN and MLP,the ESGP behaves w ith the highest approximation accuracy.Actually,in most cases,the ESGP and ESN have sim ilar approximation capabilities, but the ESGP extends ESN to combine GP and handles Gaussian noise.Meanwhile,thanks to the reservoir computing,the sparely connected hidden neurons largely reduce computational complexity and the ESGPonly needs to train a readout.Hence,its calculation efficiency ishigher than that of MLP.

Based on the above analysis, we regard the ESGP as a suitable approximator to model the dynamics of the physical PMA-driven lower limb exoskeleton system and forecast the future behaviors.

Fig.4.Prediction of thehip joint anglew ith varying input frequency.

Fig.5.Prediction error of thehip joint anglew ith varying input frequency.

Fig.6.Prediction of the knee joint anglew ith varying input frequency.

Fig.7.Prediction error of the knee joint angle w ith varying input frequency.

V.Exper imenta l Studies

The physical platform ismainly comprised of a treadm ill,a body weight support system and leg orthoses,as shown in Fig.8.Among them,the sensing and control units,including angle encoders,electromagnetic valves,force sensors,signal amplifiers,are concentrated on the leg orthoses to accomplish the favorable control performance.The devices of the platform are shown in Table II.

Fig.8.The physical platform.

TABLE II Devices of the Exoskeleton System

TABLE III In formation of Subjects

One subject(age:24,height:176 cm, weight:68 kg)participates in the first experiment to demonstrate the effectiveness of the proposed control strategy.The subject is asked to wear the body weight suit which helps the participator keep balance,and relax the leg.This allows the mechanisms to guide the lower limb movement.The experimental results are presented in Figs.9–12.Overall,the tracking performance of the knee joint isbetter than that of the hip joint.This is because the hip joint burdens the weight of the leg orthosis,causing itmore difficult to control than the knee joint in the face of the same control effect. Another significant factor is the slow response of the PMAs.When facing a relatively large load,theantagonistic configuration of PMAs requires large average pressure to provide sufficient rigidity.Remarkably,there existmultiple optimal solutions in the NMPC,and typical numerical solvers are unable to distinguish a local optimum from a global optimum.Thus,only a feasible solution is necessary. According to the results shown in Table IV,the MAE and IAE of the proposed strategy are obviously smaller than the PID controller,which indicates the effectiveness of the method.When we increase the gain of the PID controller to ensure tracking performance,a serious vibration occurs that causes larger tracking errors.Unlike the PID controller,the proposed control strategy uses the ESGP to capture the dynam ics of the system, which helps to achievebetter control performance.

Fig.9.Tracking performanceof thehip joint.

Fig.10.Tracking error of thehip joint.

Fig.11.Tracking performance of the knee joint.

Fig.12.Tracking error of the knee joint.

TABLE IV Compar ison of the Proposed Method and PID Cont rol ler

In the second experiment,the other three subjects are instructed to follow the process in the same manner as the previous experiment.The results are shown in Figs.13–16.It is clear that even though the control parameters have not changed,the tracking performances of different subjects are similar.The main reason for this is that this experiment is mainly used for passive gait training,and human initiative is not involved.In this situation,human legs can be regarded as an external load.Before conducting the experiment w ith subjects, we first roughly calibrate the weight of human legs by the body segment inertial parameters(BSIPs)[40].And then,a corresponding weight of loads is added to the PMAdriven exoskeleton for testing the controller.When thiswork is done, participators wear the exoskeleton and further verify the effectiveness of the controller.Actually,it is impossible to calibrate the weight accurately, thus this requires the controller to be robust to weights.According to experimental results,this control strategy can meet the requirement of passivegait training for different subjects.

VI.Conc lusion

Fig.13.Hip joint tracking performanceof different subjects.

Fig.14.Hip joint tracking error of different subjects.

Fig.15.Knee joint tracking performance of different subjects.

Fig.16.Knee joint tracking error of different subjects.

In this paper,we propose a neural-network-based nonlinear model predictive controller to achieve tracking control of a PMA-driven lower limb exoskeleton for passive gait training,in which the ESGP is used to approximate the dynam icsof the robot system while the nonlinearmodel predictive controller solves an optim ization problem by using the gradient descent algorithm.Ow ing to the online learning of the ESGP and the optimization,the strategy turnsout to be a data-drivenmethod where only the sensory data is required. According to the Lyapunov theorem,the stability of the closed-loop system is guaranteed.Simulations and experiments are conducted to verify the effectiveness of the proposed method and achieve gait pattern training w ith four healthy subjects.

At present,the problem of this PMA-driven exoskeleton mainly includes the follow ing two aspects.One is the slow response characteristics of PMA actuation.The other is the problem of human-computer interaction.The main reason for the former is that the PMA actuation depends on the internal air pressure, but the rate of pressure change is intrinsically small.One solution is to replace the proportional pressure relief valve w ith the proportional directional valve which can control the flow rate of gas inside the PMA to speed up the response.However, the modeling of this valve is difficult.Various factors,including aerodynam ics and fluid dynam ics,etc.,need to be considered.Therefore,one of our future tasks is to model the proportional directional valve. Another significant problem concerns human-robot interaction.In our exoskeleton system,we use soft airbags as interactive sensors.The human-machine interaction force squeezes the airbags to deform them,thereby raising their internal air pressure.Hence,such sensors can measure the interaction force after suitable calibration. Asmentioned earlier,the rate of pressure change is small.This may result in the measurement of interaction forces not being fast or accurate.Therefore,we would like to determ ine how to improve the performance of the sensors and realize impedance control to expand the application of exoskeleton.

APPENDIX Proof of the Theorem 1