Decision-Making in Driver-Automation Shared Control: A Review and Perspectives

Wenshuo Wang, Member, IEEE, Xiaoxiang Na, Dongpu Cao, Jianwei Gong, Member, IEEE,Junqiang Xi, Yang Xing, and Fei-Yue Wang, Fellow, IEEE

Abstract—Shared control schemes allow a human driver to work with an automated driving agent in driver-vehicle systems while retaining the driver’s abilities to control. The human driver,as an essential agent in the driver-vehicle shared control systems,should be precisely modeled regarding their cognitive processes,control strategies, and decision-making processes. The interactive strategy design between drivers and automated driving agents brings an excellent challenge for human-centric driver assistance systems due to the inherent characteristics of humans. Many open-ended questions arise, such as what proper role of human drivers should act in a shared control scheme? How to make an intelligent decision capable of balancing the benefits of agents in shared control systems? Due to the advent of these attentions and questions, it is desirable to present a survey on the decision making between human drivers and highly automated vehicles, to understand their architectures, human driver modeling, and interaction strategies under the driver-vehicle shared schemes. Finally, we give a further discussion on the key future challenges and opportunities. They are likely to shape new potential research directions.

I. Introduction

INCREASINGLY powerful technologies have facilitated autonomous vehicles to prevent traffic accidents, improve traffic efficiency, and make cars available for everyone, but many social and technical obstacles remain on the road to fully autonomous driving [1], [2]. Overcoming these challenges to enable autonomous cars to drive in highly complex driving situations safely may require some time [3].Analogous to the levels of automation which range from complete human control to complete computer control [4], [5],the Society of Automotive Engineer (SAE) International defines five levels to describe autonomous vehicles and have been adopted by the U.S. National Highway Traffic Safety Administration (NHTSA) in Washington, D.C., as the government’s template. As a transition to autonomous vehicles, partially automated car, in which the human driver and automated driving agent1An automated driving agent refers to as a well-designed automatic controller in automated vehicle systems.share and complete a driving task, becomes a compromise plan before the era of fully autonomous vehicles [6]. Level 3 of automated driving, called conditionally autonomous driving, enables vehicles to mutually transit driving modes between fully automated driving and full manual control [7], but this would degrade the vehicle performance, primarily when transferring the vehicle control authority from car to a driver [8], [9]. Because it requires a transition period for the driver to be resumed in the driving process [10], which can often pose difficulties when the driver has not been actively engaged in the driving process to reacquire situation awareness [7]. Research demonstrates that vehicle automation2Automation is a technology that actively selects data, transforms information, makes decisions, or controls processes [11].harms mental workload and situation awareness [12], [13] and that reaction times increase as the level of automation increases [14]. Another kind of automated driving is called semi-automated driving, in which the automation system does not take full authority from the driver,that is, the driver should keep eyes firmly on the road, though the driver feet off the pedal, hand off the steering wheel [8].However, humans will be bored and distracted during a lowlevel supervision task [8], [15] and will show over-trust [16],[17], neglect [18], and complacency [19] on automated driving systems, which requires a long time to resume control from the automation system in critical situations [10].Therefore, Gordon and Lidberg [8] hold that semiautonomous driving does not alleviate the regular task of anticipating traffic hazards.

Human interaction with automated driving agents is a kind of human-robot interaction (HRI), and one of the most influential concepts for HRI was supervisory control [20].Supervisory control usually involves a human supervisor setting up the goals while the automated driving agent applying control actions to achieve the goals [21]. However,practically it would be difficult to establish a sharing of control between the human supervisor and the automated agent in this context [22]. Sheridan and Verplank [5] defined two sub-concepts to explain the idea of supervisory control further. One is traded control which requires that both human and computer are active at the same time, and the other one is shared control in which at one time the computer is active, at another the human is [23].

The shared control scheme presents a tractable paradigm to tackle the driving authority transition issues [24], [25] in Level 3. It features achieving a continuous authority transition between the human driver and automated driving agent.Though the authorized levels of automated driving were issued in 2016, the underlying shared-control concept has already been introduced in other fields [20], [21], [23] and will be the sharp end of cooperation between agents [26].Each agent in the shared control scheme can take their advantages when performing a specific driving task [27]. It is well known that manual control is prone to human errors. On the other hand, fully automated tasks are currently subject to wide-ranging limitations in decision-making and situationawareness. To exploit full potentials of both of human and automation while overcoming the barriers of car-to-driver transition, Mulder et al. [24] presented an entirely different control scheme – shared control systems3The definition of shared control is slightly different over different research fields because there is no single definition for shared control that is used across application domains. More detailed descriptions are referred to see Section II in review paper [21].. The human driver and the automated driving agent continuously share and cooperatively complete a specific driving task, thereby allowing drivers to enjoy driving while keeping in control consistently. Moreover, the shared-control scheme can synergize innate human capacities and technological capabilities to enable us to realize our full potential [28].Previous research experimentally demonstrates that keeping driver’s haptic control authority in the loop with continuous haptic feedback to the driver not only outperforms the conventional binary switches between supervisory and manual control [29] but also reduces distraction on a secondary task[30] and drivers’ workload [31]. Fig.1 presents a drivervehicle system where the human driver and automated driving agents cooperatively share and achieve the same driving task.The shared control over manual control has shown the advantages in many applications such as lane keeping assistance [35], [36] and steering assistance system [31],[37]–[39].

To some extent, driver-vehicle shared control is a kind of driver assistance system, and from this point of view, which includes three categories: perception enhancement (e.g.,informational assistance), action suggestion (e.g., decision or action selection), and function delegation (e.g., action

Fig.1. Illustration of the interactions between the human driver and the automated driving agent [32]–[34].

implementation) [4], [40]–[42]. The first two types have been reviewed in [3], except for the third type, in which both human drivers and automated driving agents can exert their inputs to vehicles simultaneously to carry out a specific task such as the active steering assistance systems. A welldesigned driver-vehicle shared control system should allow all engaged agents to know each other very well [43], which requires addressing the following fundamental research questions:

1) What kind of role should the human driver act in the shared control system with changing situations?

2) How to allocate the driving responsibility and authority according to the ability of two agents?

3) How to on-line evaluate the respective trust levels among drivers and automated driving agents?

4) What are the temporal scales of human adaptation and learning in changing situations?

5) What novel system identification techniques for driver state and intent exist that could allow us to study time-varying and possibly nonlinear shared control systems?

Although many works of literature have been done for specific topics, there is no paper to review and discuss these research questions comprehensively. To bridge the gap, we provide an overview of the field of decision-making scheme design and human driver modeling in shared control systems,by reviewing more than 200 closely related literature covering the keywords: shared control, driver model, shared cognitive control, self-driving, and human-automation/robot/computer interaction. Instead of reviewing rigorous mathematical algorithms of decision-making and controller design, we mainly focus on the scheme design of decision-making,human driver modeling, and the open issues with potential solutions in shared control systems, thus benefiting researchers working on the considered topic. Some other partially related literature on psychology and ergonomics is only involved and referred without in-depth discussion because of page limitation. Section II describes the underlying architectures of driver-vehicle shared control systems.Section III reviews the decision-making of two agents in the driver-vehicle shared control system from the state-of-the-art literature. Section IV details human driver modeling.Section V shows and discusses some open-ended, challenging,inevitable scientific questions. Section VI gives further discussion and conclusion.

II. Shared Control Architectures

Before giving an insight review, we first discuss the architecture of the driver-vehicle shared systems. A sharedcontrol system consisting of a human driver and an automated driving agent refers to as a two-agent system that is capable of accepting and executing commands from a human driver, or an automated driving agent, or a combination of the two [44].According to the role that the human driver plays in the driver-vehicle shared systems, the shared control can occur at two different levels: the task level and the servo level (as shown in Fig.2).

Fig.2. Two kinds of shared control architectures: (a) Task-level shared control; and (b) Servo-level shared control. rh and rc represent the expected trajectories to be followed for human driver and automated driving agent,respectively; us is the input of vehicle, uh and uc are the human driver’s o peration output and automated driving agent’s output, respectively.

A. Task-Level Shared Control

In the task-level shared control4The task-level shared control refers to as a shared-control system that allows human drivers to decompose a whole driving task into subtasks and allocate some of them to the automated driving agent (namely, share at the task level[44] rather than the servo level), which is slightly different from the traded control [5], [20], [21].scheme [44]–[46], human drivers usually act as a guide and deliver a task-specific command to the automated driving agent. Namely, the human driver can allocate subtasks to the automated driving agent while authorizing other subtasks to achieve a complete driving task. After being informed the subtasks, the automated driving agent will perform the subtasks based on the current situation condition and predefined algorithms to cooperatively achieve the whole driving task together with the human driver. The task-level shared control scheme fully exploits the strength in both machines and humans, which could alleviate the driving burden of a human. A very intuitive example is that human drivers manually activate the adaptive cruise control (ACC)systems when driving on the highway [47] to allocate longitudinal control authority to the ACC agent while authorizing lateral control by himself/herself.

B. Servo-Level Shared Control

Servo-level control in the driver-vehicle system usually focuses on servo control. The control input ( us) to a vehicle is typically the combination of human drivers’ operations ( uh)and automated driving agents’ output ( uc), as shown in Fig.2(b).In the servo-level shared control, differing from the task-level shared control situations where the automated driving agent will take over the task-specific control, human drivers will always be engaged in the control process of vehicle movement at the servo level. The combination of outputs from human drivers and automated driving agents should be well designed.Analogous to human-robot shared control, researchers[48]–[52] combine them intuitively by

where λ ∈[0,1] is the weighted coefficient to adjust the proportion of uhand ucin us. The allocation of driving authority for a human driver and an automated driving agent is determined by λ which can be either fixed or continuously adaptive. Pure human control is achieved when λ=1 and pure automatic control when λ=0. Therefore, the value of λ can impact the shared control scheme by determining if the driver is presented in the loop. This parameter can be modulated manually (e.g., in [53]) and automatically based on the driver’s states in certain situations in which the driver needs assistance to perform more rapidly and safely (e.g., in [49]). The design of the λ will be discussed in Section III.

Different from linearly combining the human driver’s input with the automated agent’s input by (1), some probabilistic models [54] have also been proposed by explicitly taking into account the uncertainty in the interaction and modeling this combination as a joint probability distribution [55].

The approaches of combining uhand uccan differ from each other in different servo systems. As a consequence, the servo-level shared control can be further divided into direct shared control and indirect shared control [56], [57].

1) Direct Shared Control: The direct shared control allows both human drivers and automated driving agents to simultaneously exert actions on a control interface, of which the output remains the direct input to vehicle systems, as shown in Fig.2(b). Such systems are usually haptic shared control [58] since both human drivers and automated driving agents will directly influence the inputs on the haptic control interface (e.g., steering handwheel and brake/throttle pedals).Also, human drivers can even percept the assistance torque applied by automated driving agents through the shared haptic interface. A general architecture of haptic shared control can be seen in [59].

2) Indirect Shared Control: Differing from the direct shared-control scheme, the indirect shared control scheme shapes the input to the controlled vehicle system by mixing the out of control interface (usually as a result of human contributions) and output of the automated driving agent [37],[38], [58], formulated by us=g(rh,rc,uh) as shown in Fig.3.A typical application in the human-vehicle systems is steerby-wire (SBW) systems [60] which estimates the driver’s desired steering angle from the driver operations and then generates and applies the steering angle directly to front

Fig.3. Illustration of an indirect shared control scheme [56], also called“input-mixing shared control” [58].

wheels.

III. Decision-Making in the Servo-Level Shared Control Scheme

This paper mainly focuses on the decision-making in servolevel shared control scheme rather than the task-level shared control scheme. A well-described model of the entire drivervehicle system can offer one a better understanding of how each subsystem works and what the relationship between them is. Some researchers hold that driver modeling should be integrated into the road-vehicle system for control purposes to improve the mutual understanding between the driver and automation [61], [62]. Hence, a driver-in-the-loop (DHL)5The “l(fā)oop” can refer to an information processing control loop (i.e., attentive to driving task) or a sensory-motor control loop (i.e., vehicle control), or both[63]. Here, the “l(fā)oop” refers to as a sensory-motor control loop.vehicle model [61], [64], [65] is usually incorporated into a shared control system and then describes the complete DIL systems from a control perspective.

For this purpose, first, the road-vehicle dynamic model is usually formulated using the state-space representation

where x is the road-vehicle state, A is the road-vehicle system matrix, B is the input matrix from road-vehicle system, Bwis the system disturbance matrix, ucis the controller input, uhis the human operation input, w is a system disturbance. Then the human driver is formulated from a control perspective as

where rhis the driver’s desired trajectory or reference trajectory. H(rh,x) represents human driver model and can be formulated from control perspective [66], stochastic perspective [67]–[69], or cognitive perspective [70], which will be discussed in Section IV. The driver model towards the applications to steering system control (or lateral control,path-following control) is the preview driver model and its extensions (see review article [66]) since it is easy to integrate them into the state-space-based vehicle model. Human drivers output their operations and apply to the vehicle by comparing the desired trajectory and the current trajectory through their internal model [71], [72]. The internal model can estimate the current/upcoming states of the subject vehicle and trajectories of surrounding objects (e.g., vehicles, pedestrians, and bicycle users). Substituting (3) into (2), the driver-vehicle model (i.e.,DIL model) can be formulated by a new state-space representation

where z is the augmented state consisting of human driver model state and road-vehicle system state, ADILis the drivervehicle system matrix, Bhis the human input matrix, and Bcis the controller input matrix. The common DIL models based on the state-space representation can be found in [61], [64],[73]–[77] and some popular cases are also listed in Table I.The state-space DIL model provides an analytical way to assess the stability of the shared control system [61], [90] and an standard way to design controllers [50], [91]. A welldesigned decision-maker and controller for the shared control system should assist human drivers driving safely and smoothly while without causing any conflicts with human drivers. Based on the incorporated DIL model (4), given a desired trajectory/reference and the human driver operations,the optimal controller input can be obtained by solving an optimal problem in the general form

where J(·) is the objective function that could encompass human driver’s input and other constraints (e.g., vehicle dynamics and human driver’s physical limitations). To solve the optimization problem between a human driver and an automatic controller, one of the biggest challenges is to formulate the relationship between uc, uh, and us, e.g.,allocation of the control authority, λ. In what follows, the shared control strategies can be formulated according to prior knowledge or dynamic programming. Thus, two ways to design the decision-making strategies are listed: rule-based and game theory-based.

A. Rule-Based

For the rule-based method, one direct way is to use (1) with the requirement of designing λ(t). Most research predefined a different kind of rule according to prior knowledge, which can be roughly grouped into three categories and discussed as follows.

1) Piecewise Function: Due to the complexity of dynamic environment and disturbance, an intuitive way to design λ(t) is using rule-based piecewise function. The piecewise function is primarily developed to tackle the shared-control issues in robotics such as wheelchair and industrial robots [92], [93]and then is introduced to intelligent vehicles afterward because of its robustness and practicality in terms of controller design [94]–[96]. For instance, Jiang and Astolfi [92], [96]defined three space sets to divide the reachable set into three parts by judging the level of safety — safe, close, and dangerous. Correspondingly, a three-level piecewise function consisting of the safe Rs, hysteresis Rh, and dangerous subsets Rdwas proposed to design λ(t):

If human behaves “dangerously” then the feedback controller (i.e., automated agent) is active and resumes the control authority; if human behaves “safely” then the vehicle only responses to human’s operations; if human behaves in

“hysteresis” set, then the vehicle runs under a predesigned shared-control law.

TABLE I DIL Model Using State-Space Representation Towards the Applications in Vehicle Dynamic Control

Besides, the piecewise function is an analytical way to take account human factors and driving situations in the light of its practical integration with human’s prior knowledge. For example, Li et al. [49] utilized two piecewise functions to assess driving situations and vehicle performance separately based on their empirical knowledge and then fused these piecewise functions as the λ(t). Saito et al. [25] used an exponential function of the steer-wheel torque applied by the human driver to estimate how much torque assistance the assisting agent should deliver.

2) Exponential Function: The second approach to obtain a seamless and smooth λ is to use the family of the exponential function:

where ζ(t) is the activity factor, α0and α1are the tuning parameters, and αsafeis the parameter to guarantee safe and model convergence. The exponential function has been widely used in human-robot shared control, and then introduced to tackle control authority transfer and allocation in humanvehicle shared control afterward. For example, Sentouh et al.[50] integrated the discrete driver state (distraction) into the exponential function to obtain a continuous shared control factor λ(t). Wang et al. [37] designed a shared steering control law using an exponential function with considering different driving styles to improve vehicle performance and reduce drivers’ workload when taking curve negotiation. However, a religious setting should be made when applying this exponential function, since the derivative of (7) could be discontinuous when the denominator switches between positive and negative.

In order to determine a continuous weighted coefficient ( λ)for the shared control systems, researchers also combine the piecewise function and exponential function to evaluate the safety level for decision-making in human-robot shared control [97]. This kind of shared control strategy has been carefully borrowed to improve human-vehicle shared control performance. For example, Sentouh et al. [50] utilized the function in the format of (7) to obtain a continuous authority allocation factor from a discrete driver drowsiness monitoring factor. Besides, some probabilistic shared control strategies towards complex, dynamic environments were also proposed by modeling both the human’s intentions and the automated agent as a probabilistic function to improve the shared control performance with a exponential function [91], [98].

3) U-Shape Function: Another approach is to directly compute how much assistance the driver needs based on the U-shape function, i.e., the relationship between drivers’workload and performance as well as the need for assistance,as shown in Fig.4. For instance, Nguyen et al. [64], [101]designed an assistance torque Ta=μ(a)Tsto reduce drivers’workload and improve vehicle performance using the U-shape function of drivers’ activity μ(a), where Tsis the required input torque from the vehicle, and a is the drivers’ activity denoted as steering angle. Oufroukh and Mammar [88] also proposed a similar computation model to compute the assistance torque using U-shape representations during lane keeping or obstacle avoidance maneuver.

B. Game-Theory-Based

Differing from the rule-based method where a predefined shared control law λ(t) is adopted, some researchers treated human drivers and automated driving agents in the shared-

Fig.4. U-shape representation of driver’s need for assistance [8], [99],[100].

control system as two-player with dynamic interaction, as shown in Fig.5. The two-player assumption enables gametheoretic approaches to be available to tackle the relations between human drivers and automated driving agents.

Fig.5. Dynamic games between a human driver and the automated driving agent.

Game-theoretic approaches have been widely applied to tackle the dynamical decision-making problem where two or more agents make decisions that influence one another welfare [102] such as vehicle-to-vehicle [103]–[105], grid-tovehicle [106], [107], collision avoidance at an intersection[108]. Applications of game theory in modeling road user behaviors and traffic or transportation can refer to the review literature [109], [110]. Na and Cole made a comprehensive classification for dynamic games as shown in Fig.6.Depending on the interactive type between human drivers and automated agents, the dynamic games between two agents can be classified into noncooperative and cooperative games[112], as illustrated below.

1) Noncooperative: Drivers and automated driving agents consider themselves as individuals and concentrate on pursuing their interest. More specifically, in the noncooperative game theory, the strategy type of driver and automated driving agent can be derived using Nash equilibrium and Stackelberg equilibrium. A Nash equilibrium emerges in situations, where drivers and automated driving agents derive their strategies by considering others’ strategies,and they act simultaneously. A Stackelberg equilibrium emerges in situations, where one agent (i.e., human driver or automated driving agent) is the leader, and the other one serves as a follower.

Fig.6. Classification of dynamic games [111].

2) Cooperative: Drivers and automated driving agents have a sense of collectivity and attempt to enter into a binding agreement of interest, where the goal of each agent is identical, and their strategies are derived from global optimality [113].

Usually, the two agents are assumed to be rational with individual objectives [111], [114]–[116]. The vehicle is controlled by a human driver and an automated driving agent,modeled as

The goal of a human driver and automated driving agent is to minimize their objective function.

1) Noncooperative Nash Scheme, where the automated

driving agent will compensate for human drivers’ erroneous actions to imitate the possible opposite effects, and simultaneously, human drivers will also estimate action applied by the automated driving agent, and to neutralize its influence by further changing her/his inputs;

TABLE II Game-Theoretical Applications in the Shared Control Between Human Driver and Automated Driving Agent

2) Noncooperative Stackelberg Scheme, where the automated driving agent will compensate for human drivers’erroneous actions to imitate the possible opposite affects, and sequentially human drivers will generate actions when they have a very well knowledge of the actions applied by automated agents, and vice versa;

3) Cooperative Pareto Scheme, where both human drivers and automated driving agents try to account for each other’s desired trajectory, and simultaneously will react to both each other’s actions.

The Nash and Stackelberg equilibriums can be employed to solve the closely coupled optimization problem in drivervehicle interactions. Either analytical or approximated solutions can be derived as the human driver’s and the automated driving agents’ control actions. When the approximated solution is concerned, the following two expressions hold:

For example, Flad et al. [116] introduced an approximated Stackelberg solution [122] to solve the problem between a human driver and ADAS controller (i.e., automated driving agent) by treating one of them as a leader and the other one as a follower. In [77], Li et al. designed a continuous role adaptation of human-robot shared control using the Nashequilibrium, where human and the automatic controller (i.e.,automated driving agent) can simultaneously exert control to the robot, instead of directly using the Stackelberg solution[114], [107]. An adaption law was also designed by comparing the difference between measured human drivers’inputs and predefined Nash equilibrium. Besides, the game theory could also integrate the uncertainty from human drivers and external factors into the shared control system [121] with a stochastic dynamic programming solver.

Although the above mentioned two typical methods offer us ways to design the shared control strategies between a human driver and automated driving agents, the two agents could also fail to cooperate when a wrong estimation of current states regarding driving situation perception and human driver intent occurs. Also, a poor-designed shared control strategy could even bring four main adverse effects [123], [124]: loss of expertise, complacency, trust, and loss of adaptivity.Therefore, studies dealing with human-vehicle cooperation have to consider the human’s characteristics such as uncertainty according to reduce the conflicts with the driver.In what follows, we will discuss about modeling human drivers from different perspectives.

IV. Human Driver Modeling

In order to obtain well-designed shared control interactions between a human driver and a highly automated vehicle,Norman [125] stated that human must always be in control,must be actively engaged and adequately informed, and that human and automated vehicle must understand each other’s intents correctly in the complex driver-vehicle systems.Therefore, understanding and modeling human drivers’sensory dynamics, cognition processes, hidden states, and operation characteristics are equally important as dynamic vehicle systems for driver-vehicle shared control systems.Human drivers usually complete a driving task at three levels[126], [127] (Fig.7): strategic level, tactical level, and operational (control) level. The first two levels involve cognition while the third level involves execution. According to the previous review of all literature, we introduce and discuss the driver model from its functional modules,modeling approaches, and driver’s intent inference as well as state detection.

Fig.7. A hierarchical structure for modeling human drivers when driving[126]. EI = Environment inputs.

A. Functional Module

1) Sensory Dynamics: A recent review in [128]demonstrates that drivers’ sensory dynamic characteristics play an important role in driver-vehicle system design,making them interact more friendly and safe. Driver’s sensory dynamics used for vehicle speed and direction control mainly include [128] (ranked by the level of importance for a regular driving task [71]):

a) Visual — The visual system is the human driver’s only means of detecting the future road trajectory. A general review of overall literature demonstrates that visual information is the highest significant in normal driving process, accounted for about 90% of all sensory information[71]. Driver’s visual information (e.g., eye gaze [129], [130])can also reflect driver’s underlying intentions (e.g., lane change intents [131], readiness to take over from automation[132]), mental or physical states (e.g., fatigue detection [133]),and upcoming actions [134].

b) Vestibular and kinesthetic — Human drivers usually use the motion information (e.g., acceleration and rotation of vehicles) derived from vestibular and kinesthetic channels as a supplement to visual information for control task, which could also contribute to the human combined arm-trunk motion[135].

c) Somatosensory — The somatosensory information used in control of vehicle steering and speed mainly includes tactile and haptic information such as pressure on the gas/brake pedal[136] and steering torque on the steering handwheel [39].

d) Auditory — The auditory information usually is used as a supplementary cue within a multi-channel environment in a normal driving process.

The sensory dynamics have their physical characteristics such as time delay, perception limitations, and coherence zones because of human’s physical, biochemical, mental limitations and the human’s ability to perceive and process information. A driver model which integrates human drivers’sensory dynamics could help understand humans, thus for improving comfort, safety and driveability of driver-vehicle shared control systems and reducing the incompatibility6Compatibility here is referred to as the quality describing the fit or match[32] between a human driver and automated driving agents, regarding the outer (e.g., interfaces) and inner (e.g., cognition) interactions [137] between them.or negative interference [123] between the human driver and automation. The haptic information on the steering wheel has been widely used in the shared steering control systems to reduce the driver’s mental/physical workload and to improve driving skills [29], [138], [139]. A sensorimotor driver model was developed to improve the shared-control performance by considering both visual and kinesthetic perception and including compensatory and anticipatory processes [81].

2) Cognition: John A. Michon has mentioned the cognitive driver model from a critical view of introducing the behavioral sciences and psychology to understand the human decision-making processes [126]. Much of the decisionmaking generated by human drivers is over discrete actions,such as choosing whether/when/how to lane change [140]. To capture these discrete features, researchers modeled human drivers’ high-level tactical behavior (e.g., speed selection and decision making) and strategic behavior (e.g., route planning and navigation). One of the most utilized means is based on the “adaptive control of thought-rational (ACT-R) [141]”cognitive where the discrete nature of drivers’ control actions is captured from a cognitive perspective. For example,Salvucci et al. developed an integrated cognitive pathfollowing driver model [142] and lane-change driver model[143] using the combination of the ACT-R cognitive architecture and perceptual-motor process.

Some researchers also developed cognitive driver models based on on-hand knowledge or with new insights in experimental data. Misener et al. [144] developed a cognitive car-following model to avoid rear-end crashes with a stopped lead vehicle by fusing current knowledge derived from experimental data. Liang et al. [145] developed a system to detect driver distraction in real-time by cognitively analyzing three indicators: how to define distraction, which data were input to the model, and how the input data were summarized.Liang also demonstrated that combining cognitive and visual distractions can improve vehicle performance than each of them [146]. Much more literature on modeling and analyzing drivers’ cognitive distraction can see references [147]–[150].A review paper for modeling driver behaviors in a cognitive architecture refers to [70], [72].

Based on the cognitive driver model, a cognitive assist can be potentially provided in the context of driver-vehicle shared control to reduce harmful interference between two agents.Cai and Lin [151] proposed a coordinating cognitive assist to determine when an assist should be provided and how much assistance to be supported for a steering assist control system.The cognitive assistance was divided into three stages to assist drivers in acquiring information, analyzing information and making a decision, and implementing action [4].

3) Neuromuscular-Skeletal Dynamics: Existed research has demonstrated that a good understanding of neuromuscularskeletal dynamics has constraints upon the dynamics of perception-action coupling [152] and is significantly vital for a well-designed shared control system, for example, avoiding subtle conflict between a human driver and automated agent[153]. For human drivers, the neuromuscular-skeletal dynamics in human-vehicle systems mainly include the arm and foot, representing the lateral and longitudinal control,respectively.

For the dynamic properties of a driver’s holding a steering wheel, Pick and Cole [154], [155] investigated the effect of the driver opposing a constant offset torque and the effect of the driver co-contracting the muscles, and found that both actions will increase the stiffness and damping of the arms. A linear model of the neuromuscular system, muscles, limbs,and the vehicle was then set up and applied to a driver simulator [156], a guideline to shared control [58], a pathfollowing driver–vehicle model [157], and a driver lateral control model [80], [158]. Besides, the characteristics of driver neuromuscular dynamics are different from each other and affected by steering systems (e.g., active and passive) and hand positions on the steering wheel [159], which should be considered when designing a controller.

For the dynamic properties of a driver’s low leg and foot hitting the gas or brake pedal, researchers mainly focus on the ankle-foot neuromuscular dynamics, which enables us to gain insight into both responses to visual feedback and haptic feedback. Abbink et al. [136] investigated the biomechanical properties of the ankle-foot complex (i.e., admittance) [160]and developed a driver support system that uses continuous haptic feedback on the gas pedal to inform drivers of the separation to the lead vehicle.

B. Modeling Approaches

1) Control-Theoretic Driver Model: Understanding the potential and the limits of a tightly coupled driver-vehicleroad system are not trivial. Many kinds of driver models towards the application in vehicle dynamics control can be found in review articles [66], [161], [162], in which literature before 2014 was listed, including lateral and longitudinal driver models. Table I lists some popular driver models toward vehicle dynamics control applications and indicates that the single/two-point visual preview driver models or their extensions are highly preferred since they are easy to be integrated with vehicle models. However, one of the most significant limitations is that the single/two-point visual preview driver models assume that human drivers’ references or desired trajectories are exactly known, which is not always available in the real application. Besides, for such two-point visual preview models, drivers’ essential physical characteristics such as the reaction time delay, anticipatory and compensatory can be reflected, but not cognitive characteristics such as neuromuscular dynamics of driver arms and decision-making processes. For example, the controltheoretic driver models (see review [66]) usually ignore issues of whether or how to perceive the model inputs from the external environment and how drivers correctly interact with other automatic controllers [70] via visual, haptic, or auditory sensors [71].

Most control-theoretic driver models have been used in the driver-vehicle shared control systems with a game-theoretic scheme. With this purpose, the human driver and the automated driving agent should be first modeled, which makes it feasible to estimate each other’s actions. One of the most popular approaches for modeling automated driving agents is using control theory (e.g., model predictive control)since it can describe drivers’ ability to predict vehicle’s future states [163] based on their internal model or individual’s driving skills [164]. Na and Cole [111], [114] applied the combination of the distributed MPC and linear quadratic dynamic optimization (LQDO) to formulate human drivers and automated driving agents. The objective function with quadratic structure is also widely used to design an automated driving agent considering human drivers’ forthcoming motion primitives [115] or drivers’ haptic inputs on the control interface [77].

2) Learning-Based Driver Model: Though the driver models as mentioned above could describe and predict drivers’behavior, actions, and states with relatively satisfied performance, they did not concern the dynamic, stochastic decision-making processes of driver behavior, which requires models capable of connecting temporal and spatial processes.For this purpose, some researchers also utilized learning-based approaches to deal with highly nonlinear properties of driver behaviors, such as neural networks [165], [166]. Research in[68], [167]–[169] demonstrates that the Markov models combined with Gaussian mixture models achieve a satisfied performance of capturing driver intent and action. Besides, the Bayesian inferences [170], [171], autoregressive exogenous(ARX) [172], and deep learning [173] were also developed.These learning-based approaches highly depend on the collected training data, and some of them are data-hungry such as deep neural networks [174].

C. Intent Inference and State Detection

Correctly inferring drivers’ intents and states is profoundly essential to design an automatic controller capable of delivering an adequate input not only to follow/track the desired trajectories but also to avoid intrusive interventions between the human driver and the automated driving agent[175].

1) Intent Inference: Steering wheel, as a direct interface,allows human drivers and automated driving agents to act and exchange information in a simultaneous and continuous way[176], [177]. Therefore, human drivers’ intents can be directly captured through the torque applied on the steering wheel by drivers. For example, in a torque-based steering assistance system [39], the automated driving agent uses sensors to obtain the steering torque forced by drivers, and inversely,human drivers can also react to his/her haptic perception information from the steering wheel. Nguyen et al. [74], [101]utilized the torque applied to the steering wheel by drivers as an indicator to compute how much assistance torque an automated driving agent should provide. Li et al. [77]proposed a continuous adaption law for the human-robot shared control system to determine the automated driving agent’s role (i.e., leader or follower) by comparing the measured torque applied by the human to the predefined Nash equilibrium computed through optimal control techniques.

In addition to using the torque applied on the steering wheel by a human, the dynamic neuromuscular analysis of driver arms or legs can also provide a guideline for shared control design of a steering system [39], [58], [80], [157], [177] and the gas/brake pedal control [136], [177]. For instance, in order to reduce the intrusive intervention between human drivers and automated systems, Ziya et al. [39] modeled human drivers’ steering behavior by combining the neuromuscular response of drivers and the desired steering angle that was a function of vehicle states and road geometry. Moreover, the impedance of a haptic torque was used as an indicator of the drivers’ intents. Some researchers also designed a guidance torque to assist drivers to keep the vehicle in the driving lane[178] and to improve vehicle safety for fatigue-related driver behavior [179].

Much literature on modeling human drivers’ neuromuscular dynamics of steering behavior has been found in [76], [155],[157]. The model of driver’s arm and steering dynamics usually combines with a path-following control model,obtaining a linear driver model with neuromuscular dynamics[80]:

where J?are the inertia, damping and stiffness of ?, with ?being driver arm or steering systems; nsis the steering gear ratio; MTis the torque arising from the lateral forces and selfaligning moment; Tmis the muscle torque; and θsis the steering wheel angle. Model (13) has been used to infer drivers’ intents. For example, the changes of damping and stiffness of driver’s arms can reflect whether the automatic controller’s outputs satisfy drivers’ desired trajectory or the conflict level between drivers and automated driving agents[58], [59], [154]. Instead of using indirect cues (e.g., steering angle, vehicle dynamics), direct human-observing cues (e.g.,body gesture, head, hands, feet, and gaze direction) can also be used to predict driver intent [180]–[183].

In addition to directly using the haptic information, driver intent can also be inferred and predicted according to dynamic driving environments such as peripheral vehicles [184] and vehicle position to lane edges [185]. The dynamic Bayesian networks (DBN) [186], [187], Markov decision processes(MDP) [167], [181], and partially observable Markov decision processes (POMDP) [188] are one obvious place to start,assuming that one can extract the underlying latent processes of driver behavior which is a dynamic and stochastic process.Besides, the correlation between human driver gaze behavior and steering moments will decrease while increasing the allocation ratio of control authority for intelligent driving agents [189], indicating that driver gaze behavior could be used to infer driver’s intent and avoid conflict.

2) State Detection: Correctly detecting driver state (e.g.,sleepiness, drowsiness, fatigue, distraction, impairment) offers an opportunity to make a practical decision of authority allocation, thus improving vehicle safety. For example,Saito et al. [36] proposed a dual control scheme for lanekeeping assistance systems by detecting the driver’s sleepiness level using eye blinking frequency and facial information. More driver intent detection and inference using direct human-observing cues can refer to the review paper[133], [190].

Visual distraction or cognitive distraction have been investigated by combining vehicle state [36], [147], [148],drivers’ visual state [145]–[147], [149], and operations [147],[149], [191]. Answers to the question of how to measure a driver’s cognitive distraction have been given in [150].Learning-based approaches such as deep sparse autoencoders[192], deep belief networks or DBNs [188], support vector machines (SVM) [145], [193] have been widely used to detect and classify driver distraction.

V. Future Challenges and Opportunities

Due to the limited ability to understand human drivers,many open-ended questions still exist in driver-vehicle shared control systems. This section will present and discuss some open-ended, challenging, inevitable questions regarding the shared control strategies, the trust or over-trust, and authority allocation, followed by future opportunities.

1) How to design the adaption law or adaptive/adaptable shared control? Also, what kind of role of the human driver should be in driver-vehicle shared control systems?

In Section III, we have discussed different ways to design adaptive shared control between the human driver and an automated driving agent. Most of them were from U-shape,noncooperative game theory, and the torque applied on the steering wheel by the human. The role of human drivers in highly automated vehicles can be defined as

a) Convertible role between leader and follower [115](game theory);

b) Symbiotic relations with the automated driving agent(page 24 in [194]);

c) Being an active driver, passengers, or passive drivers, but they may still be required to take over control [190];

d) Or being parallel [46], [195].

The different roles of human drivers in driver-vehicle systems result in various shared-control paradigms. In terms of the methods researchers utilized, the shared control performance can be improved by considering individual characteristics, classifying the levels of human-automation interaction, and integrating with cognitive psychology.

The U-shape control law only qualitatively describes the relations between drivers’ workload and the needs for assistance as well as the driving performance, but not a quantitative expression. As a result, the adaption control laws derived from U-shape was greatly different, for example, in literature [64] and [88]. Many factors could cause the difference, such as the diversity in individuals’ driving experience and physical/psychological status. Classifying the type of human driver [196], regarding their abilities and characteristics, and then designing a personalized driver model [197] capable of describing and adapting this driver’s characteristics [198] could be an efficient way to tackle this kind of problems.

In terms of the game-theory-based adaptive law, researchers usually modeled human drivers by assuming that drivers had a perfect internal model [72] for understanding and predicting vehicle states as the same with an automatic controller, that is,both agents had an identical, deterministic objective functions[114], [115], [119]. In the real world, however, humans driving is not always a deterministic process [199], but in nature, a stochastic and dynamic process [167], [200], and even impaired behaviors (e.g., fatigue and drunk). Therefore,the stochastic driver behavior and divergences among drivers should be modeled and accounted in future work. Besides, the underlying relations (the role of a human driver in the drivervehicle system with shared control schemes) between human drivers and automated driving agents remain open-end.

In response to the problems above, classifying the types and levels of human-automation interaction [4], [12], [201] and increase the agent’s adaptability [202] could be an applicable approach. The automation functions usually cover four types:information acquisition, information analysis, decision and action selection, and action implementation. Within each of these types, the automation is defined and treated as a continuous level from low (i.e., fully manual) to high (i.e.,fully automatic). Human performance consequences in terms of types and levels constitute primary evaluative criteria for automation design. This approach has been used in driver assistance system design [151].

Besides, Beetz et al. [203] and Heide and Henning [204]proposed the idea of cognitive car separately — a technological cognitive system that can perceive itself and its environment, as well as collect and structure information in an autonomous way. In the cognitive cars, some key issues remain regarding action implementation (i.e., function delegation):

a) What kind of actions should be implemented?

b) When to add the actions?

c) How to implement the function delegation appropriately?

To date, we are not able to answer the three questions systematically, but most research focuses on i) the effects of haptic support systems on driver performance; ii) vehicle stability control for collision avoidance, and iii) active steering system with adaptive assisted systems. One of the potentially useful approaches is to consider and built cognitive cooperation based on cognitive psychology, as mentioned in[123]. Li et al. also holds that the cognitive cars will be a new frontier advanced driver assistance system for research [205].Besides, the Petri net modeling has been demonstrated as an alternative solution [206].

2) What is the appropriate trust? Alternatively, would drivers take a seat as passive occupants, who fully trust their vehicles?

Over-reliance on automated driving and driver complacency are often problematic [19], thus resulting in, for example, a long reaction time [10]. The driver, who fully and deeply trusts the ability of driving automation, therefore failed to intervene and take manual control even as the driver crashed the car [11], [17]. Fortunately, human drivers have an appropriate level of trust in the automated driving agent, such as being aware that the automation system can better sense and faster response [207] and displaying system situation awareness [208], but sometimes can also make mistakes.Driver performance can also be improved by effectively conveying the limitations of automated driving to human drivers [209], [210]. Besides, an appropriate, elaborate practice for drivers could mitigate the negative impact of over-trust in the automated driving agent on reaction time[41].

Besides, when the desired trajectories derived from the driver and the automated driving agent are similar, the vehicle inputs from both of them locally differ but combine without conflict. However, when the desired trajectories of two agents are different, things will become intractable and bring a question: which one input should be trusted and exerted to the vehicle? One potential way for tackling this issue is to develop a psychologically-grounded cognitive-physical model [194]capable of correctly describing and predicting the driver’s desired trajectory from the operational level, tactical level, and strategic level by understanding information processes(neuroscience) and cognitive abilities (psychology) of the driver. The cybernetic driver models integrating visual(including anticipation and compensation) systems with neuromuscular systems (or motor processes) are an efficient way to benefit shared control. The readers can refer to the review paper [211] for details about the model structure and parameter identification. Another potential way is to design a metric to evaluate and analyze the conflicts between a human driver and automated driving agent [212]. Many indicators have been developed to evaluate shared control performance,and they mainly concern four aspects: accuracy (e.g., pathtracking errors) [49], [50], [211], safety (e.g., risk metrics such as time to lane crossing) [49], [120], compatibility (e.g.,existing conflicts or not) [50], [211], and robustness (e.g., the vibration in the resonance response of vehicle) [50], [61]. In the haptic shared control system, the directions and periods of the torque applied to the steering wheel by a human driver and automated driving agent are most commonly used. For example, researchers in [211] considered four aspects of steering torque to evaluate conflicts in haptic shared control:consistency rate, resistance rate, contradiction rate, and contradiction level.

3) Which way is the best transition in authority, ability,responsibility, and control (A2RC)?

Before presenting the open-ended questions, we should define some basic concepts in interactions between human driver and automated driving agents according to [213]:

a) Authority, which can be defined by what the human driver or automated driving agent is allowed to do or not to do. Further, the authority can be partially and continuously transferred between human drivers and automated agents.

b) Ability, which can be defined as the possession of the means or skill to perceive and select adequate action and act appropriately.

c) Responsibility, which can be assigned beforehand to motivate certain actions and evaluated afterward, where the human driver or automated driving agent is held accountable or to blame for a state or action of the driver-vehicle system and consequences resulting thereof.

d) Control, which means having the power to influence the vehicle states.

Fig.8 shows the relations between the four concepts. Based on these basic definitions, the issues that existed in humanautomation interaction systems are also encountered in the human-vehicle shared control systems since the highly automated vehicle is a case-specific automation system [4],[213], for example:

i) How do we balance between exploiting increasingly powerful technologies and retaining authority for human drivers?

ii) How can we define clear, safe, efficient and enjoyable roles between human drivers and automated agents?

Fig.8. Relations between ability, authority, control, and responsibility[213].

iii) Which of the subsystems of future human-vehicle systems should have which ability, which authority and which responsibility? Alternatively, which system functions should be automated and to what extent?

iv) What other concepts besides authority and responsibility do we need to describe and shape a dynamic but stable balance between human drivers and automated driving agent?

According to these predefined terms, there is an allocation problem in terms of A2RC between human drivers and automated driving agents. Most literature only focuses on the control authority between human drivers and automated driving agents, but ignore the relations regarding the A2RC.In the real world, the authority, ability, responsibility, and control are not independent. In [99], an A2RC diagram was developed to analyze and design human-machine systems,which can be used to tackle the transition issues of A2RC in driver-vehicle shared control systems. Besides, Acarman et al.[214] also proposed a control authority transition system for collision and accident avoidance considering drivers’ physical and cognitive capacities as well as the situation/danger/hazard analysis. In addition to the A2RC, the H-metaphor (H-mode or horse metaphor) [215], i.e., a metaphor of the relationships between a horseback rider and the horse, could be as a guide to open new horizons in the shared-control systems, which has shown its remarkable achievement in aircraft co-pilot design[216] and highly automated driving [217].

On the other hand, exploiting brain-related signals could directly offer rich information about human driver intent,ability and thus allow us to optimize the allocation of authority and reduce the conflict between two agents. A welldesigned human-vehicle interaction interface integrated with human driver psychological and psychobiological characteristics [218], [219] and active capabilities [220] could benefit shared-control tasks by offering much cognition information. Moreover, the feedback from the interaction interface could also show if a poor-designed interface, adverse effects on human driver [221], and control performance [222].Similar to the human-computer interaction design (see pages 29–48 in [223]) from an empirical research perspective,human factors regarding perception sensors (i.e., vision,vestibular and kinesthetic, somatosensory, and auditory, as discussed in Section IV-A-1)), action responders (i.e., limbs and legs, as discussed in Section IV-A-3)), and brain factors(i.e., perception, cognition, and memory, as discussed in Section IV-A-2)) could be carefully considered and implemented based on the methods of human-robot shared control (see the review literature in [224], [225]). One of the typical applications is to use brain-related signals to enhance shared-control performance through the brain-vehicle interface [226]–[229]. Various brain-vehicle interfaces with different capabilities (e.g., adaptive brain-vehicle interface[230]) have been designed based on EEG signals [231]). More literature refers to the review literature [21], [232].

Moreover, human factors (e.g., the way to shared information with the human driver) can influence the transitions of A2RC. An inefficient design of shared control interface would cause conflicts between human drivers and automated driving agent, even the catastrophic consequences.Eriksson et al. [233] investigated different ways to support driver decision-making during automation-to-manual transitions in a take-over scenario. Most related works have been comprehensively discussed in the recent review paper [234].However, it is still not entirely clear about the influence of different ways (such as torque, steering angle, and vibration)to information transition between a human driver and automated driving agents on the performance during shared control.

VI. Discussion and Conclusion

The motivation for this review paper is to illustrate how to model a driver-vehicle shared control system and understand the challenges and opportunities for highly automated vehicles with human drivers still retained in the control loop. We have discussed the architectures of driver-vehicle shared control systems, the approaches to modeling the complex systems,and the future challenges and opportunities. We have provided a survey of the progress over the past decades in drivervehicle shared control technologies. In order to understand the complex driver-vehicle systems, we decoupled it into different subsystems and summarized how to model them by reviewing the state-of-the-art literature. Finally, we have provided discussions on the challenges and opportunities in this field.While advanced driver assistance systems have been developed and introduced over the past decade, a deeper and more holistic understanding of the relationship between human drivers and automated driving agents and the way that human drivers cognitively interact with the driving environment will remain an active area of research in next few years.