Review on bio-inspired polarized skylight navigation

Fng KONG, Yingjing GUO, Jinhu ZHANG, Xiojing FAN,Xiohn GUO

a School of Electrical and Automation Engineering, Shandong University of Science and Technology, Qingdao 266590, China

b School of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590, China

c Qingdao Intelligence Ocean Technology Co., Ltd., Qingdao 266590, China

d School of Information Science and Engineering, Qingdao School of Shandong University, Qingdao 266237, China

KEYWORDS

Abstract In this review,the research progress of bio-inspired polarized skylight navigation is evaluated from the perspectives of theoretical basis, information detection, sensor design, and navigation realization.First, the theory for characterizing the polarization mode of the skylight was introduced.Second,using sunlight,moonlight,and ocean as backgrounds,the measurement results of skylight polarization distribution under different weather conditions are described to compare the variation patterns.Third,the development history and research outcomes of bionic polarization navigation sensor for polarized skylight detection and navigation information calculation are categorized into two types, namely non-imaging and imaging types.In precision measurement, the non-imaging type is higher than the imaging type, and the accuracy that it can reach is ± 0.1°of navigation accuracy without drift error.Fourth, two polarized skylight orientation algorithms,E-vector-based method and Solar Meridian-Anti Solar Meridian (SM-ASM)-based method are summarized.Fifth, this review details the combined application of polarized skylight navigation sensors and Inertial Navigation System(INS),Global Navigation Satellite System(GNSS),Vision,Simultaneous Localization and Mapping(SLAM),and other navigation systems.The yaw and trajectory accuracy can be increased by about 40% compared to classical navigation system in complex outdoor environments.Finally, the future development trends of polarization navigation are presented.

1.Introduction

Autonomous vehicles use detection equipment to adjust their position, attitude, and speed relative to a coordinate system in real time without relying on any external signal to automatically navigate roads.1

INS is a self-contained navigation system that provides excellent concealment, high independence, and strong antiinterference ability while providing considerable information,including speed, position, height, orientation and attitude of moving vehicles.Although the INS can provide accurate navigation information within a short time,maintaining the highprecision level throughout a mission is difficult for a pure system because of the accumulation of errors over time.

In addition to inertial sensors,autonomous navigation systems are typically equipped with auxiliary navigation technologies, such as astronomical navigation2, visual navigation3,acoustic navigation4,and geophysical navigation,which operate at various frequencies and accuracies5.Although these guidance systems can correct the positioning error of the INS, problems, such as vulnerability to interference and high cost of data acquisition, remain.The existing conventional navigation technologies cannot satisfy the navigation requirements of high precision, reliability, and autonomy.

The navigation skills from organisms in nature have attracted considerable research for incorporation into novel navigation technology.Bio-inspired polarized skylight navigation is a novel passive autonomous navigation method in which the attributes of earth-polarized skylight patterns are used as compass messages.

2.Polarized skylight navigation in nature

Since the early 20th century,studies have focused on the ability of African sand ants6, locusts7, cuttlefish,8and north American monarch butterflies9to navigate by sensing the polarized skylight for foraging, homing, and migration.Dung beetles10can hold their course using polarized moonlight.As documented in birds, navigation using photoreceptor arrays is specifically sensitive to the patterns of the polarization of light in the sky.Birds,especially among long-range migratory birds11,such as swifts12,pigeons,13and Savannah sparrows14,may use polarization patterns in the sky during sunrise and sunset to calibrate their magnetic compasses.The polarized visual navigation of some special living creatures is displayed in Fig.1.6–14

Navigation research using sky polarized light information is inspired by the polarized visual navigation abilities of these creatures.Such bionic autonomous navigation has been preliminarily realized in engineering.Anatomical studies revealed that polarized vision or sensitivity to polarized light in nature requires some unique structures in the compound eye—small eyes arranged orderly in the Dorsal Rim Area (DRA).As displayed in Fig.215, the small eye sensory rod structure microvilli in the DRA of various organisms exhibit a certain selectivity for polarized light of various types and different polarization directions.This phenomenon determines that organisms exhibit selective sensitivity to environmental polarized light.

DRA sensitivity to polarized light is typically monochromatic and can mitigate interference of color visual signal on polarization visual signals for orientation and navigation purposes.The bio-polarization navigation target band of various living creatures is displayed in Fig.316,17, which reveals that many insect species use only the Ultraviolet (UV) component of the polarized skylight, whereas both the intensity and the degree of polarization of light from the clear sky are lower in the UV than at longer wavelengths.This phenomenon is called the ‘‘UV-sky-pol paradox.” Gao et al.18,19discussed the advantage for certain animals to detect celestial polarization in the UV and revealed that UV celestial cue-based navigation is efficient under all weather conditions.

Bio-inspired polarized skylight navigation is a novel autonomous navigation method based on skylight polarization characteristics and pattern.This method can be used to realize vehicle heading information by the detecting and calculating skylight polarization patterns.20Fig.4 displays the key contents of bionic polarized light navigation sensors.This paper details three aspects,namely skylight polarization pattern theory,skylight polarization pattern information acquisition technology, and polarized skylight navigation.

3.Theory of skylight polarization pattern

Establishing a theoretical model to accurately describe skylight polarization pattern characteristics is critical for realizing bioinspired polarized skylight navigation.

3.1.Skylight scattering models

3.1.1.Rayleigh scattering

Polarization was first coined by Malus in 1809, a year after Arago observed the phenomenon.In 1871, British physicist Rayleigh21quantitatively analyzed scattering skylight by gases in atmosphere to explain the polarization and proposed the inverse fourth power of wavelength law.He detailed the cause of the blueness of the sky and revealed that the primary cause of polarization is the scattering of atoms and molecules of atmospheric gases, such as N2, CO2, O3, and O2.

Fig.1 Polarized visual navigation of some special living creatures.6-14

Fig.2 Compound eye morphology of various living creatures.15

When sunlight passes through the atmosphere, the floating particles in the atmosphere scatter and absorb incident light.Rayleigh scattering occurs when the diameter of the floating particles in the atmosphere is considerably smaller than the wavelength of incident light.The skylight scattered by particles in the atmosphere is idealized into linearly polarized light and single scattering light based on Rayleigh scattering theory.As displayed in Fig.5, O-XYZ represents the East-North-Up(ENU) geographic coordinate frame.A general scattering process description of the skylight at any point P can be described based on the Rayleigh model in the ENU coordinate frame.The symmetry line running through the sun (S) and zenith (Z) is called SM on the side of the sun and ASM on the opposite side.Here, P(r,θP,φP) represents any point in the sky.The zenith angle and azimuth angle of P are described by θPand φP, respectively.The scattering angle and polarization angle of P are described by γ and α,respectively.The solar space position is represented with S(rS,θS,φS), where θS,φS,and 90?-θSdenote the zenith, azimuth, and elevation angles of the sun, respectively.

The polarization parameters of skylight are calculated by using the geometric relationship among the observation point,the position of the sun, and the coordinate origin in the horizontal coordinate system.The Angle of Polarization (AOP)α and Degree of Polarization (DOP) d can be calculated as follows:

The variation of polarized skylight distribution, characterized by the Rayleigh model, over Shandong University of Science and Technology, simulated according to Eq.(1) is displayed in Fig.6.The simulation time is from 6:00 to 12:00 at 1 h interval on 2022-06-21.The geographic coordinates and location of the sun are presented in Table 1.

Fig.5 Skylight scattering process in ENU coordinate frame.

However, the Rayleigh scattering model cannot express complex optical processes such as multiple scattering of skylight under complex weather conditions.The actual skylight polarization mode differs considerably from the theoretical mode calculated by using the Rayleigh model.This difference considerably reduces the accuracy of heading calculation.22–24To finally establish the Hannay model,Hannay25first deduced the multiple Rayleigh scattering sequence, obtained the characterization of the nth Rayleigh scattering sequence, and obtained a new characterization method of the polarization mode of sky light based on the approximate solution of the secondary Rayleigh scattering process.Powell et al.26established an underwater polarization transmission model considering the skylight refracted by the water surface and single scattering by water molecules to predict underwater polarization distribution.Chu et al.also simulated a skylight polarization distribution model under wavy water surfaces using the Stokes vector and Mueller matrix.The Cox–Munk sea wave model was used to describe the wavy water surface.27

3.1.2.Mie scattering

Complex scattering processes,such as multiple scattering,cannot be expressed using the Rayleigh scattering model.In 1908,Mie28established the meter scattering theory by proposing the exact solution of plane electromagnetic scattering in an isotropic uniform sphere.Mie scattering typically occurs when the diameter of suspended particles in the atmosphere is identical or similar to the radiation wavelength.

Compared with Rayleigh scattering(Fig.729),Mie scattering light does not satisfy the law that the vertical component is always greater than or equal to the horizontal component in Rayleigh scattering, and the vibration direction of the maximum electric vector (E-vector) of the scattered polarized light is not necessarily perpendicular to the scattering plane.Thus,the fixed geometric relationship between the vibration direction of the maximum E-vector of polarized skylight caused by Mie scattering and the position of the sun does not exist.Therefore, using the degree of polarization of skylight and the azimuth of the sun for orientation is difficult.30

Fig.6 Simulation results of polarization distribution pattern based on Rayleigh model.

Table 1 Sun location and geographical coordinates.

Fig.7 Comparison of Rayleigh scattering and Mie scattering.29

Berry et al.31supplemented the Rayleigh model and multiple scattering theory by using singularity theory to understand sunlight polarization and established the Berry model considering the multiple scattering effect on the skylight polarization state.The Berry model was established for the four neutral points, namely Arago, Brewster, Babinet, and Fourth neutral points, of the skylight polarization pattern.32Comparison with the polarization distribution pattern revealed that the atmospheric turbidity parameter was corrected to obtain a polarization pattern characterization method.Among the existing scattering models, the Berry model was the closest model to the actual distribution pattern.Liang et al.33mentioned that the Rayleigh scattering model contains only single solar vector information and cannot be used to determine the three-dimensional attitude in real time.The Berry sky model is plane symmetric but not rotationally symmetric.The model contains information other than the solar vector, which may be used for real-time three-dimensional attitude determination.However,this model cannot reasonably explain the formation mechanism of the neutral point in the skylight polarization mode.In addition to its inaccuracy and highly idealized results, the model limits the value of applied research related to the atmospheric environment and polarization modes.The Berry characterization model is displayed in Fig.8.

In the analytical model of sky polarization mode proposed by Willie,light intensity and sunny weather were considered.34Hosek and Wilkie35established a full-wave band analytical model of sky polarized light.Wang et al.36proposed an analytical model of atmospheric polarization mode by combining Dennis, Berry singularity theory, and Perez intensity.

Fig.8 Berry characterization model of AOP (red circle is the position of the neutral point, the solar meridian is the horizontal axis,and the angular distance between two adjacent neutral points is 40°).

3.1.3.Vector radiative transfer theory

Vector Radiative Transfer Equation(VRTE)is a simple equation describing the propagation and redistribution of electromagnetic waves in media and is used to study the influence of atmospheric particles,such as aerosols,on polarization patterns.Unlike Mie scattering, VRTE can be applied to the study of the correlation between skylight polarization mode distribution and physical factors.

The specific solution methods of the equation include the discrete coordinate, cumulative doubling, spherical harmonic,multi-component, and Monte Carlo methods.According to various solution methods, the skylight polarization mode description models proposed by researchers include Radiative Transfer 3 (RT3) and RT4 models37, Ray model38, Vector Linearized Discrete Ordinate Radiative Transfer (VLIDORT)model39, Multi-Variate Distribution of Optically-Active Molecules (MVDOM) model40, MYcrophysics for Stratocumulus with Implicit Treatment of Clouds (MYSTIC) model41, Pseudo-spherical Atmospheric Radiative Transfer(PSTAR) model42, and Line-by-line Radiative Transfer(LINTRAN) model.43

Table 2 Comparison of models based on Rayleigh, Mie, and VRTE scattering theory.

However, such modeling of the transmission processes requires completing the complex transmission process modeling and accurate atmospheric composition structure and dynamics.Therefore,the implementation of such models is difficult.Although inherently accurate for the parameters modeled, the process is highly computational and heavily reliant on the accuracy of the initial conditions.Limited studies have focused on bionic navigation using VRTE.

3.2.Summary and comparison of skylight polarization pattern models

The scattering process of atmospheric particles is typically analyzed by using the Rayleigh scattering,Mie scattering,and the VRTE model.Rayleigh scattering is used as an ideal polarization skylight model only under clear and cloudless conditions.Mie scattering theory and vector radiative transfer equation are polarization skylight models in which the influence of cloud scattering close to true skylight is considered.

Improved models have been proposed to model the particle scattering process in the sky.Simple analytical models were established by analyzing the distribution law of polarized skylight using the Rayleigh and Mie models.Another model was realized by analyzing the composition, structure, and changes of the atmosphere and solving VRTE.The comparison of polarization distribution model dominated by skylight is presented in Table 2.21,25–27,31,34–43.

A perfect model for analyzing skylight polarization pattern is yet to be realized.A realistic and multifactor controllable model was developed according to the environment, climate,space time, and other factors of the observation site.

4.Technology for detecting the skylight polarization pattern

The skylight polarization distribution contains considerable directional information,44such as the solar meridian, the angle of polarization, the symmetrical distribution, and the neutral point, which plays a key role in polarization navigation.Specific acquisition technology is used to acquire skylight polarization patterns.

4.1.Detection principle

To investigate the characteristics of polarized skylight, many description methods are used to characterize the direction,shape, or intensity of polarized skylight, such as Jones vector,Stokes vector,and Poincare sphere.For satisfying the application requirements of polarized light bionic navigation, the Stokes vector method is widely used to calculate the polarization state as follows:

where I represents the total light intensity, Q and U represent the light intensity of two mutually orthogonal linearly polarized lights, and V represents the difference of circularly polarized light intensity.

Mueller matrix M can be used to describe the relationship between S′=[I′,Q′,U′,V′]Tof outgoing polarized light and S of incident light as follows:

By detecting the polarization information of linear optical polarizers in various directions (α), all the parameters of the Stokes vector can be obtained and various attributes of AOP, DOP, Degree of Linear Polarization (DOLP) and Degree of Circular Polarization (DOCP) can be calculated as shown in Table 3.45

Few circular polarization components in natural light can be ignored.Therefore, when detecting the polarization distribution of skylight, only AOP and DOLP are calculated.

4.2.Detection result of polarized skylight patterns

The skylight polarization mode contains various spatiotemporal continuous distribution features under the joint influence of time, geographical location, and environmental factors.Numerous studies have been conducted on the detection of the polarized skylight patterns in various conditions.The polarizations distribution of sunlit or moonlit skies has been investigated over land or underwater.

4.2.1.Daytime sky polarization

Horva′th and Wehner46detected the change in the skylight polarization mode during sunrise and confirmed the relationship between the polarization distribution mode and solar position and observation position.Subsequently,they detected that the skylight polarization angle distribution in cloudy days is similar to that in sunny days47.Suhai and Horva′th48revealed that in the presence of most small particles in the atmosphere, the polarization distribution mode of short band skylight under sunny and cloudy weather conditions accurately follows Rayleigh scattering theory.Particularly,AOP detected under sunny sky is almost consistent with the calculation results of the Rayleigh theoretical model.

Kreuter et al.49measured the all-sky distribution of polarized radiance by using an automated fish-eye camera system with a rotating polarizer and investigated the influence on the degree of polarization and sky radiance for numerous aerosol and surface albedo situations.Aycock et al.developed the sky polarization azimuth sensing system sensor and tested and analyzed the polarization azimuth under various weather conditions.The results revealed that the accuracy of polarization azimuth in cloudy and haze weather is lower than that in sunny sky.However, the azimuth can still provide information for polarization navigation.50Zhao23and Chen et al.24revealed that both the aerosol and cloud disturbed the polarization patterns of the skylight,but the patterns of AOP revealed considerable robustness.DOLP and AOP patterns of the sky dome measured under various aerosol optical thickness or cloudy skies are displayed in Fig.9.23Ahsan et al.51proposed a color mixing model based on hue for improving the symmetry of polarization pattern and detection accuracy of polarized skylight information on sunny, cloudy, or foggy days to avoid potential problems caused by detection uncertainties, such as data collection, sky model uncertainty, sky polarization change, and instrument limitation.

Table 3 Formula of polarization parameters.

The results revealed that AOP exhibits a highly stable distribution pattern under all possible daytime conditions,whereas DOLP in complex weather conditions differs considerably from that in sunny days.However, studies have revealed numerous polarization characteristics resulting from the shielding of the sun during a total solar eclipse.52,53Fig.1052reveals that at the beginning and end of the solar eclipse,when the shielding area of sun is less than 88%,the distribution pattern of polarized skylight is identical to that during the nonsolar eclipse.The distribution pattern of polarized skylight changed and the DOLP decreased considerably during the total solar eclipse.Thus,obtaining navigation information becomes difficult.

4.2.2.Night-time sky polarization

Sunlight, moonlight, and other cosmic light comprise the night-time light source.The sunlight reflected by the moon is the main part of the night light source.Ga′l et al.measured the patterns of the degree and angle of polarization of the moonlit clear night sky at intervals of 30 min throughout the night at full moon by using full-sky imaging polarimetry.The observed patterns including the positions of the Arago and Babinet neutral points of the moonlit night sky and sunlit day sky are identical if the zenith angle of the moon is identical to that of the sun.54Dacke et al.55revealed that dung beetles can use the polarization mode of moonlight to realize navigation and positioning.

The detection of sky polarization patterns during lunar and solar eclipses is similar.Yang et al.56revealed that during the super blue blood moon night, as the obscuration ratio of the moon increased, the distribution of the DOLP gradually concentrated on a lower DOLP,the AOP was asymmetrical about the meridian, and the neutral point was not located on the lunar-anti-lunar meridian.These characteristics differed from those of the conventional neutral point.The simulation and experimental results are shown in Fig.11.56

4.2.3.Ocean sky polarization

(1) Above water

Bio-inspired polarization vision enables underwater geolocalization.Weather conditions over the ocean are typically cloudy and foggy, and the scattering of skylight is complex.Hegedu¨s et al.47performed full-sky imaging polarimetry and revealed that the pattern of α of light transmitted through the ice or water clouds of totally overcast skies is qualitatively identical to that of the AOP pattern of the clear sky on the Arctic Ocean and in Hungary.Barta et al.57obtained the cloud coverage over the ocean by detecting polarized skylight aboard the research vessel Polarstern.Guan et al.58captured the polarization patterns over the East China Sea and the Yellow Sea continuously during daytime and night-time by using a full-sky imaging polarimetry system and revealed that the skylight polarization distribution over the sea has a similar pattern as that on the land.He revealed that when the ship sailed on the sea, the direction of the real meridian was close to the solar azimuth during the daytime and close to the lunar azimuth during the night-time.The test and simulation results of skylight polarization distribution over the sea are presented in Fig.12.58

Fig.10 DOLP and AOP patterns of skylight during total solar eclipse.52

(2) Under water

The skylight polarization pattern under wavy water surfaces presents a regular distribution related to the sun position after air–water interface refraction and water scattering.26,27.

When observing the polarized sky light distribution mode from underwater, the Field of View (FOV) above the water surface is compressed into a Snell window with an aperture angle of approximately 97.5° because of the refraction effect(Fig.1359,60).

Sabbah et al.61used Stokes vector and Mueller matrix to simulate underwater polarization distribution of sky light refracted by static air–water interfaces.Powell et al.26verified the possibility of using the polarization detection instrument to realize navigation underwater by measuring the underwater polarized light in many places, different depths, and time.Cheng et al.62established the model to simulate underwater polarization patterns under various conditions by considering atmospheric polarization distribution, the refraction of the air–water interface, and the scattering of underwater particles(Fig.1462), and analyzed the influence of wavelength, water turbidity, water composition, and particle size on underwater polarization patterns.

4.3.Summary

The celestial distribution of the direction of polarization is a highly robust pattern that is qualitatively always the same under all possible sky conditions.The results proved that the polarization distribution law of the weather is similar to the Rayleigh scattering model and shares the same change rules under various weather conditions but with distinct polarization indexes.

Fig.11 DOLP and AOP measured during super blue blood moon and ideal celestial polarization patterns based on single-scattering Rayleigh model.56

Fig.12 Test and simulation results of skylight polarization distribution from 09:00 to 22:00 over the sea.

Fig.13 Snell’s window.

Fig.14 Test and simulation results of polarization patterns of AOP from 09:00 to 20:00 underwater.62

However, many studies have focused on the polarization distribution under the static water surface, whereas limited studies have been conducted on the polarization distribution under fluctuating water surfaces.Establishing atmospheric polarization models in various weather and different periods is critical for the prediction of skylight polarization pattern and solution of navigation information.

5.Polarized skylight navigation sensors design

The polarized visual organs of animals have been imitated to design various observation navigation and orientation devices.Polarized light navigation sensors are typically classified into two categories, namely non-imaging-based and imagingbased sensors according to the sampling mechanism.63Influential sensors are introduced in this subsection.

5.1.Non-imaging-based sensors

Labhart64and Meyer65introduced a prototype non-imagingbased sensor in 1988.The polarization information detection unit of the polarized light navigation sensor includes a polarizer(s), photodetector(s), and circuit processing module(s) to simulate the microvilli structure in the DRA, the photoreceptor that senses the light intensity, and the part of the optic nerve lobe that processes the light signal of the compound eye organism.The working principle of the non-imagingbased sensor unit is displayed in Fig.15.64The polarization directions of the polarizers in each unit are mutually orthogonal.Using polarized light in the appropriate wavelength range as a source of information for polarized light sensor is more advantageous than using full-band polarized light without any processing.Therefore, mounting filters should be considered when designing the non-imaging-based sensor.Although the UV band is selected by most insects as the polarizationsensitive band for their DRA, obtaining light in the UV band is difficult and more expensive.The blue band optical filter is highly suitable for polarized light detection and widely available.27

Chahl and Mizutani66developed a polarized sensor consisting of eight photodiodes and four ultraviolet/green pairs to mimic the function of dragonfly eyes.The sensor is installed on the head of the unmanned aerial vehicle, and the heading angle is determined.

Chu et al.67designed non-imaging-based bionic navigation sensors with six polarization channels by using units and an ARM microprocessor.They used two such polarized light sensors and a three-axis compass for autonomous real-time positioning.68Furthermore, they combined five such polarized light sensors to measure the polarized sky light in five directions simultaneously.69The test results revealed that position accuracies were ± 0.4° and ± 1.2° in longitude and latitude,respectively, which indicate that the navigation prototype is feasible and stable and applied to real navigations.They used such sensors to provide heading angle reference for the formation cooperation of multiple agents.The experimental data revealed that the heading angle error was within 2°.70To improve integration, the team integrated a bilayer nanowire polarizer with a photodetector by using nanoimprint lithography as a polarization navigation sensor to fabricate nanowires with multiple orientations simultaneously and eliminate the alignment error71.The angle output experiment revealed that the angle output error was within ± 0.1°.

Fig.15 Non-imaging-based sensor unit structure model.64

Hu et al.72,73designed a three-channel bionic polarization navigation sensor and calibrated the sensor by using the least square algorithm and Non-dominated Sorting Genetic Algorithm-II(NSGA-II).Yang et al.74designed a sensor consisting of three polarizing beam splitters to effectively reduce the quadrature error caused by the non-imaging-based sensor.This device can acquire six channel light intensity measurements simultaneously and thus achieve an accuracy of 0.18°.A hemispherical structure sensor consisting of nine polarization sensor units (each unit containing six photodetectors)and distributed in an array on the surface of the hemisphere was designed.75Furthermore, they designed a tri-channel polarization sensor with high spectral adaptability (400–760 nm) for underwater orientation.76.

Yang et al.77from Northwestern Polytechnical University assembled two non-imaging-based sensors with each sensor containing four six polarization channels, for measuring solar vector by observing different directions.The accuracy of the solar azimuth angle was 0?2?(1σ),and the accuracy of solar elevation angle was approximately 0?4?(1σ).Inspired by ants,Dupeyroux et al.developed two-pixel celestial non-imagingbased compass in the UV range.In this compass, polarizers were placed on rotating gears actuated by a stepper motor.6The mechanical rotation eliminated excessive detection units,and the occurrence of heading angle errors ranged from 0.3°to 2.9°.

The development of non-imaging-based bionic polarization navigation sensor using photodetector array is displayed in Fig.16.6,66,68,69,71–77

5.2.Imaging-based sensor

With the continuous development of semiconductor technology, novel imaging photosensitive devices, such as Charged Coupled Devices (CCDs) and Complementary Metal-Oxide Semiconductor (CMOS), are maturing.Imaging photosensitive device based on MEMS technology increases the number of photosensitive pixels per unit area by hundreds of thousands of times, and its data stability is superior to that of the conventional photodetector array.Imaging-based sensors are composed of len(s), polarizer(s), filter(s), and image sensor(s),as displayed in Fig.17.In the sensor, skylight polarization information is calculated from multiple angles.

(1) Time-sequence-based sensors

Time-sequence-based sensors are a type of the imagingbased sensor in which polarization images in various directions are obtained by rotating a single polarizer.Wolff78designed a monocular time-sequence polarizing camera with a fish-eye lens, linear polarizer, and CCD camera, and measured the all-sky polarization image for the first time by manually rotating the direction of linear polarizer.This device is widely used in polarization bionic navigation to conduct experimental research.However, this method is not suitable in navigation sensors for motion vehicles because of the poor real-time performance and large size.

(2) Multi-channel-based sensors

Multi-channel-based sensors can simultaneously obtain polarization images in different directions.Wang et al.79developed a real-time three-channel camera-based polarized skylight sensor in a triangular layout to obtain the threechannel signal synchronously.The sensor was less susceptible to interference by the external environment.The orientation accuracy obtained in the vehicle navigation experiment can reach 0?32?.Fan et al.80used a four-channel imaging polarized skylight to detect polarized light in the whole sky and suppressed maximum orientation error to 0?5?in clear and cloudless weather.Sturzl81proposed a compact imaging-based sensor composed of three synchronous cameras that could be integrated into micro aircraft for determining azimuth in the presence of clouds.Zhang et al.82introduced a novel multichannel sensor to acquire the polarization pattern across the full sky with a single camera and without a rotating polarizer.The main lens of camera was mounted with a linear polarizer triplet which divided the subimage beneath each microlens into three areas to calculate the DOP and AOP in a single shot.

Fig.16 Development of non-imaging-based navigation sensor.

Fig.17 Image-based bionic polarization navigation sensor model.

(3) Sensors based on division of focal plane

Polarized skylight sensors with split focal plane have been widely used to develop image-based polarization navigation sensors.Sarkar et al.83directly integrated the polarizer on the CMOS photosensitive array through micro nano processing technology and preliminarily realized the integrated integration of polarized light sensor for detecting the incoming polarized skylight ray direction.Similarly, Han et al.84integrated four-channel focal planes on the CMOS image sensor that could receive linearly polarized light in four directions(0?, 45?, 90?, 135?) simultaneously and calculate orientation according to the sky polarization field map.The standard deviation of orientation error at sunset was 0?15?.Chu et al.85constructed a compact real-time polarization navigation sensor in which the nanowire polarizer was mounted on the photosensitive area of the CMOS.The sensor was composed of four bilayer nanogratings with orientations of 0?, 60?, 90?, and 150?, respectively.Hu et al.60assembled an underwater imaging polarimeter system as the underwater navigation sensor composed of a fish-eye lens, an imaging polarimeter, and a level platform with a seal cavity.The root-mean-square errors of solar zenith and azimuth using this algorithm were 0?3?and 1?3?respectively.

The development of image-based bionic polarization navigation sensors using a photodetector array is displayed in Fig.18.60,79–85

5.3.Summary and Comparison of sensors

Non-image-based navigation sensors are composed of numerous discrete photosensitive elements.The sensor has a simple structure, few data units, and a simple data processing program.The response speed and instantaneous accuracy can reach a high level.If the weather is good, the heading angle accuracy calculated by non-image-based navigation sensors can reach ± 0.1° in dynamic outdoor environments.The indoor accuracy can even reach 0.009°.However, nonimaging-based sensors are easily affected by inclement weather because of the small amount of data obtained, which reduces the output accuracy of the sensor under nonclear conditions.Linear polarizers spliced in blocks affect detection accuracy,and the sleeve structure is not conducive to eliminating stray light.

Imaging-based sensors exhibit a superior anti-jamming capability to non-imaging-based sensors.Imaging-based sensors can detect considerable polarized skylight in a region with different polarization directions.The heading accuracy calculated by the imaging sensors is lower than that calculated by the non-imaging sensors.In clear sky, the sensor can reach a dynamic outdoor accuracy of 0.5°.As for cloudy conditions,the accuracy will be affected.However,the cost of such sensors is high because of the requirement of image processing chips.In particular,the use of a polarization CCD camera is not conducive to the miniaturization and cost minimization of the navigation sensor.Precise and smaller bio-inspired polarization skylight sensors for navigation are the focus of polarization navigation research.

The comparison between non-imaging-based and imagingbased sensor is presented in Table 4.6,60,64–85

Fig.18 Development of image-based navigation sensor.

Numerous correction methods have been devised for minimizing generated errors such as polarizer installation error84,extinction ratio error86, Muller matrix error87, and optical geometry error88,89.Error measurement models considering multi-source factors have been proposed.90,91Ref.91 presented a multi-source measurement error model of the imaging sensor based on the Stokes vector.The error model can be used as a guideline for calibrating the coordinate for the deviation of the principal point, the installation angle error of micro-polarization array, the attenuation and distortion of the lens, and the grayscale response inconsistency of CMOS.A specific calibration method for these errors was based on geometric parameters and the Mueller matrix of the optical system.The outdoor dynamic result indicated that the Root Mean Squared Error (RMSE) of the heading angle output by the calibrated sensor can be reduced by half,which has considerable engineering significance.

In an unstable environment, the carrier cannot be always horizontal.Thus, the bionic polarization sensor cannot be aligned with the zenith, which considerably affects the measurement accuracy of the polarized skylight pattern and orientation of the skylight.92Zhi et al.93proposed the use of Global Positioning System (GPS) data to correct the attitude measurement error using the five-channel non-imaging-based sensor tilt.Gkanias et al.94introduced a novel method to correct sensor array tilting caused by travel over uneven terrains.Han et al.95proposed a method to improve the orientation accuracy of the imaging-based sensor under tilt conditions.

With the development of photoelectric detection technology, novel sensors have been designed.For example, Chu et al.designed two novel skylight orientation sensors with various structures.The first sensor consists of a microlens array,an aperture diaphragm, and a field diaphragm.96In the other sensor,a lightwave modulator consisting of filter,S-waveplate,and linear polarizer is used.97The results of the study can be used to develop polarized light navigation sensors with high integration, precision, and strong robustness.The two novel bio-inspired polarization navigation sensors are shown in Fig.19.96,97

6.Polarized skylight navigation system

6.1.Orientation determination approaches

6.1.1.Based on cross product of any two E-vectors

According to Rayleigh scattering theory,the E-vector of polarized skylight at arbitrary point in the sky is perpendicular to the sun direction vector so that the angle between the sun direction and the vehicle axis can be calculated, and then the heading angle by consulting the astronomical calendar is obtained.In this paper, the method is called the E-vector.

The non-imaging-based polarization navigation sensor can measure the E-vector of several points in the sky.Chu et al.67–69calculated solar vector using the cross product of any two-directional or multidirectional polarized-light vec-tors measured by non-imaging-based polarization navigation sensor.The error of heading angle in sunny weather is 0.2°.

Table 4 Comparison between non-imaging-based and imaging-based sensors.

The imaging-based polarization navigation sensor can obtain more the E-vector distribution of part of the sky or the whole sky.Wang et al.98reported the first successful implementation of the polarized light compass in foliage environment with heavy occlusions.A classifier based on the primal Support Vector Machine (SVM) is trained to detect the sky area, where the measurements are used to calculate the solar meridian by optimal estimation (RMSE = 1.58°).The result of estimating SM-ASM based on SVM is displayed in Fig.20.98Fan et al.99proposed a real-time method that uses the gradient of the degree of the polarization to remove the obstacles and select E-vectors for cross multiplication in the effective polarized skylight image.The RMSE in the dynamic experiment is 0.81°.

6.1.2.Based on extracting SM-ASM

Researchers have determined the heading angle by identifying the SM-ASM based on the symmetry of the skylight polarization distribution pattern.According to Section 3.2,we can see that the SM-ASM is consistent with the symmetric axis of DOLP image or the antisymmetric axis of AOP.This method is called SM-ASM in this paper which is the mainstream polarization orientation algorithm.Extracting or fitting SM-ASM from polarization image is the mainstream polarization navigation algorithm.

Lu et al.100designed an algorithm to solve the solar azimuth, which contains two parts: threshold extraction and 1D Hough Transform (HT), and a simple mean filter was introduced into the parameter space to overcome the ‘‘inherent defect” of the HT for obtaining the solar meridian and highprecision orientation (less than 0.37° in experimental test).Then the nonlinear operation is simplified to linear operation to reduce the difficulty of algorithm operation.101The algorithm flow is shown in Fig.21100.Zhang et al.102roughly determined the symmetrical axis of redrawn AOP images and selected several alternative lines beside the initial one with an interval of 1° as multiple symmetry axes.Each axis will be tested to determine whether it is an ideal symmetry axis.The testing method involves subtracting the two halves of a circular image divided by the axis, and the best symmetry axis is the one with the smallest average gray value obtained.The images after subtracting by two axes are shown in Fig.22102.According to the change of gray level in the generated image Fig.22(c), the appropriate axis of symmetry is selected as the solar meridian, and the static orientation accuracy under cloudy weather conditions is better than 0.2°.Guan et al.103proposed a method based on polar coordinate transformation to convert polarization images into the polar coordinate form to calculate not only the solar meridian but also the relative rotation angles of the solar azimuth,and outdoor static experimental precision is within 0?63?while dynamic experimental precision is about-0.7°–0.5°.Schematic diagram of the same pixel matching for two consecutive single pixel rings is shown in Fig.23103.

Artificial intelligence algorithms, such as SVM and Neural Network (NN), can independently learn the potential laws of things and fit complex nonlinear functions within a small error range.Therefore, it has a certain application in mining the relationship between the polarization distribution characteristics of skylight and the position of the sun.

Wang et al.104proposed a model for calculating the direction of the sun based on Polarization-analyzing Artificial Neural Network (POL-ANN), which has high efficiency.The artificial neural network model is shown in Fig.24104.However,it is not robust in practical application because the training set of model is composed of many simulation images.By image segmentation, connected component detection and inversion, Liang et al.105transformed the heading determination problem into a binary classification problem and the binary classification problem is solved by the soft-margin SVM for fitting SM-ASM.The local classifiable AOP images and the enlarged drawing of it are shown in Fig.25105.The polarization angle distribution of the polarized skylight pattern presents a stable ‘‘∞” characteristic, which reflects the macroscopic distribution characteristics and changing rules of the polarized skylight pattern.Liu et al.106proposed an improved harmony search algorithm based on this characteristic to determine the position of the sun: by searching in the Rayleigh ∞characteristic image database, the solution vector corresponding to the Rayleigh ∞characteristic image with the highest feature similarity index is the solar position.Even in cloudy weather,the average error of the sun azimuth is within 0.8°.

Fig.19 Two novel bio-inspired polarization navigation sensors.

Fig.20 Result of estimating SM-ASM based on SVM.98

Fig.21 Compass calculation algorithm based on HT.100

Fig.22 Compass calculation algorithm based on HT.102

There are some studies on the presence of occlusion in the polarized image.As shown in Fig.26, Tang et al.107proposed a method to filter the destroyed AOP map judged by DOLP map using Pulse Coupled Neural Network(PCNN)algorithm,which only retains the effective region in the corresponding AOP and fits the solar meridian for orientation through the least square method, as shown in Fig.26.When there are few clouds,trees and buildings,the compass accuracy is better than 1°.Wan et al.108proposed a robust Gradient Amplitude and Binary Integration(GABI)azimuth method for a polarization imaging sensor to fit the solar meridian, and when branches and buildings shield more than 80%of the AOP map,the azimuth measurement method remains effective.The results are shown in Fig.27.108

In order to obtain actual navigation direction after identifying the solar meridian and determining the included angle between the line and the axis of the vehicle,it is also necessary to calculate the included angle between the direction of the solar meridian and a specific direction (such as due north)through the astronomical calendar, so as to obtain the actual navigation direction.There are mature algorithms for calculating the reference direction through the astronomical calendar.109If the field of view angle taken is too small,there may be no neutral point or solar meridian in the obtained sky polarization field map, and it will be difficult to further determine the heading angle by SM-ASM method.79

Fig.23 Schematic diagram of the same pixel matching for two consecutive single pixel rings.103

Of course,there are many other ingenious orientation algorithms,such as using DOLP pattern to measure the sun elevation angle so that the absolute position of users can be deduced.110Efficient navigation information extraction algorithm is also one of the research focuses in the future.The research orientation algorithm working in the polarized skylight sensor with good robustness,strong adaptability and high precision can provide reliable navigation information for the bionic navigation system.

6.2.Integrated navigation system

By analyzing rotational symmetry and plane symmetry of the Rayleigh model, Liang theoretically proved that the model only contains single solar vector information.Based on the Rayleigh sky model, the polarization navigation sensor can only detect the angle in the two-dimensional plane,and cannot independently determine the three-dimensional attitude of the vehicle and complete the navigation task.33In other words,the bio-inspired polarization skylight navigation system can only provide reference direction and be used for twodimensional navigation and orientation of ships or ground vehicles.

In recent years, for the modern navigation application of long-distance and complex scenes, the polarized light navigation method has received close attention from many research institutions.The combination of polarized skylight information and classical navigation system, such as INS, GNSS,and SLAM, makes full use of the advantage that polarized skylight navigation error does not accumulate with time.It can correct the zero bias and other errors of other navigation sensor.Generally, the classical Kalman Filter (KF) which sets the position error, velocity error, attitude error, gyro bias or accelerometer bias as state variables, and the polarization angle provided by the polarized light sensor as the observations are used to realize the fusion of two kinds of navigation information.

6.2.1.INS/PSNS integrated navigation

The most common integrated navigation system is the combination of Polarized Skylight Navigation System (PSNS) and inertial sensor, as displayed in Table 5.111–116

Fig.24 Flowchart of information polarization based on artificial neural networks.104

Fig.25 Local classifiable AOP image and its enlarged drawing.105

Li et al.111proposed a 3D attitude calculation method, in which E-vectors detected by non-imaging-based sensor and the one-dimensional component with the minimum motion acceleration from the output of the three-dimensional accelerometer are used.The proposed method considerably reduces the influence of motion acceleration on attitude measurement.Du et al.112developed the INS/PSNS integrated navigation error equations to achieve autonomous and fast initial alignment, and designed KF to estimate the unknown state based on the measurements from the inertial measurement unit and PSNS.The CKF-MRC with a multi-rate residual correction is put forward by Zhao et al.113to integrate the INS and polarized skylight navigation data output with different sampling frequencies respectively, and a Long Short-Term Memory(LSTM)was trained for polarization navigation output prediction and fusion in the absence of actual polarized skylight data.Yang et al.114designed the CKF model to fuse the heading angle measured by the PSNS and INS which improved the accuracy of calculating the heading angle of the polarized light/inertial integrated system in nonhorizontal road environment.Zhang et al.115established a INS/PSNS/ Celestial Navigation System (CNS) integration navigation model using KF which has a better correction capability of the attitude and heading error.The state model of KF is established according to the error model of SINS and the measurement model is based on the difference between the attitude angle calculated by the sun vector and the star vector and the attitude angle calculated by the SINS.Dou et al.116proposed adaptive anti-disturbance navigation method for SINS/PSNS/ Odometer.They introduced the AOP error into the existing SINS/PSNS models to improve the accuracy of the system models and an innovation-based UKF to update the polarization skylight measurement.The system information fusion architecture is illustrated in Fig.28.116

6.2.2.GNSS/PSNS integrated navigation

PSNS performs integrated navigation with global navigation satellite system, as presented in Table 6.117–119.

He et al.117proposed a MIMU/polarized camera/GNSS integrated algorithm based on Kalman filter.The system error equations of KF were strapdown INS error equations and the observation vectors included GNSS position,velocity and yaw angle from polarized light orientation.Shen et al.118improved MR-STSCKF to solve the problem caused by different sampling frequencies while MEMS-INS, GPS and polarization compass were integrated together to improve the accuracy of the integrated navigation system with a high sampling frequency,as shown in Fig.29118.Cao et al.119proposed a Federal Unscented Kalman Filter (FUKF) with reset-free structure for the in-flight alignment problem of the integrated SINS/GPS/polarization/geomagnetic navigation system.In the master filter,attitude angles and gyro drift of geomagnetic and polarization subsystems are estimated to improve the filtering accuracy with low computational burden.

6.2.3.SLAM/PSNS integrated navigation

SLAM has also shown the capability of navigating autonomous robots in global-navigation-satellite-system-denied environments.The application of integrated navigation with PSNS is the current research focus, as shown in Table 7.120–124

Fan et al.98,120have presented a novel multi-sensor navigation system for the urban ground vehicle which is composed of three kinds of sensors, polarized skylight sensor, MIMU and monocular camera and can realize a high-precision position in the motion.The architecture of the multi-sensor navigation system is shown in Fig.30.120.

Du et al.121designed a six-channel polarized skylight sensor imitating desert ant ommatidium to improve the accuracy of SLAM.They adopted EKF to fuse the polarized skylight sensor,Light Detection and Ranging(LIDAR),and odometry to estimate the position, orientation, and map.The algorithm flowchart of the designed EKF–SLAM with the polarized skylight sensor which can reduce the error of positioning and mapping error and improve the accuracy of orientation is shown in Fig.31121.They also proposed a loosely coupled INS/Lidar odometry/PSNS integration based on factor graph optimization for sparse scenes in another work.122The polar-

Fig.26 Process of SM-ASM estimation based on PCNN.107

Fig.27 Process and result of SM-ASM estimation based on GABI.108ized camera was introduced to obtain the absolute yaw angle to compensate for the accumulated drift error from the INS/Lidar odometry integration in the spare scene.Li et al.123designed a tightly coupled navigation system using a polarization sensor to provide absolute heading constraints for SLAM.The system gave good reliability and robustness in environments with complex interferences.By means of graph optimization modeling, Xia et al.124described a methodology to fuse global heading data calculated by polarized light with visual and inertial information.The heading, as the absolute orientation reference, is estimated by the Berry polarization model and continuously updated in a graph structure.

Table 5 Solutions for INS/PSNS integrated navigation.

6.3.Summary

The heading data calculated by polarized skylight sensors could usefully complement measurements provided by the existing classical navigation system.The highly integrated polarized skylight navigation system will become hotspot in the autonomous navigation field and provide a new direction for exploring navigation technology.

Research scholars or engineers can fully learn from the above mature research results to further improve the performance of bionic polarized light navigation in complex environments.PSNS and its integrated navigation system will be able to operate in the near future in many contexts such as cluttered and confined environments including harbor, urban canyons,forests and ocean.

Fig.28 SINS/PSNS/ Odometer integrated model.

Table 6 Solutions for GNSS/PSNS integrated navigation.

7.Conclusions and outlook

Polarization navigation technology for biomimetic robots is a hotspot in navigation and bionic fields, which is in the ascendant and research stage.The key to bionic polarized skylight navigation is to build an accurate skylight polarization pattern model under complex weather conditions and an effective physical model of polarized skylight navigation sensor, and establish an angle calculation model with random characteristics so that the high-precision heading information can be obtained in real time from the polarized skylight mode.At present, the orientation precision of the bionic polarization navigation sensor can be less than 0.5 ° without drift.It can assist the classical navigation system to improve the accuracy of positioning and attitude estimation significantly.It is in line with the scientific concept of pursuing a new type of navigation and positioning technology that is fully autonomous, integrated and real-time.However, there are presently few examples of practical applications for polarization navigation technology in actual projects.The main reason is its weak robustness under cloudy weather conditions or in underwater complex environment.

Looking forward, we can see that bio-inspired polarized skylight navigation technology still needs more breakthroughs in the following three aspects: adaptive prediction model of skylight polarization pattern, effective detection of polarized skylight, design of polarized skylight navigation sensors, and 3D attitude measurement as well as its combined navigation system.

(1) Although researchers have developed many prediction models close to the actual skylight polarization distribution, none of them, except the Rayleigh model, can be used to guide the extraction of navigation information.People still need to investigate the correlation between the polarization pattern map in various complex weather conditions and the navigation sensor.For the ongoing effort, attempts are needed to build strong adaptivity models for different weather and time periods for guiding the calculation of polarization information and navigation data.

Fig.29 SINS/PSNS/GPS integrated model.118

Table 7 Solutions for visual/PSNS integrated navigation.

(2) The mechanism of polarization navigation in animals is not yet fully understood by humans.Further in-depth research on the sensory mechanism of animal organs,sensor structure design, and sensor processing process is needed.A variety of different approaches should be tried to duplicate the functions of natural eyes and develop bioinspired polarized navigation sensor.

Furthermore, the optical devices are becoming more and more sensitive and the color depth supported by the device will become higher and higher, which can receive polarized light information with higher accuracy.Advances in optical detection technology will no doubt advance bionic polarization navigation application.In the future, continuous attempts can be made to improve the performance of the sensor using new optical devices.

(3) In addition to the demand for effective polarization distribution models and reliable data sources,efficient navigation information extraction algorithms are also one of the research priorities in polarization information processing.The study of robust, adaptive and highly accurate bionic polarization orientation algorithms can provide reliable navigation information for bionic navigation systems.

The future polarization navigation system could function well in various environments in the sky, land, and sea, as well as in various weather conditions such as sunny, rainy, and foggy days.It will be combined with various navigation and directional devices to play an important role in practical engineering.

Fig.30 MIMU/PSNS/camera integrated model.120

Fig.31 PSNS/SLAM integrated model.121

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This study was co-supported by the Natural Science Foundation of Shandong Province, China (No.ZR2022MF315) and the National Natural Science Foundation of China (Nos.61471224 and 61801270).