Orbital angular momentum (OAM) of an optical vortex beam, arising from the helical phase e^imφ of its spatial wavefunction [1], has served as an extra degree of freedom for information coding [2, 3]. In principle, different OAM modes are orthogonal and the OAM number m can be arbitrary integers, which results in great potential in optical communication (see [4, 5] and references therein). For instance, both the information capacity and security of communication links can be enhanced [6, 7]. For free-space communication links, nonetheless, the OAM states may be affected by atmospheric turbulence, which causes crosstalks and reduces the information capacity as first reported in the case of Kolmogorov model [8]. The turbulence may not obey the Kolmogorov statistical properties in portions of the troposphere and stratosphere [9, 10]. In some cases such as slant channels [11–13], the consideration of non-Kolmogorov effect [14–18] for the atmospheric turbulence is indispensable since the refractive index structure parameter varies with altitude.

On the other hand, different vortex wave functions can be found when solving the paraxial Helmholtz equation in different coordinate systems. For instance, a novel family of vortex beams, called hypergeometric (HyG) beams, has been reported [19]. The HyG beams have several properties similar to the ideal Bessel beams, carrying divergent power as well. To achieve a more realistic version, the hypergeometric-Gaussian (HyGG) beams [20] as well as its improved version, the hypergeometric-Gaussian-II (HyGG-II) beams [21], have been put forward. The HyGG-II beams have better properties including higher focusing ability and lower beam divergence. The influences of non-Kolmogorov atmospheric turbulence on the spiral spectrum of HyGG beams have been investigated by us [22]. However, to the best of our knowledge, the investigation of an HyGG-II vortex beam propagation in non-Kolmogorov turbulence is still lacking. Recently, it has been shown that the low-order aberrations, related to the Zernike polynomials of small radial indexes, play crucial roles in the impact mechanism of atmospheric turbulence on OAM beams [23]. In this paper, we consider the propagation of HyGG-II modes via non-Kolmogorov atmospheric turbulence and explore the distribution of quantum states in the slant channel case. The influences of low-order aberrations including tilt, coma, astigmatism, and defocus types on the OAM beams are also explored.

The HyGG-II modes are a family of solutions for the scalar Helmholtz paraxial wave equation, the wave function of which in the cylindrical coordinates (r, φ, z) is given by [21]where ${C}_{q,m}=\sqrt{\tfrac{{2}^{q+| m| +1}}{\pi {\rm{\Gamma }}(q+| m| +1)}}\tfrac{{\rm{\Gamma }}(q/2+| m| +1)}{{\rm{\Gamma }}(| m| +1)}$ is the normalization factor with m the OAM number, q ≥ − ∣m∣ the holloareess parameter and Γ(·) the Gamma function, ₁F₁(a; b; c) is the Kummer confluent hypergeometric function, and ${z}_{R}=\pi {\omega }_{0}^{2}/\lambda $ is the beam Rayleigh range with λ the beam wavelength and ω₀ the beam waist. For all q and m, the HyGG-II modes form a non-orthogonal set as ${\int }_{0}^{\infty }{\int }_{0}^{2\pi }{u}_{q,m}(r,\phi ,z)$ ${u}_{{q}^{{\prime} },{m}^{{\prime} }}^{* }(r,\phi ,z)r{\rm{d}}r{\rm{d}}\phi $ = ${\delta }_{m,{m}^{{\prime} }}\tfrac{{\rm{\Gamma }}(q/2+{q}^{{\prime} }/2+| m| +1)}{\sqrt{{\rm{\Gamma }}(q+| m| +1){\rm{\Gamma }}({q}^{{\prime} }+| m| +1)}},$ where ${\delta }_{m,{m}^{{\prime} }}$ is the Kronecker delta function. It is clear, for specific q, different HyGG-II modes are still orthonormal, i.e. ${\int }_{0}^{\infty }{\int }_{0}^{2\pi }{u}_{q,m}(r,\phi ,z){u}_{q,{m}^{{\prime} }}^{* }(r,\phi ,z)r{\rm{d}}r{\rm{d}}\phi ={\delta }_{m,{m}^{{\prime} }}$. For later use, we will divide the wave function into ${u}_{q,m}(r,\phi ,z)={R}_{q,m}(r,z)\exp ({\rm{i}}m\phi )/\sqrt{2}$, where R_q,m(r, z) is the radial part. The wave function at the source plane can be simply obtained by setting z = 0 in equation (1) without singularity in the Kummer confluent hypergeometric function, which is a merit of HyGG-II beams. Besides, the HyGG-II modes have very low diffraction, i.e. the diffraction at the waist is the lowest among those realistic OAM modes carrying finite power [21]. It is worth mentioning that certain sub-family beams (termed as ‘modified Bessel-Gaussian modes of type-II’, ‘elegant Laguerre–Gaussian’, and ‘modified polynomial Bessel-Gaussian’) can be deduced by controlling the hollowness parameter q. Figure 1 gives some instances of the transverse intensity and the phase distributions at the source plane for different hollowness parameters q = 0, 0.5, and 5, which display the modulation ability of the hollowness parameter q. The type for the fractional q = 0.5 beam appears as a mixture of an elegant Laguerre–Gaussian (LG) type (even q) and modified polynomial Bessel-Gaussian (BG) type (odd q). In what follows, we will discuss the specific case q = 0.5 and give an explanation of why we choose it.

After introducing the OAM source, we start to present the communication scheme, i.e. HyGG-II beams, which is launched from the ground by a sender, propagate non-horizontally via the free space, and arrive at a detector located at a mountain of altitude h above the ground. The zenith angle of the communication channel is θ. The detector may consist of a digital micromirror device (DMD), lens and charge-coupled-device (CCD) camera to give the spectrum of the received modes [24]. Given an initial OAM mode m₀, the detection probability of the OAM mode m due to turbulence can be expressed as [8, 25, 26]where D is the diameter of the detector, andwith δm = m − m₀ and ${ \mathcal D }(r,\delta \phi )$ denoting the phase structure function of the aberrations. The non-Kolmogorov spectrum can be generally expressed as [14]where $A(\alpha )={\rm{\Gamma }}(\alpha -1)\cos (\alpha \pi /2)/(4{\pi }^{2})$ with α the power-law exponent (or spectral index), κ is the spatial wave number, and the generalized refractive-index structure parameter ${\tilde{C}}_{n}^{2}(\alpha ,z)$ (in units of m^3−α) can be expressed by [27]Here κ₀ is a fixed spatial wave number (in units of m⁻¹) related to the outer scale of turbulence as κ₀ = 1/L₀, and ${C}_{n}^{2}(z)$ (in units of m^−2/3) describes the refractive index structure parameter, related to the strength of the aberrations. For a slant channel, the refractive index structure parameter ${C}_{n}^{2}(z)$ is dependent on the altitude $h=z\cos \theta $. Here, we adopt the commonly used Hufnagel–Valley model given by [11]where v = 21 m s⁻¹ is the root-means-square (rms) wind speed and C₀ is a nominal value of the refractive index structure parameter at the ground. We also would like to explore the low-order effects contained in the total phase aberration S which can be expanded by Zernike polynomials as S = S₀ + S_tilt + S_def + S_asti + S_coma + …, where S_x (with x = 0, tilt, def, asti and coma) denote piston, tilt, defocus, astigmatism and coma aberrations, respectively. This view has been proved crucial to understanding modal crosstalk (the OAM exchanges between the beam and the aberrations) [23]. The Zernike-coefficient variances have been derived as [28]where ${\tilde{r}}_{0}$ is a Fried-like parameter that reduces to the Fried parameter r₀ for the case of α = 11/3. Besides, n ≥ 1 is the radial index of Zernike polynomials and we have n = 1, 2, 2 and 3 for the tilt (j = 2, 3), defocus (j = 4), astigmatism (j = 5, 6) and coma (j = 7,8) cases, respectively. The phase structure function as well as its tilt aberration, defocus, astigmatism aberration, and coma components can be derived as follows [12]where ${\tilde{\varrho }}_{0}$ is the spatial coherence radius given by [16]It can be easily checked that ${\tilde{\varrho }}_{0}$ is actually in the unit of length (m). The Fried-like parameter ${\tilde{r}}_{0}$ in equation (7) can be expressed byThe Kolmogorov turbulence is a special case with α = 11/3, in terms of which one recovers ${\tilde{r}}_{0}=2.1{\tilde{\varrho }}_{0}$ and ${\tilde{r}}_{0}={r}_{0}$. Hereto, the OAM detection probabilities can then be numerically calculated according to equation (2). It is worth mentioning that, the non-Kolmogorov structure constant model [17, 18] considered in [13] for the slant channel might have the drawback of dependence on optical propagation parameters such as wavelength and optical path length. To let the non-Kolmogorov structure constant model depend on the parameters of turbulence itself, an alternative approach for the non-Kolmogorov structure constant has been put forward and been experimentally confirmed [27]. In this work, we employ the non-Kolmogorov constant suggested in [27], which makes the physical mechanism of our channel different from that of [13].

In this section, we explore the turbulent effects on the channel capacity of a quantum communication scheme. According to the principle of quantum mechanics, the ensemble of sending a series of pure quantum states ρ_i with probabilities p_i can be characterized by a mixed quantum state ρ = ∑_ip_iρ_i. Here ${\rho }_{i}={\sum }_{m,n}{c}_{m}{c}_{n}^{* }\left|m\right\rangle \left\langle n\right|$ with $\left|m\right\rangle $ the OAM eigenstates and c_m the probability amplitudes. To evaluate the transfer quality of the turbulent quantum channel (since we consider the case of sending a series of quantum states which are superposition of OAM eigenstates), one needs to use the quantum generalization of the Shannon capacity according to the Holevo-Schumacher-Westmoreland theorem [29, 30]where the turbulent channel operator [31] $M\,={\sum }_{m,n}\sqrt{P(m| n)}\left|m\right\rangle \left\langle n\right|$ is constructed via the transition probabilities P(m∣n) according to equation (2). It shall be noted that the Holevo channel capacity is defined based on von Neumann entropy which is appropriate to sending quantum states while the Shannon channel capacity as consider in [8] is defined based on Shannon entropy and thus appropriate to classical states (such as an ensemble of eigenmodes). As an example, we consider a quantum communication scheme of transmitting three quantum states $(| 1\rangle +| -1\rangle )/\sqrt{2}$, $(-| 1\rangle +| 0\rangle +| -1\rangle )/\sqrt{3}$ and $(| 1\rangle +2| 0\rangle -| -1\rangle )/\sqrt{6}$, which is the same as the Case 3 of [32]). The optimization according to equation (11) for sending these orthogonal quantum states is simply achieved by uniform probabilities with p_i = 1/N with N the state number, in terms of which the initial capacity reaches its maximal value ${C}_{0}={\mathrm{log}}_{2}3$ here. Unless mentioned otherwise, the parameters for numerical simulations are set as follows: q = 0.5, λ = 1550 nm, ω₀ = 0.05 m, θ = π/3, α = 10/3, C₀ = 10⁻¹⁵ m^−2/3, L₀ = 10 m, D = 1 m and z = 3 km.

In figure 2(a), we display the Holevo channel capacity C as a function of the hollowness parameter q at different distances. For comparison, the results for distributing LG (radial index p = 0) quantum states of the same OAM modes are denoted as the horizontal lines. We clearly observe the modulation ability of the hollowness parameter q by which the channel capacity of HyGG-II modes can outperform that of LG modes with the same OAM number. Roughly speaking, larger q is beneficial to the capacity of a short channel as illustrated by the case of z = 500 m while small ∣q∣ is beneficial to the capacity of a much longer channel as illustrated by the case of z = 3000 m. When q is approaching the threshold −∣m∣ (here m = 1), the capacity displays a sharp drop for both cases. The reason for choosing the specific value q = 0.5 in this work is that the capacity of HyGG-II modes can outperform that of LG modes in both cases. In figure 2(b), we give an illustration of the the signal detection probabilities P(m₀∣m₀) (m₀ = 1) according to equation (2) under the same turbulence as functions of z, which further confirm the more robustness of HyGG-II beams against turbulence at q = 0.5. As we have demonstrated in figure 1, the case of fractional q = 0.5 can be viewed as a mixture of elegant LG and modified polynomial BG beams, both of which have been reported to have advantage over the LG one under the same turbulence conditions. Figure 3 displays the Holevo channel capacity C as a function of the rescaled propagation distance z/z_R. It is seen that the capacity is severely affected by the total turbulence aberration. For the low-order aberrations of tilt, astigmatism and coma, the Holevo channel capacities decay in a much slower manner. Among the three low-order aberrations, tilt aberration plays the most important role in the low-order contributions. Its decay trend is the most similar to that of total decay. The astigmatism aberration can be safely ignored all the time and the coma aberration contributes slightly although it is non-ignorable at a long distance. Equivalently, we will show that the coma aberration may also be non-ignorable such as increasing the turbulence strength or the zenith angle of the channel. Our result agrees well with that of [23], i.e. lower-order terms will dominate overall and are responsible for the lower-order crosstalks while higher-order terms are responsible for higher-order crosstalks.

In figure 4, the influence of the ground refractive index structure parameter ${C}_{n}^{2}(0)$ is plotted at the long distance z = 3000 m. It can be seen that the decay trend of the total aberration resembles that of the tilt aberration only for the weak turbulence ${C}_{n}^{2}(0)\lt {10}^{-15}\,{{\rm{m}}}^{-2/3}$, which convinces ourselves that the dynamics are dominated by the low order aberrations. As ${C}_{n}^{2}(0)$ increases from 10⁻¹⁴ to 10⁻¹³ m^−2/3 (strong turbulence regime), the decay of the capacity under the tilt or the coma aberration appears to be faster and faster. The coma aberration also causes explicit drops in capacity, which means it plays a more significant role as the turbulence becomes stronger.

To explore the effect of the zenith angle θ of the slant communication channel, figure 5 shows the Holevo channel capacity C as a function of the zenith angle θ. It is found that the capacity is more sensitive to the larger zenith angle θ. As θ is close to π/2 (the near horizontal channel case), the capacities for the total, tilt and comma aberrations all drop rapidly. This is due to the fact that the turbulence strength near the ground (with the parameters of this work, the condition is h < 5908.35 m) is always weaker at a higher altitude h, as can be verified by equation (6). For a specific value of the channel length z, the channel with a smaller zenith angle definitely reaches a higher altitude with weaker turbulence due to $h=z\cos \theta $. As a consequence, the beams will experience fewer aberrations by the weaker turbulence. Therefore, near the ground, the capacity is more robust for a vertical channel than that for a horizontal channel when the channel lengths are the same.

At last, we would like to discuss the impact of the power-law exponent α which account for the non-Kolmogorov effect. In figure 6(a), the Holevo channel capacity C as a function of α is plotted. The capacity under the total aberration drops sensitively to the variation of α near its two limits 3 and 4. In the region 3.20 < α < 3.78, the capacity is much less sensitive to α, i.e. it grows smoothly with the increase of α. This total decay behavior is in accordance with the variation of the spatial coherence radius ${\tilde{\varrho }}_{0}$ in equation (9) versus α, as shown in figure 6(b). For the tilt aberration, the capacity decays faster and faster as α increases. The variation of α has almost no effect on astigmatism and coma aberrations. Our results indicate that tilt aberration plays a crucial role in weak non-Kolmogorov turbulence and the coma effect one becomes considerable for strong turbulence. The astigmatism effect can always be neglected as the defocus one as shown in equation (8).

In conclusion, we have investigated the influences of non-Kolmogorov atmospheric turbulence on HyGG-II modes by exploring the properties of the Holevo channel capacity of a slant quantum channel. Via the modulation of the hollowness parameter q, capacity with HyGG-II modes can outperform that with LG modes. The contributions of low-order turbulence aberrations including tilt, defocus, astigmatism, and coma on capacity decay are also discussed. Among the low-order aberrations, the tilt one is the main contributor to the capacity decay while the effects of the defocus and astigmatism ones can be neglected. The coma one contributes slightly but its effect can be considerable for stronger turbulence and near horizontal channels. The total and tilt aberrations are sensitive to the variation of the power-law exponent α while the others are not. We believe our results may contribute to the quantum optical communication and aberration compensation for the novel family of vortex beams via turbulent free space.

Project supported by the National Key Research and Development Program of China (Grant No. 2022YFE0122300) and the National Natural Science Foundation of China (Grant Nos. 61871202 and 11811530052).