An Adaptive Image Registration Technique to Remove Atmospheric Turbulence

Turbulence/Heat Scintillations is the change that arises in the refraction index of air with the temperature. The distortion created by the atmospheric turbulence is proportional to the distance between object and camera. In last few years, several approaches have been proposed to estimate and eliminate geometric distortion and blur. In this paper, a novel technique is proposed that improves the visual quality of video sequences affected by atmospheric turbulence. The proposed method is based on adaptive control grid interpolation (CGI). CGI approximates accurate motion vectors to generate a geometrically correct frame using certain reference frames. For high scintillation sequences, CGI doesn′t mitigate scintillations completely. The new methodology is proposed with updated trajectory estimation. The proposed method can effectively reduce the influence even for high atmospheric turbulence. Experimental results also prove that proposed approach is time efficient compare to traditional CGI.


Introduction
The visual quality of long rang imagery capturing can be influenced by various types of atmospheric distortion [1]. Hot air, fog, and haze are the primary reason behind geometric distortion and space and time-arying blur [1] [2]. Captured frames can be mathematically represented as Where f k represents k th observed frame, D k and B k denote the deformation matrix and the blurring matrix respectively for the k th frame, I k denotes original image without any effect of turbulence and ϵ denotes the noise [3]. Heat scintillations are reliable on certain parameters like temperature, the speed of wind, the distance between object and camera, and time. Two major techniques are mainly used for mitigation. First one is based on frame reconstruction using sequential video frames [4] [5]. This method uses non-rigid image registration technique to register each observed frames with using turbulence free reference frame [3][4] [5]. Non-registration based Sobolev Gradient and Laplacian approach is proposed by Y. Lou et al et.al to stabilize the video sequence [6]. To mitigate the effect cause by turbulence is necessary for long distance image capturing. This is useful for surveillance, defence and in many commercial applications.Turbulence caused by atmosphere leads to most of the oscillation in the video

Proposed Approach
The main challenge that traditional CGI face is that partial scintillation is still left out and (with) higher CPU computations. To remove both disadvantages this paper proposes a novel CGI based technique that adaptively updates trajectory estimation. A block diagram that includes the basic flow of the proposed system is as described in fig 1. The distorted video sequence is only input to mitigate the problem. Spatial transformation is applied to the

Distorted Sequential Video Frames
Most commonly, High-resolution visible band video capturing is preferable for applications like surveillance and defense. The removal of geometrical distortion is not potential using one observed video frame, Hence distorted video sequences are used to compensate distortion. In related work certain approaches are described that performs registration by converting sequences into grayscale. Here, a standard dataset of Open Turbulent Image Set (OTIS) is used for testing the proposed approach that contains visible band sequences [18]. In this research the electric fields inside the insulation of the construction were determined from a boundary element method simulation: along a circle among the phases and the tank, along vertical line starting from the bottom of the tank, along line connecting the centers of conductor phases, and around the conductor. The electric field calculation is two dimensional.  Non-label based spatial transformation is used to minimize some index of difference between the target image and reference image [19]. Two-dimensional basic derivative filters are used to convolve image horizontally and vertically. As shown in Fig. 3 The grayscale frame is used to find the spatial transformation.

Bi-linear Interpolation for Motion Estimation
The purpose of motion estimation technique is to allow similar features of the different images to be registered [20]. In CGI, registration is performed among the small and same size blocks of the target and reference frames.
The relationship between pixels among the block in Here we can use first-order Taylor series approximation Which discard the high order terms from (3). Bilinear parameters of the calculated motion vector are used to recover the displacement of the pixels within the region.

Adaptive Trajectory Estimation
Several approaches are proposed in [2] [17]. One of them is to estimate the final trajectory of each new frame using the trajectory of the previous frame. This is Mathematical represented as Where F 0 denotes target frame, T represents trajectory, v represents motion vector between the target frame and reference frames. Total n trajectories for single target frame where n denotes the number of frames used as reference frame [2] [17]. This method repeats the error of trajectory calculation for all compounded frames [2]. Another technique is to find the target frame trajectory to fix the target frame. This will increase the computational load. The motion vector is required to be calculated each time. Therefore, the error-propagation problem is solved, but the computation is greatly increased. This happens as target frame shifts and each motion field with the previous frame is required to be recomputed [18]. The total number of trajectories is 2 × n, due to change in selecting reference frame. Consecutive (2 × n) + 1 number of frames used as reference except (n + 1) th frame which is target frame [2]. From input video sequences, total 2 × n number of video frames will never be turbulent free. The first and last n frames in the video sequence will be used as a reference frame but not as a target frame. Here, a novel trajectory estimation technique is proposed, which mitigates both arrived problems in previous techniques. For calculating an efficient trajectory for target frame, calculate all trajectory of the sequential turbulent frame using its next frames as a reference frame. Mathematical representation of proposed technique is as follow Here, Total n frames of input video frames. Total n − 1 number of trajectories is calculated for n turbulent frames. Each trajectory is being calculated using its next frame as a reference frame. In process of calculating T k , the k th frame is target frame and (k + 1) th is the reference frame. As shown in (7), T 0 to T n−1 are trajectories for n number of turbulent frames. To calculate the final trajectory matrix of each frame except first and last frame, mean is calculated using two continues trajectory metrics. The final trajectory of the k th frame using mean trajectory Here, the sum matrix is divided by 3 to find mean, due to three frames are involved to calculate trajectory.

Image Warping
Warping is the process to digitally manipulate the image. In brief, the points are projected to points without varying its colors. In this approach, warping is used to correct distortion. Pixels of input video frames are horizontally and vertically mapped using trajectories of each pixel, which generates the geometrically corrected frames -Registered sequences. In case of color sequences, after estimating trajectory, warping is applied on all three planes (r-g-b) of the target frame. This simple technique is useful because the distortion of colors is less influence on human visual perception.

Threshold C Blind Iteration
The global MSE (Mean Square Error) [21] is calculated of registered sequences. MSE is used to measure the average of squares of the errors. Global MSE of n registered frames is calculated by taking the mean of an array of MSE values. The MSE of an array is of n-1 size which contains MSE between consequences frames. The global MSE will be compared with a threshold value. If it is smaller than the threshold, the registration process will continue by using registered frames as input video sequences. If global MSE is larger than the threshold value, iteration will be stopped.

Turbulence Free Video Sequences
After getting global MSE which is lesser than the threshold, the registered frames will be final output. The output video sequence is achieved resultant turbulence free video sequence on the basis of the threshold value.

Experiment and Simulation
Adaptive CGI approach has been tested using three sequences of a state of the art dataset C OTIS [18]. These datasets are simulated datasets containing sequences of optical observations made through atmospheric turbulence.  The results represented in fig. 4 shows that CGI mitigate the turbulence but the intensity is more accurate in ACGI. Certain parameters can be used to measure the similarity between images. Here, SSIM (Structure similarity index) is calculated between consecutive frames to estimate similarity. Fig. 5 represents graphs of calculated SSIM of scintillated datasets. ACGI output sequences lead SSIM of consecutive frames near to 0.95 which is better compared to CGI and turbulent sequences. Overall comparison depicts that the proposed approach outperforms state of the art methods.

Conclusion and Future Work
This paper introduces a novel approach to restore a high-quality image from a video sequence degraded by atmospheric turbulence. Time and computational cost of the proposed approach are proportional to the heat scintillation. This approach modifies the CGI by changing reference frame selection technique and converting blind iteration using threshold value. Registration is performed between target frames with updated consequent reference frames. Experiments with degraded sequences show the more accurate performance compare to the existing method. Measured parameters clearly represent the superiority of proposed method. In future, the algorithm should also work for real motion detection and restoration. The approach should also work for video sequences containing scene change.