Mikhail Konnik's
Researches and Developments

Mikhail V. Konnik,

Optical-digital encryption system

2004 - 2009 year

The hybrid optical-digital system is intended for optical encryption of documents printed on a paper. Optical encryption is provided during registration: optical convolution of a point spread function (PSF) of the diffractive optical element (DOE) with an input scene's image is performed. The result of optical convolution (i.e., encoded image) is registered by a digital photo sensor. Digital deconvolution is required for the restoration of encoded images.

Encryption process is shown in Fig. 1.1. The image of the input scene (see Fig. 1.1a) is convolved optically with the PSF of the DOE (see Fig. 1.1b). The result of convolution (i.e., encoded image) is registered by the digital photo sensor hence the image became unreadable (see Fig. 1.1c).

a)	b)	c)

Figure 1.1: The process of optical coding: the image to be encoded (a) is convolved with the PSF of the DOE (b); as a result, the encrypted image (c) became unreadable.

The decryption process is carried out by digital deconvolution algorithms based on inverse filtering. The encoded image (see Fig. 1.2a) is convolved digitally with the inverse PSF (see Fig. 1.2b) that is found by digital deconvolution filter. Such operation compensates the introduced distortion and allows to restore the encoded image. The decoded image (see Fig. 1.2c) is slightly degraded because of imperfect manufacture of the DOE and noise that is introduced by the photo sensor. Such encryption is like a lossy-compression: the quality of the decoded image degrades but resistance of encryption increases. 

a)	b)	c)

Figure 1.2: Decoding of the optically coded image: a) encoded image, the inverse PSF that is found by deconvolution algorithm, c) the image that is restored digitally.

Hybrid optical-digital imaging systems based on ``wave-front coding'' paradigm [1,2,3,4] can be used for correction of aberrations [5] as well as for improvement depth of field of optical systems [6,1].

Subject of research

1. optical coding of information using convolution of a DOE's PSF and an image of an input scene;
2. resolution limits of such optical-digital systems and the influence of noise on digital deconvolution results;
3. influence of a monochromaticism on the decryption's quality results;
4. inverse ill-posed problems of images deconvolution, digital deconvolution filters based on an inverse filtration and regularization.
5. solid-state photo sensors: CCD, CMOS, Foveon, Pixim.

Methods used

1. optical convolution, Fourier-filtration;
2. ill-posed problems and its applications for image processing: linear one-step deconvolution filters with regularization (Tikhonov, Wiener, evolution filters) and iterative deconvolution filters (Metz, Lucy-Richardson, Van Cittert).

Personal participation

The main task was to create portable, inexpensive and reliable optical-digital system for encryption/decryption of printed documents. My part of the work:

1. modification of previous optical setup and application of commercial digital photo cameras in order to create compact and inexpensive optical encryption system;
2. linearization of RAW files using specialised converter DCRAW for obtaining linear data from the commercial camera;
3. testing, selection, and implementation of iterative (Metz, Lucy-Richardson, van Cittert) and one-step (Tikhonov, Wiener, evolution filters) deconvolution algorithms for images' deconvolution;
4. development of evolution filter that can estimate noises using Gauss noise distribution from bound of an encoded image;
5. implementation of deconvolution programs in MATLAB and accessorial programs in Perl/Python.

Results

1. mathematical and optical modelling of encoding and decoding processes was carried out;
2. requirements for spectre's width for light sources were provided;
3. portable optical-digital encryption system based on ``wavefront coding'' paradigm was developed and the portable commercial digital camera as a photo registrar was used;
4. digital deconvolution algorithms for images' decoding, which uses information about photo sensor' noise, were implementated as MATLAB scripts.

Main results were published in [7,8,9,10,11,12] and several papers in Russian as well.

Optical-digital pattern recognition correlator

2006 - 2009 year

The project of the optical-digital pattern recognition correlator is closely connected to the optical-digital encryption system (see Project 1) and uses the same ``wavefront coding'' paradigm. The correlator is intended for pattern recognition in the quasimonochromatic spatially incoherent light. The correlator's hardware consists of the digital photo camera with a diffraction optical element (DOE) inserted as a correlation filter.

The correlation of the input scene's image (see Fig. 2.1a) with the DOE's point spread function (PSF) is optically provided. Produced image is the correlation of the input scene's image with the reference image. The reference image (see Fig. 2.1b) is the DOE's PSF rotated on 180⁰ . Correlation signals (see Fig. 2.1c) can be viewed through the viewfinder or registered by the digital photo camera's sensor.

a)	b)	c)

Figure 2.1: Pattern recognition correlator: a) input scene for correlation analysis; b) the restored by point light source reference image that is correlation filter recorded in the DOE; c) bright spots on the registered correlation signals that indicate recognized objects.

Such correlator has the ability of the input scene's restoration that can be performed by digital deconvolution algorithms. In some applications it is required to look at the recognized objects after correlation search.

Methods used

1. Fourier-filtration and Fourier-optics;
2. ill-posed problems and its applications for images processing;
3. Gerchbert-Saxton's and Fienup's methods of synthesize of a DOE.

Subject of research

A possibility of correlation objects recognition with optically provided convolution of the input scene's image and with DOE's PSF.

Personal participation

Main goal was to make a portable optical-digital system, so there were performed significant amount of engineering works:

1. application of the consumer-grade digital cameras, using specialised RAW-converter;
2. increasing of the correlator's dynamic range using HDR imaging (see Project 4);
3. light source's influence investigation (LEDs, laser diodes);
4. research concerned to the correlator's reliability and accuracy.

Results

There were studied and implemented correlation technique based on the ``wavefront coding'' paradigm. Good recognition's accuracy was obtained, ability of input scene's restoration was implemented. The correlator can distinguish objects even on a HDR input scenes.

Main results were published in [13,14,15,16,7]

Edge detection of images taken in difficult weather conditions for target recognition

2006 - 2008 year

This work was a part of the project of MePHI's Optical Target Detection Lab. The aim was to develop a correlator based on LPCC filters for target detection regardless of target's orientation and scale. The optical-digital correlator is better to operate with binary images; hence fast and reliable edge detection methods were required.

There were tested and implemented several approaches of edge detection, such as first (Sobel, Prewitt, Kirsh) and second derivative (LoG, Laplas) edge detection methods as well as morphological methods of edge detection. The results are briefly summarized in Fig. 3.1. The original registered image of the area is presented in Fig. 3.1a. The result of contouring by Sobel's algorithm, which is shown in Fig. 3.1b, can be considered as well enough but many edges are incomplete and there are false contours on the edge detected image. Canny's algorithm (see Fig. 3.1c) allowed to decrease the noise on the images but it takes too much machine time. The image that was contoured by LOG algorithm (see Fig. 3.1d) suffers from noise and cannot be used in such tasks.

a) b)  c)  d)  e)

Figure 3.1: The comparison of edge detection by different algorithms: a) the original registered image, b) Sobel's algorithm, c) Canny's algorithm, d) LoG's algorithm, e) developed Morphological algorithm.

The best algorithm in our case edge detected images were obtained by the developed morphological algorithm (Fig. 3.1e). Such algorithm, which is based on morphological gradient, employs fast noise-cancelling OCCO [17] technique and hence is well-suited for contouring images that are taken in difficult weather conditions.

Methods used

First-derivative methods of edge detection (Sobel, Canny, Prewitt, Roberts, Frei-Chen), second-derivative methods (LoG, Laplas), and morphological edge detection methods.

Subject of research

The main subject was the development of edges detection algorithms for contouring images that were taken in difficult weather conditions and to make contours independent from weather conditions as much as it possible.

Personal participation

1. development, implementation, and testing various of edge detection approaches;
2. development of the morphological edge detection algorithm based on morphological gradient and OCCO noise-cancelling technique;

Results

There were developed and implemented the algorithm based on the morphological edge detection and OCCO noise-cancelling technique.

Results were published only in Russian [18,19] because of non-civil status of the project.

High dynamic range imaging

2007 - 2009 year

High dynamic range (HDR) imaging is desired in many practical applications. Among others, Spatially Varying pixels Exposures technique can be applied for HDR imaging. It was shown [20,21] that Bayer CFA can be considered as an array of attenuating filters in the quasimonochromatic light. When pixels related to wavelength of the light source are saturated, pixels covered by other colour filters are far from saturation. Extracting data from pixels of different colour-filter type it is possible to recover an HDR image from a single registered oversaturated image.

The process of reconstruction of the oversaturated image (see Fig. 4.1a) is the following. If the exposure value is long enough, some of pixels may be oversaturated but neighbour pixels under other light filters are generally not. Utilizing data from the neighbour pixels it is possible to restore oversaturated regions of the captured image into an HDR image (see Fig. 4.1b).

a)	b)

Figure 4.1: Reconstruction of the oversaturated image (a) into HDR image (b) using Spatially Varying pixels Exposure technique.

Methods used

Optimization methods for fitting experimental data (Least squares, Trust-Region).

Subject of research

Utilization of the information from neighbour pixels of Bayer's CFA in quasimonochromatic light increasing the dynamic range of images of correction signals (see Project 2). Approximation methods of radiometric curves using Least-Squares and Trust-region methods.

Personal participation

This is the first independent project for me started on my own initiative, so task proposal and realisation was carried out.

1. mathematical modelling and numerical experiments of achievable dynamic range of such method;
2. development of the HDR reconstruction algorithm based on the Spatially Varying pixels Exposures technique;
3. experimental restoration of the correlation signals from the pattern recognition correlator; the increase of the dynamic range of registration of correlation signals from 58 dB up to 73 dB was obtained.

Results

Method of images reconstruction based on Spatially Varying pixels Exposures technique was developed and applied to the optical-digital pattern recognition correlator. Using only first accessorial pixels, I have obtained the increase of dynamic range of correlation signal's registration from 58 dB up to 73 dB.

Results were published in [13,14,22].

Unmanned vehicle system [planned]

Projected, initiative project

Portable unmanned vehicle systems are under intensive researches now. Such systems are useful both in military and civil applications.

The main idea is to develop and make researches of bounds detections, making decisions, and implement of some ideas behind image processing of in-vehicle systems.

Figure 5.1: Unmanned vehicle system: the chassis of the vehicle system;

Bibliography

1

W.T. Cathey and E.R. Dowski.
New paradigm for imaging systems.
Applied Optics, 41:6080-6092, 2002.

2

Sudhakar Prasad, Todd C. Torgersen, Victor P. Pauca, Robert J. Plemmons, and Joseph van der Gracht.
Engineering the pupil phase to improve image quality.
In Proc. SPIE, Visual Information Processing XII, volume 5108, pages 1-12, 2003.

3

J. van der Gracht, V. P. Pauca, H. Setty, R. Narayanswamy, R. Plemmons, S. Prasad, and T. Torgersen.
Iris recognition with enhanced depth-of-field image acquistion.
In Proc. SPIE, Visual Information Processing XIII, volume 5438, pages 120-129, 2004.

4

R. J. Plemmons, M. Horvath, E. Leonhardt, V. P. Pauca, S. Prasad, S. B. Robinson, H. Setty, T. C. Torgersen, J. van der Gracht, E. Dowski, R. Narayanswamy, and P. E. X. Silveira.
Computational imaging systems for iris recognition.
In Proc. SPIE, Advanced Signal Processing Algorithms, Architectures, and Implementations XIV, volume 5559, pages 346-357, 2004.

5

H. Wans, E.R. Dowski, and W.T. Cathey.
Aberration invariant optical/digital incoherent systems.
Applied Optics, Vol. 37, No. 23:5359-5367, August 10, 1998.

6

Daniel L. Barton, Jeremy A. Walraven, Edward R. Dowski Jr., Rainer Danz, Andreas Faulstich, and Bernd Faltermeier.
Wavefront coded imaging systems for MEMS analysis.
Proc. of ISTFA, pages 295-303, 2002.

7

Sergey N. Starikov, Nikita N. Balan, Mikhail V. Konnik, Vladislav G. Rodin, Ivan V. Solyakin, and Ekaterina A. Shapkarina.
Input scene restoration in pattern recognition correlator based on digital photo camera.
In Proc. SPIE, Optical Pattern Recognition XVIII, volume 6574:65740J, 2007.

8

M.V. Konnik, E.A. Manykin, and S.N.Starikov.
Digital image restoration in imaging systems with optical coding.
In Holography Expo-2006, September 26-28, Moscow, Russia, 2006.

9

Konnik M.V., Manykin E.A., and Starikov S.N.
Images reconstruction in optical-digital imaging systems based on ``wavefront coding'' principle.
In Proceedings of the topical meeting on Optoinformatics'2008, pages 24-28 (September 15-18, Saint-Petersburg, Russia), 2008.

10

Konnik M.V., Serejkin L.B., and Starikov S.N.
Analisys and application of iterative deconvolution methods in optical-digital systems [in russian].
In in Proceedings of Moscow Telecommunication Conference, Moscow, MePHI, Russia, 2009.

11

Konnik M.V., Manykin E.A., and Starikov S.N.
Images linearization of commercial digital camera for holographic optical-digital imaging systems.
In Holography Expo-2007, September 25-27, Moscow, Russia, 2007.

12

Konnik M.V., ``Increasing dynamic range of commercial photo sensors by RAW image processing'', Russia, Moscow, Russian Exhibition Center, Official catalog of VIII All-Russian Exhibition ``Sci-Technical Creativity of Youth'' (HTTM'2008), page 135, 2008.

13

Konnik M.V., Manykin E.A., and Starikov S.N.
Application of spatially varying pixels exposure technique for increasing dynamic range of correlation signals' registration [in russian].
In in Proceedings of Moscow Telecommunication Conference, Moscow, MePHI, Russia, 2009.

14

Konnik M.V., Manykin E.A., and Starikov S.N.
Increasing dynamic range of optical-digital correlator [in russian].
In in Proceedings of Moscow Telecommunication Conference, Moscow, MePHI, Russia, 2009.

15

S.N. Starikov, N.N. Balan, V.G. Rodin, I.V. Solyakin, and E.A. Shapkarina.
Pattern recognition correlator based on digital photo camera.
In Proc. SPIE, Optical Pattern Recognition XVII, volume 6245, page 62450C, 2006.

16

Sergey N. Starikov and Mikhail V. Konnik.
Using commercial photo camera's raw-based images in optical-digital correlator for pattern recognition.
In Proc. SPIE, Optical Pattern Recognition XIX, volume 6977:69770R, 2008.

17

R. A. Peters.
A new algorithm for image noise reduction using mathematical morphology.
IEEE Transactions on Image Processing, Volume 4, Number 3:554-568, 1995.

18

Konnik M.V., Starikov R.S., Starikov S.N., and Evtikhiev N.N.
Edge detection algorithms for aerophotography images contouring taken in difficult weather conditions [in russian].
Naykoemkie tehnologii (Knowledge-intensive Technologies), (to be published).

19

Evtikhiev N.N., Konnik M.V., and Starikov S.N.
The development of edge detection algorithms for optical-electronical correlator for images distinguishing [in russian].
In Proceeding of Moscow Telecommunication Conference, Moscow, MePHI, Russia, 15:137-138, 2007.

20

S. K. Nayar and T. Mitsunaga.
High dynamic range imaging: Spatially varying pixel exposures.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 1:pp.472-479, June, 2000.

21

Srinivasa G. Narasimhan and Shree K. Nayar.
Enhancing resolution along multiple imaging dimensions using assorted pixels.
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, No. 4:pp. 518-530, April 2005.

22

Konnik M.V.
Numerical simulation of images resocnstruction with high dynamic range [in russian].
In Proceedings of 5th Conference ``Mathematical Modeling and Boundary Problems'', (Russia, Samara) part 4, pages 88-90, 2008.