Integrating Algorithms, Computation, and Sensors for Imaging
February 5, 2014, Electronic Imaging Conference, San Francisco—Charles Bouman from Purdue discussed integrated imaging that takes advantage of the optimizations possible with different imaging techniques that integrate algorithms, computation, and sensors. Unlike current systems that just extract data from the sensors, an integrated design system could have better performance at lower power.
Today’s sensors are fairly limited, even though the latest ones have multiple functions. This increased device integration means that the sensors are not repairable, leading to the use of the most informative measures possible being barely what we need.
To compound the problems, the adaptive imaging only considers a small fraction of all the information available. We use these extracted images to develop understanding of an event, which is an inverse problem solution of model-based iterative reconstruction. The prior model is a non-linear, thin manifold with constraints on the solution. It is transformed to a forward model that is changed to map to the system. For most systems, the sensors for the most information is used to form the images.
For example, a CT scan uses the inverse solution to reconstruct a 3-D vision from1-D projections, a forward model and prior models. The uncertainty is reconstructed as noise. Further optimization is possible, but necessarily involves high compute. Faster algorithms can converge in 3-6 iterations, so a change in algorithms reduces scan time and increases resolution. The iterations allow for differential image resolution or noise optimization at the cost of computational intensity.
Synchronous imaging is an application that is used to study the solidification of solids. This is a time and space reconstruction that traditionally used many views causing slow capture. Now, the images are interlaced by rotating the object and generating k sub-frames which can be reconstructed at any time. The synergy between the view sample and MBIR recorded can develop a 4-D (3-D plus time) high frame rate image.
In transmission electron microscopy (TEM) the images are in bright-field or dark field, and the sample can be tilted. In a bright-field, the detector is at the beam point and depends upon Bragg scattering, while in a dark-field the detector is in the center of a corona. A filter back reconstruction is generally of poor quality, but by changing weighting, the Bragg anomaly classification can provide segmentation and signatures. These classes can be extracted to generate Bragg features.
Any advanced prior model has many levels, which all need training, and some type of bilateral filtering. The majority of filters are Gaussian and much of the filtering is just for de-noising. Work on the priors shows that the many de-noise algorithms work with different data types. A model-based dynamic sampling gives a faster sample, but each sample is optimally bad; each sample has the least information.
A better sampling strategy is random, which will contain more information and uncorrelated data points. A greedy optimal strategy uses recursion to select the pixels with the greatest variances. This will result in a non-Gaussian distribution, and the dynamic samples are compute intensive. Instead, use more samples near areas of high change, a “battleship” search.
In a SEM, a good reconstruction is an analog to human vision. In a bottom-up flow, the SEM looks for individual sensor features, while in a top-down flow, the cognitive priors are discovered. By co-designing the sensors, algorithms, and computational workloads, you can optimize for maximum information retrieval and image quality. The image formation comes from a forward model to account for complex non-linear parameters and models. The priors account for properties of the image. You need interdisciplinary teams to address all of the variables and domains under investigation.
