[Biography] [ Research ] [ Publications ] [ Collaborations ] [ Miscellaneous ] [ Home]

Research - Ha NGUYEN

[Research Interests ] [ Coursework ]

 

Research Interests

 

 

Image-Based Rendering with Depth Information:

 

We propose a new approach, called the Propagation Algorithm, for the image-based rendering (IBR) application that synthesizes novel images, as taken by virtual cameras at arbitrary viewpoints, using a set of acquired images. The Propagation Algorithm proactively propagates all available information from actual cameras to virtual cameras, using the depth information at all or some of intensity pixels. This process turns the IBR problem into a two-dimensional interpolation problem at virtual image planes. Each virtual image thus can be efficiently rendered at once for all image pixels. Experimental results show the Propagation Algorithm produces accurate rendering, even around object boundaries -- where most of the existing IBR techniques fail.

Teddy Demo : the inputs of the algorithm is four images below (from left to right: left disparity map, left image, right image, and right disparity map). We render a virtual camera moving from the left camera to the right camera. The images are download from http://cat.middlebury.edu/stereo/newdata.html. Please download the movie here: Teddy_Demo.avi.

                                

 

Cones Demo: the inputs of the algorithm is four images below (from left to right: left disparity map, left image, right image, and right disparity map). We render a virtual camera moving from the left camera to the right camera. The images are download from http://cat.middlebury.edu/stereo/newdata.html. Please download the movie here: Cones_Demo.avi.

 

                                

 

 

Image-Based Rendering Using Sparse Correspondences from Uncalibrated Images:

 

We propose a new algorithm, called the Propagation Algorithm, for the image-based rendering (IBR) application that synthesizes images of novel views using a set of acquired images. The proposed algorithm takes as inputs a sparse set of feature correspondences and applies structure from motion techniques to obtain a quasi-affine reconstruction for the set of features. We interpolate the projective depth for other actual pixels based on the Delaunay triangulation of feature points before propagating the intensity information to the virtual cameras. At the virtual cameras, occluded points are removed using the projective depth. Each virtual image then can be efficiently interpolated at once for all image pixels using the intensity of remaining points. Unlike existing IBR algorithms using uncalibrated images, our algorithm does not require the costly processes of Euclidean reconstruction, image rectification, dense matching, surface fitting, texture mapping, and ray-tracing. In our experiments with real images, the rendered images are of excellent quality with little artifact. 

 

Model House Demo: using two actual images below and a set of sparse correspondences between two images, we render a virtual camera smoothly moving and rotating from the left camera to the right camera. The images are downloaded from http://www.robots.ox.ac.uk/~vgg/data1.html. Please download the movie here: Model_House_Demo.avi.

 

        

 

 

Corridor Demo : using two actual images below and a set of sparse correspondences between two images, we render a virtual camera smoothly moving and rotating from the left camera to the right camera. The images are downloaded from http://www.robots.ox.ac.uk/~vgg/data1.html. Please download the movie here: Corridor_Demo.avi.

 

         

 

 

Sampling the Plenoptic Function:

 

The plenoptic function is the 7D function representing the intensity of the light observed from every position and direction in 3D world. In Image-Based Rendering, the goal is to reconstruct the plenoptic function from a set of its samples (or actual images). Once we can reconstruct the plenoptic function, new images are simply generated by sampling the plenoptic function at the appropriate positions and directions.

 

Characterizing the Plenoptic Function using signal processing tools is an important problem. However, few papers in literature addressed the problem, and most of them analyze the plenoptic function in the frequency domain. These techniques assume that the plenoptic function is bandlimited, although this assumption is never realistic. In fact, the behavior of the intensity function in images should be considered locally rather than globally. In this research we take a different approach, which assumes the plenoptic function is uniformly continuous, and we analyze the reconstruction error in the case we interpolate the plenoptic function using a nonuniform set of samples.

  

 

Optimal Super-Resolution without Input Assumption:

 

We study the problem of super-resolution (SR) that synthesizes a high resolution signal from multiple low resolution signals. Instead of relying on assumptions about input signals (such as

band-limitedness), we develop a system that minimizes the worst error for all finite energy input signals. This is equivalent to minimize the $H_\infty$ norm of an induced error system. The system to provide SR consists of a (IIR or FIR) synthesis filter bank that upsamples, filters each low resolution input and sum up the channels to obtain the high resolution signal. We show that the system is equivalent to a discrete-time system with IIR analysis filters. To design the SR synthesis filter banks, we use powerful tools in control theory including model-matching and linear matrix inequality that have implementation available such in Matlab. Numerical experiments show our approach to be superior to existing techniques. 

  

 

Estimation of Fractional Delays (ongoing):

 

In many data-collection systems, continuous-time signals are sampled at different sampling grid at different sensors. The correlation between the collected data at different sensors reveals important information of the systems. Most current techniques detect the fractional delay by the argmax of the autocorrelation function. Hence they require a lot of computation and are sensitive to noise. In this project, we want to propose a new method to estimate the fractional delay that is efficient and robust to noise.

 

 

Text-to-Synthesis and Implementation on Texas Instrument TMS320C54x Digital Signal Processor (ECE 420):

 

This project has two parts. The first part is Speech Analysis, in which we analyze a set of speech signals using mainly Linear Predictive Coding (LPC) technique. For each speech signal, a set of coefficients such as LPC coefficients, PARCORR numbers will be stored. The second part is Speech Synthesis, in which we use the coefficients stored in the first part and produce the synthesized speech signals. Various techniques will be use to make the speech sound more human than computer-generated.

 

 

Making Videos from Images of Non-Rigid Objects (ECE 598 YM):

 

This project aims at creating intermediate views of a sparse sequence of aquatic images containing fishes. Our algorithm performs two main tasks: motion analysis and image-based rendering (IBR). The fish is considered as a non-rigid body made of ellipsoids which are observed under orthographic projection. Their poses are recovered from the apparent contour of the fish. Actual images are then interpolated to generate virtual ones at different time slots. We have also investigated the image rendering problem in the case where the calibration matrix is not available.

 

 

 

Coursework:

Spring 04:

Fall 04:

Spring 05:

Fall 05: