The Shannon-Nyquist sampling theorem

I was going to do a proper blog post to interrupt my silence; unfortunately I’m having difficulty in getting the TeX to work properly.  In particular, for some reason the formulas don’t appear in the right place.

Anyway, this is material that I learned from a course of Sinan Güntürk; once they’re ready, I’ll post the notes I take. 

What I’m posting here now states that signals whose frequencies are bounded can be recovered entirely from sampling at a discrete set of points provided the sampling mesh is sufficiently small.  This fact was popularized by Shannon in his fundamental paper on information theory.

Here is the file.

 

The Fourier transform, the heat equation, and fundamental solutions

I have now discussed what the Laplacian looks like in a general Riemannian manifold and can thus talk about the basic equations of mathematical physics in a more abstract context. Specifically, the key ones are the Laplace equation

for a smooth function on a Riemannian manifold. Since , this often comes up when is the potential energy function of a field which is divergence free, e.g. in electromagnetism. The other major two are the heat equation

for a smooth function on the product manifold for a Riemannian manifold, and the wave equation

in the same setting. (I don’t know the physics behind these at all, but it’s probably in any number of textbooks.) We are often interested in solving these given some kind of boundary data. In the case of the Laplace equation, this is called the Dirichlet problem. In 2-dimensions for data given on a circle, the Dirichlet problem is solved using the Poisson integral, as already discussed. To go further, however, we would need to introduce the general theory of elliptic operators and Sobolev spaces. This will heavily rely on the material discussed earlier on the Fourier transform and distributions, and before plunging into it—if I do decide to plunge into it on this blog—I want to briefly discuss why Fourier transforms are so important in linear PDE. Specifically, I’ll discuss the solution of the heat equation on a half space. So, let’s say that we want to treat the case of . In detail, we have a function , continuous on . We want to extend to a solution to the heat equation which is continuous on and smooth on . To start with, let’s say that . The big idea is that by the Fourier inversion formula, we can get an equivalent equation if we apply the Fourier transform to both sides; this converts the inconvenience of differentiation into much simpler multiplication. When we talk about the Fourier transform, this is as a function of . So, assuming we have a solution as above:

Also, we know what looks like. So this is actually a linear differential equation in for each fixed with initial conditions . The solution is unique, and it is given by

 

Tempered distributions

The next basic tool we’re going to need is the theory of distributions.

Distributions

Distributions are extremely useful because they are both fairly general (including both all integrable functions and things like the Dirac delta function) but also allow for operations such as differentiation. So oftentimes we can obtain distribution solutions to differential equations we are interested in.

Actually, we’ll only discuss here tempered distributions. A tempered distribution is a linear functional . Clearly the tempered distributions form a vector space ; it is a locally convex space if we endow it with the weak* topology. It must now be seen how distributions generalize functions. So, if for any , then can be made into a distribution

In fact, we could just assume that grows polynomially. In particular, we have an imbedding .

An example of a distribution that is not a function is the Dirac distribution mapping .

 

The Plancherel theorem

This post’ll be pretty quick—the Plancherel theorem, a basic result on Fourier transforms, is a quick corollary of what I’ve already talked about.

We have shown that the Fourier transform is an isomorphism of onto itself; the inverse is given by the inverse Fourier transform. The next step is to extend this to an isometry of onto itself. Since is dense in , it will be sufficient to show that

for . We will do this by proving the identity

for . This is a simple computation:

In other words, the Fourier transform and its inverse are adjoints. If we take and use the inversion formula, it becomes clear that the Fourier transform preserves the -norm, whence follows

Theorem 1 (Plancherel) The Fourier transform extends to an isometry of onto itself.

 

Incidentally, for a function , it is not necessarily true that

because that integral need not exist. It is, however, true that the integral will exist almost everywhere in a “principal value” sense, which we do not need to bother with here.

In a sense, this is a continuous analog of the Parseval theorem, which states that the Fourier coefficient map from (for the space of double-sided, square-summable sequences) is an isometry.

 

The Fourier transform and the inversion formula

So, as I’ve already indicated, I’m planning to talk about PDEs for the next month or so, both the general theory and specific posts on the equations of mathematical physics.  There are some preliminaries I’ll have to do first such as Fourier transforms.  Today, I’ll get up to the inversion formula for Schwarz functions.

The Schwarz Class

The Schwarz class consists of smooth functions such that for all multi-indices ,

is bounded. For instance, the function is in , as is any function with compact support. Elements in are loosely speaking, functions that decrease rapidly at with all their partial derivatives. 

There is a way to make the space into a Frechet space, by the countable family of seminorms