Now that I’ve discussed some of the basic definitions in the theory of Lie algebras, it’s time to look at specific subclasses: nilpotent, solvable, and eventually semisimple Lie algebras. Today, I want to focus on nilpotence and its applications.

Engel’s Theorem

To start with, choose a Lie algebra {L \subset \mathfrak{gl} (V)} for some finite-dimensional {k}-vector space {V}; recall that {\mathfrak{gl} (V)} is the Lie algebra of linear transformations {V \rightarrow V} with the bracket {[A,B] := AB - BA}. The previous definition was in terms of matrices, but here it is more natural to think in terms of linear transformations without initially fixing a basis.

Engel’s theorem is somewhat similar in its statement to the fact that commuting diagonalizable operators can be simultaneously diagonalized.

Theorem 1 (Engel) Let {L \subset \mathfrak{gl} (V)} consist of nilpotent operators. Then there exists a vector {v \in V}, {v \neq 0}, such that {Xv = 0} for all {X \in L}.

The theorem can also be stated in the form where {V} is a representation for {L} where the elemetns of {L} act via nilpotent operators; in that case there is a vector annihilated by {L}. I will tacitly use this equivalence below.

The corollary makes the analogy between commuting diagonalizable operators clearer:

Corollary 2 Under the same hypothesis, there exists a basis of {V} with which each element of {L} is represented as a strictly upper-triangular matrix.

The proof of the theorem goes by induction on {\dim L} (and not on {n}!). When {\dim L=0}, the result is immediate. Otherwise, if {\dim L \geq 1}, we have a proper Lie subalgebra {L' \subset L}. Choose {L' \subset L} to be a maximal subalgebra.

Claim 1 {L'} has codimension one in {L} and {L'} is an ideal.

Proof: [Proof of the claim] Well, {L'} acts via {\mathrm{ad}} on both {L'} and {L'}. In the latter case, we know by Engel’s theorem for {L'}, and the inductive hypothesis, that there exists a nonzero {\bar{r} \in L/L'} such that {[l', \bar{r}] = \bar{0}} for {l' \in L'}; lifting {\bar{r}} to {r \in L - L'}, it follows that {[L' + kr, L'] \subset L' + L' = L'}. Moreover {[L' + kr, L' + kr] \subset L' + L' + L' + 0 = L'}. Both these imply {L' + kr} is a Lie subalgebra of {L}, and contains {L'} as an ideal. By maximality of {L'}, it follows that {L' + kr=L}, so we’re done. \Box

Now, to prove the theorem, choose {L'} as before, and use the inductive hypothesis again to see that the vector space

\displaystyle  W := \{ w \in V, L' w = 0 \}

is nonzero.

Claim 2 {W} is stable under {L}.

In other words, if {w \in W}, {l \in L}, then {l'( lw ) = 0} for any {l' \in L'}. But

\displaystyle  l' (lw) = l (l'w) + [l', l] w = 0 + [l', l]w,

and {[l',l] \in L'} since {L'} is an ideal, so {[l',l]w=0}. This is essentially a new version of the “fundamental calculation” in discussing representations of {\mathfrak{sl}_2}.

Now, we have {L = k r + L'} for some {r \in L}; we know that {r} is a linear operator on {W} by the previous claim, and is by assumption nilpotent. So {\ker(r) \cap W \neq 0}. Choose {v} nonzero in that intersection; then both {L', r} annihilate {v}, proving Engel’s theorem.

Proof: [Proof of the corollary] By linear algebra, the statement of the corollary is equivalent to the statement: There exists a flag of {L}-stable spaces {0 = V_0 \subset V_1 \subset \dots \subset V_n = V} such that {\dim V_i/V_{i+1} = 1} for each {i}, and {L} acts trivially on the quotients {V_i/V_{i+1}}. To construct the flag, proceed as follows. Choose {V_1} to be spanned by a vector {v} as in Engel’s theorem. Then {L} acts as a family of nilpotent operators on {V/V_1}. Choose {V_2/V_1 \subset V/V_1} such that {V_2/V_1} is generated by an element {v'} annihilated by {L}, by Engel again. Repeat until the process terminates, since {\dim V} is finite. \Box


The notion of nilpotence in Lie algebras is slightly more complicated than simply considering strictly upper-triangular matrices. The idea instead is to say that a Lie algebra is nilpotent if it is at least “almost” commutative, in that taking successive brackets eventually yields zero.


Definition 3 The lower central series of a Lie algebra {L} is defined by {L_0 := L, L_1 := [L, L_0], \dots, L_n := [L, L_{n-1}], \dots}. A Lie algebra is said to be nilpotent if its lower central series eventually becomes zero.

If a Lie algebra is nilpotent, then {\mathrm{ad} \ x: L \rightarrow L} must be nilpotent for any {x \in L}. Indeed, {\mathrm{ad} \ x} maps {L_n \rightarrow L_{n+1}} since {\mathrm{ad} \ x(y) = [x,y]}.

The converse is also true:

Theorem 4 Suppose {L} is a Lie algebra such that each {\mathrm{ad} \ x} is nilpotent. Then {L} is nilpotent.

Induction on {\dim L}, as usual. The image {N := \mathrm{ad} (L) \subset \mathfrak{gl}(L)} is a Lie subalgebra (since {\mathrm{ad} } is a homomorphism of Lie algebras). Each element of {N} consists of a nilpotent operator on {L} by assumption. So by Engel, there exists {l \in L} such that {[L, kl]=0}. In other words, {l} “commutes” with all of {L}, or lies in the center:

Definition 5 The center of a Lie algebra {L} is the set of all {x \in L} such that {[x,l]=0} if {l \in L}.

The center is actually a Lie ideal of {L}. We have just shown that under the hypotheses of the theorem, the center {Z} of {L} is nonzero. Now {L/Z} still satisfies the conditions of the theorem, and by the inductive hypothesis is nilpotent. So we are reduced to:

Lemma 6 Suppose {L} is a Lie algebra, {Z} its center, and {L/Z} is nilpotent. Then {L} is nilpotent.

Indeed, it follows by the nilpotence of {L/Z} that some {L_n \subset Z}, which implies {L_{n+1} = [L, L_n] \subset [L, Z]=0}.