Mfrhtyj

The following question has already appeared on mathstack:

If $B$ is the subgroup of ${rm GL}(n,F)$ consisting of upper triangular matrices then normalizer of $B$ in ${rm GL}(n,F)$ is $B$ itself.

I know a proof of this using Bruhat decomposition of ${rm GL}(n,F)$.

Question: Can we prove above theorem without using Bruhat decomposition?

Why came to this question: Consider the general linear Lie algebra $L=mathfrak{gl}(n,F)$; in it, let $T=mathfrak{t}(n,F)$ be the upper triangular sub-algebra. Then normalizer of $T$ in $L$ is $T$ itslef, and this can be proved just by considering a very simple decomposition of ${mathfrak gl}(n,F)$: write any element as sum of upper triangular matrix and lower triangular matrix whose diagonal is $0$.

But then for problem above, is it necessary to use Bruhat decomposition?

$DeclareMathOperator{GL}{GL}$$newcommand{Span}[1]{leftlangle #1 rightrangle}$$newcommand{Set}[1]{left{ #1 right}}$Let $e_0, e_1, dots, e_{n-1}$ be a basis with respect to which $B$ is upper-triangular, and write
$$
V_i = Span{ e_j : j ge i}.
$$
Allow me to use row vectors, so that the group $G = GL(n, F)$ acts on the right.

Then
$$
B = Set{b in G : V_i b subseteq V_i text{ for each $i$}}.
$$
Moreover,

$V_{n-i}$ is the unique subspace $W$ of dimension $i$ such that $W B subseteq W$.

This is proved by induction on $i$. For $i = 1$ we have that $V_{n-1}$ is the unique common eigenspace for the elements of $B$ (just consider the element of $B$ which is a single Jordan block of size $n times n$ and eigenvalue $1$, say), then pass to $V / V_{n-1}$ and use induction.

Let $g in N_{G}(B)$. Then for each $b in B$ we have $g b g^{-1} in B$, that is for all $i$
$$
V_{i} g b g^{-1} subseteq V_i
$$
or
$$
(V_{i} g) b subseteq V_i g.
$$
It follows from the above that $V_{i} g = V_{i}$, so that $g in B$.

It can be shown by direct computation. First, we need the following lemma.

Lemma. Let $gin GL(n,F)$. Let $h:=g^{-1}$. If there are entries of $g$ and $h$ such that
$$g_{ij}neq 0,h_{kj}neq 0,i>j,kgeq j,$$
then $g$ is not in $N(B)$, the normalizer of $B$.

(Proof)Let $b=I+E_{jk}$, where $E_{jk}$ is the matrix whose entries are all zero except for the $(j,k)$-entry. Since $kgeq j$, $b$ is an element of $B$. Notice that $(gbg^{-1})_{ij}=(g)_{ij}(h)_{kj}neq 0$. So $gbg^{-1}$ is not upper triangular(, for $i>j$). Hence $gbg^{-1}not in B$. Therefore $gnot in N(B)$. $square$

Now we prove that $N(B)=B$ by induction on $n$.

Let us denote by $B_n$ the set of invertible upper triangular matrices of size n and by $N(B_n)$ its normalizer in $GL(n,F)$. Assuming that $N(B_{n-1})=B_{n-1}$, we prove $N(B_{n})=B_{n}$.

Suppose that $g_0in N(B)$. Let $h_0:=g_0^{-1}$. Since $g_0$ and $h_0$ are invertible, neither of them has a column of zero. If we have $(g_0)_{i,1}neq 0, (h_0)_{k,1}neq 0$ for some $i,k$, then the above lemma implies that $i=1$. On the other hand, Since $g_0$ and $h_0$ are invertible, neither of them has a column of zero. Hence $g_0$ has the form$$g_0=left[begin{array}{cccc}
d_1 & ast & cdots & ast\
0 & ast & cdots & ast\
vdots & & vdots\
0 & ast & cdots & ast
end{array}right].$$
We then multiply a product $c_1$ of elementary matrices that corresponds to adding multiples of the first column to other columns, which is in $B$, to obtain an element $g_1:=g_0c_1$ of the form
$$g_{1}=left[begin{array}{cccc}
d_1 & 0 & cdots & 0\
0 & ast & cdots & ast\
vdots & & vdots\
0 & ast & cdots & ast
end{array}right].$$

The $(n-1)times (n-1)$ matrix $g_1'$ that is obtained by deleting the first row and the first column of $g_1$ is in $N(B_{n-1})$; thus by induction hypothesis, $g_1'in B_{n-1}$. It follows that $g_1in B_{n}$. Hence $N(B_{n})subset B_{n}$.The converse inclusion holds trivially.