Mfrhtyj

Suppose $f,g:mathbb{R} rightarrow mathbb{R}$ are such that $f(x)$ is continuous at $x=alpha$ and $f(alpha)=a$ and $g(x)$ is discontinuous at $x=a$, but $gbig(f(x)big)$ is continuous at $x=alpha$. Also, $f(x),g(x)$ are non-constant functions. Then, can it be said that $x=alpha$ is an extremum of $f$ and $x=a$ is an extremum of $g$?

I have tried to construct examples of functions, but never could figure out a rigorous proof
For example take the function $f(x)=x^2$ which is continuous at $0$, $g(x)=[x]$ which is discontinuous at $0$, but $gbig(f(x)big)$ is continuous at $0$.

Here are some counterexamples, using the terminology of OP, i.e. $f$ continuous in $alpha$, $g$ not continuous in $a=f(alpha)$ and $g circ f$ continuous in $alpha$, always with $alpha = 0$ and $a = 0$.

In this example $alpha$ is an extremum for $f$ but $a$ is not an extremum for $g$:
begin{eqnarray}
f(x) &=& x^2\
g(x) &=& begin{cases}
x+1 & (x geq 0)\
x-1 & (x < 0).
end{cases}
end{eqnarray}

In the following example $alpha$ is not an extremum for $f$, but $a$ is an extremum for $g$:
begin{eqnarray}
f(x) &=& begin{cases}
x & (x mbox{rational})\
0 & (x mbox{irrational})
end{cases}
\
g(x) &=& begin{cases}
1 & (x mbox{rational})\
0 & (x mbox{irrational}).
end{cases}
end{eqnarray}

Finally an example where neither $alpha$ nor $a$ are extrema of $f$ and $g$ respectively:
begin{eqnarray}
f(x) &=& begin{cases}
x & (x mbox{rational})\
0 & (x mbox{irrational})
end{cases}
\
g(x) &=& begin{cases}
1 & (x mbox{rational})\
x & (x mbox{irrational}).
end{cases}
end{eqnarray}
Concering the last two examples, note that $f(mathbb R) subseteq mathbb Q$, so that $g circ f$ is constant.

Here is a possible variation of OP. I don't think it is of much interest, and worked it out just to do a little practice myself. But you might want to have a look and see if it works.

Consider two functions $f,g:mathbb R rightarrow mathbb R$, such that $f$ is continuous in $alpha$ and not constant in any neighborhood of $alpha$. Let then $f(alpha) = a$ and suppose furthermore that
begin{equation}
lim_{xrightarrow a^+} g(x) = ell_+ tag{1}label{piu}
end{equation}
and
begin{equation}
lim_{xrightarrow a^-} g(x) = ell_- tag{2}label{meno}
end{equation}
exist and they are different, so that $g(x)$ is not continuous in $a$. If $(gcirc f)(x)=g(f(x))$ is continuous in $a$, then $alpha$ is an extremum of $f$.

Proof. Suppose that $alpha$ is not an extremum of $f$. Since $f$ is not constant in any neighboorhood of $alpha$, then in the neighborhood
begin{equation}
I_{2n} = left(alpha-frac{1}{2n}, alpha+frac{1}{2n}right), nin mathbb Z^+
end{equation}
there must be a value $alpha_{2n-1}$ for which $f(alpha_{2n-1})< a$ and a value $alpha_{2n}$ for which $f(alpha_{2n})> a$.

The sequence $(alpha_n)$ converges to $alpha$. Thus, by continuity of $f$, $(f(alpha_n))$ converges to $f(alpha) = a$. However, the sequence $(g(f(alpha_n)))$ does not converge, since the subsequence $(g(f(alpha_{2n}))$ converges to $ell_+$, by eqref{piu}, and the subsequence $(g(f(alpha_{2n-1}))$ converges to $ell_-$, by eqref{meno}. This contradicts continuity of $gcirc f$ in $a$.