We study stochastic dynamics of an inverted pendulum subject to a random force in the horizontal direction. Considered at the entire time axis, the problem admits a unique solution which always remains in the upper half plane. We develop a new technique for treating statistical properties of this unique non-falling trajectory. In our approach based on the supersymmetric formalism of Parisi and Sourlas, statistics of the non-falling trajectory is expressed in terms of the zero mode of a corresponding transfer-matrix Hamiltonian. The emerging mathematical structure is similar to that of the Fokker-Planck equation, but it is rather written for the «square root» of the distribution function. We derive the specific boundary conditions that correspond to the non-falling trajectory. Our results for the distribution function of the angle and its velocity at the non-falling trajectory are in perfect agreement with direct numerical simulations of the stochastic pendulum equation. In the limit of very strong noise, an exact analytical solution is obtained.