Motivated by the stability of multi-marginal Schrödinger bridges with respect to an increasing number of marginal constraints, we derive an asymptotic expansion of Schrödinger potentials with respect to the diminishing regularization coefficient $\varepsilon$. This result can also be applied to deriving asymptotic expansion of entropic Brenier maps in entropic optimal self-transport problems. As byproducts of our analyses, we also establish the asymptotic expansion of entropic optimal transport cost with respect to the regularization coefficient when two marginals are sufficiently close, as well as a stability property of the Schrödinger functional.