Logit Transformation

The question is to show the equivalence of \(\frac{e^{\beta X}}{1+e^{\beta X}}\) and \(\frac{1}{1+e^{-\beta X}}\), which are two way of expressing the standard logistic function.

There are some mathematical facts that will help us out here:

• \(\frac{\frac{a}{b}}{c} = \frac{b}{c+a}\)
• \([e^x]^{-1} = e^{-x}\)

Allowing \(X\) to be some matrix of predictors, we can now show the relation above \[ \begin{align*} \frac{e^{\beta X}}{1+e^{\beta X}} &= \\\\ &= \frac{\frac{e^{\beta X}}{e^{\beta X}}}{\frac{1+e^{\beta X}}{e^{\beta X}}} \\\\ &= \frac{1}{\frac{1+e^{\beta X}}{e^{\beta X}}} \\\\ &= \frac{1}{\frac{1}{e^{\beta X}} + \frac{e^{\beta X}}{e^{\beta X}}} \\\\ &= \frac{1}{1 + \frac{1}{e^{\beta X}}} \\\\ &= \frac{1}{1 + e^{-\beta X}} \end{align*} \]

The standard logistic function is important in statistics because it is a natural way to bound the probability of events between 0 and 1, with the log-odds (\(logit(p) = log( \frac{p}{1-p} )\)) representing a map from probability to real numbers in \((-\infty ,+\infty)\).

Finally, lets show the relationship between the standard logistic function and the logit function, where \(logit(p) = log( \frac{p}{1-p} )\)

\[ \begin{align*} log \Big( \frac{p}{1-p} \Big) & = \beta_0 + \beta_1 + \cdots \\\\ \frac{p}{1-p} & = e^{\beta_0 + \beta_1 + \cdots} \\\\ \frac{1-p}{p} & = e^{\frac{1}{\beta_0 + \beta_1 + \cdots}} \\\\ \frac{1}{p} - \frac{p}{p} & = e^{\frac{1}{\beta_0 + \beta_1 + \cdots}} \\\\ \frac{1}{p} - 1 + 1 & = 1+e^{\frac{1}{\beta_0 + \beta_1 + \cdots}} \\\\ \frac{1}{p} & = 1+\frac{1}{e^{\beta X}} \\\\ \frac{1}{p} & = \frac{1}{1} \cdot \frac{1}{e^{\beta X}} \\\\ & = \frac{e^{\beta X}}{e^{\beta X}} \cdot \frac{1}{1} + \frac{1}{e^{\beta X}} \\\\ & = \frac{e^{\beta X}}{e^{\beta X}} + \frac{1}{e^{\beta X}} \\\\ & = \frac{e^{\beta X} + 1}{e^{\beta X}} \\\\ p & = \frac{e^{\beta X}}{1+e^{\beta X}} \end{align*} \]

The logit function if then the inverse of the standard logistic function. Logit is also called the log-odds since it is equivalent to the natural logarithm of the odds-ratio, with \(p\) in \(logit(p) = log( \frac{p}{1-p} )\) representing the probability of an event.