Please enable JavaScript.
Coggle requires JavaScript to display documents.
Logistic Regression, Logistic Function, Maximum Likelihood, Confounding,…
Logistic Regression
Algorithm
-
One-vs-All
X = pd.concat([pd.Series(1, index=df.index, name='00'), df], axis=1)
y1 = np.zeros([df.shape[0], len(y.unique())])
//y=1 in true class, 0 otherwise
- Calculate predicted value;
def hypothesis(theta, X):
return 1 / (1 + np.exp(-(np.dot(theta, X.T)))) - 0.0000001

- Calculate loss;
def cost(X, y, theta):
y1 = hypothesis(X, theta)
return -(1/len(X)) x np.sum(y x np.log(y1) + (1-y) x np.log(1-y1))

- Gradient descent algorithm to find for minimal loss;

def gradient_descent(X, y, theta, alpha, epochs):
m = len(X)
for i in range(0, epochs):
for j in range(0, 10):
theta = pd.DataFrame(theta)
h = hypothesis(theta.iloc[:,j], X)
for k in range(0, theta.shape[0]):
theta.iloc[k, j] -= (alpha/m) * np.sum((h-y.iloc[:, j])*X.iloc[:, k])
theta = pd.DataFrame(theta)
return theta, cost
- Run Gradient Descent algorithm;
theta = np.zeros([df.shape[1]+1, y1.shape[1]])
theta = gradient_descent(X, y1, theta, 0.02, 1500)
-
-
Fitness of Model
R-squared
Binary classification
-
From Best fitting line =>

LL(fit) = -3.77
-
p-value

Degrees of freedom = 2 -1 = 1
LL(fit) = 2 parameters (y-intercept, slope)
LL(overall) = 1 (y-intercept)
-
P.S for other GLMs
2[(LL(saturated) - LL(overall)) - (LL(saturated) - LL(fit))]
Logistic regression: LL(saturated model) = 0;
-
Logistic Function
What
C = 2; Inflection point = (0,1)
Steepest point = Most rapid change
Represent exponential growth with UPPER bound due to limitations on food, space or other scarce resources
-
-
-
Example calculations
Total student population = 1200; 4 people know the rumor when it starts; 3 days later 300 know the rumor;
How many people know the rumor by the 4th day?
c = 1200; 2 points on graph = (0,4), (3,300)
-
-
Maximum Likelihood
-
-
What
Can be used to find Optimal value for mean or std dev for any continuous distribution given observed data
e.g Normal, exponential distributions
Normal distribution
-
- To solve for μ, take σ as constant and find where the slope of the likelihood function ∂L/∂μ = 0
-
- To solve for σ, take μ as constant and find where the slope of the likelihood function ∂L/∂σ = 0
-
-
-
-
Confounding
-
-
A student is riskier than a non-student if no information about the student’s credit card balance is available. However, that student is less risky than a non-student with the same credit card balance
-
-
-
-
-
-
-