estimation logistic regression using newton Raphson

How can I solve logistic regression model using newton raphson?

[Ive J]

https://tomroth.com.au/logistic/

https://www.gomatlab.de/download,id,1794.html

Answer 1

[Sudarsanan A K]

Hi Nurmalitasari,

To solve a logistic regression model using the Newton-Raphson method in MATLAB, you will need to follow these steps:

1. Define the logistic regression function.
2. Implement the gradient and Hessian computation for logistic regression.
3. Apply the Newton-Raphson iterative update until convergence.

Below is a MATLAB example that demonstrates how to solve a logistic regression model using the Newton-Raphson method. This example includes a simple synthetic dataset for illustration purposes and visualizations of the decision boundary.

%% Example MATLAB code for solving logistic regression using Newton-Raphson method

% Create a synthetic dataset
rng(0); % Seed for reproducibility
n = 100; % Number of samples
x = linspace(-5, 5, n)'; % Feature values
y = double((x + randn(n, 1)) > 0); % Labels with some noise

% Visualize the dataset
figure;
scatter(x(y==0), zeros(sum(y==0), 1), 'r', 'filled'); hold on;
scatter(x(y==1), zeros(sum(y==1), 1), 'b', 'filled');
xlabel('Feature x');
ylabel('Probability');
title('Synthetic dataset');
legend('Class 0', 'Class 1');

% Logistic regression function
sigmoid = @(z) 1 ./ (1 + exp(-z));

% Initialize weights
w = [0; 0]; % [intercept; slope]

% Add intercept term to feature matrix
X = [ones(n, 1), x];

% Newton-Raphson iterations
max_iter = 10;
for iter = 1:max_iter
    % Compute the predicted probabilities
    p = sigmoid(X * w);
    
    % Compute the gradient of the log-likelihood
    gradient = X' * (y - p);
    
    % Compute the Hessian matrix
    W = diag(p .* (1 - p));
    Hessian = -X' * W * X;
    
    % Update weights using Newton-Raphson update rule
    w = w - Hessian \ gradient;
    
    % Visualize the decision boundary during iterations
    % Define a range of x values for plotting the decision boundary
    x_range = linspace(min(x), max(x), 100);
    % Compute the corresponding y values using the vectorized function
    y_range = sigmoid([ones(size(x_range')), x_range'] * w);
    % Plot the decision boundary
    plot(x_range, y_range, 'LineWidth', 1); drawnow;
end

% Final visualization with decision boundary
figure;
scatter(x(y==0), zeros(sum(y==0), 1), 'r', 'filled'); hold on;
scatter(x(y==1), zeros(sum(y==1), 1), 'b', 'filled');
% Define a range of x values for plotting the decision boundary
x_range = linspace(min(x), max(x), 100);
% Compute the corresponding y values using the vectorized function
y_range = sigmoid([ones(size(x_range')), x_range'] * w);
% Plot the decision boundary
plot(x_range, y_range, 'k', 'LineWidth', 2);
xlabel('Feature x');
ylabel('Probability');
title('Logistic Regression with Newton-Raphson');
legend('Class 0', 'Class 1', 'Decision Boundary');

Below are the documentation links for the important MATLAB functions utilized in the logistic regression example:

• Anonymous Functions ("At symbol"): https://www.mathworks.com/help/matlab/matlab_prog/anonymous-functions.html
• "diag": https://mathworks.com/help/matlab/ref/diag.html
• "mldivide": https://mathworks.com/help/matlab/ref/mldivide.html

I hope this helps!