How to predict responses of new data from linear SVM model?

Hello, I am trying to calculate new predictions with trained linear SVM regression (4 input variables and 1 output).

from this equation, I tried:

x_new = [8.9 10.2 42.8 44.8]; % New input variables (4 variables)

supportVectors = Mdl.SupportVectors;

alpha = Mdl.Alpha;

bias = Mdl.Bias;

w = (alpha .* supportVectors)';

prediction = (w .* x_new) + bias;

-------------------------------------------------------------------

However, when I compare the results of the code below, there's a different value:

prediction = Mdl.predict(x_new);

If there is a code that calculates the output value when a new observation comes in through the trained SVM, please help me.

Answer 1

[Angelo Yeo]

Here is the example I cooked up for you. I assumed that you would want to use a linear kernel. For more information on manual prediction, see the doc below.

Support vector machine (SVM) for one-class and binary classification - MATLAB (mathworks.com)

%% Setup
% I will generate random data for two groups.

X = randn(100, 1) * 1 + 9; % It's just my random guess.
X = [X; randn(100, 1) * 1 + 43]; % It's just my random guess.

Y = [ones(100,1)*0; ones(100,1)*1];

Mdl = fitcsvm(X, Y, "KernelFunction", "linear", "Standardize", true);
%% prediction test
x_new = [8.9 10.2 42.8 44.8]; % New input variables (4 variables)

% We need to normalize the new input for manual prediction
x_new_norm = (x_new - Mdl.Mu) / Mdl.Sigma;
supportVectors = Mdl.SupportVectors;
alpha = Mdl.Alpha;
bias = Mdl.Bias;
% w = (alpha .* supportVectors)';
s = Mdl.KernelParameters.Scale;
beta = Mdl.Beta;
prediction = (x_new_norm/s)'*beta + bias;
prediction_label = prediction > 0

prediction_label = 4x1 logical array
   0
   0
   1
   1

[minhyuk jeung]

Thank you for your kindly answer.

However, I'd like to know the Y output from the regression SVM method.

I think your example is for the classification (ex., 0 0 1 1 for output).

When I run the code, I believe the "prediction" will be the output from the regression SVM.

(prediction = (x_new_norm/s)'*beta + bias;)

Despite having only one output, I always get the "4 x 1 double" value (4 input, 1 output for my research).

Can you solve this problem again?

Thank you

[Angelo Yeo]

Sorry I think I didn't realize you asked a question on SVM regression. Can you share your code? I want to know what you mean by you have 4 x 1 double output when you trained the regressor for 1-dimensional output.

[minhyuk jeung]

Thank you for reply, you can refer to the code and attachments below.

inputdata = xlsread('inputdata.xlsx','Sheet1');
outputdata = xlsread('outputdata.xlsx','Sheet1');

predictor = inputdata(1:100,1:4);  % 4 input variable
target_variable = outputdata(1:100,1); % 1 output variable

X = predictor;
Y = target_variable;

Mdl = fitrsvm(X,Y,"Standardize",true);
pred_val = predict(Mdl, X);

%% For the new prediction

x_new = [8.9 10.2 42.8 44.8]; % New input variables (4 variables)

% We need to normalize the new input for manual prediction
x_new_norm = (x_new - Mdl.Mu) / Mdl.Sigma;
supportVectors = Mdl.SupportVectors;
alpha = Mdl.Alpha;
bias = Mdl.Bias;
% w = (alpha .* supportVectors)';
s = Mdl.KernelParameters.Scale;
beta = Mdl.Beta;
prediction = (x_new_norm/s)'*beta + bias;  %% 4 x 1 double output??
prediction_label = prediction > 0

prediction_label = 4x1 logical array
   1
   1
   1
   1

Thank you

[minhyuk jeung]

방금 연구원님 블로그보고 한국어로 질문하는게 나을것 같다고 판단되서 질문 다시 올립니다 ㅠ

현재 SVM regrssion을 사용하고 있으며 4개의 input 그리고 1개의 output을 가진 자료를 활용하고있습니다.

제가 하고싶은 부분은 SVM을 training 한 후에 새로운 input data가 들어왔을때 Y를 어떻게 예측하는지 증명하고자 합니다는 (Y = x1*a + x2*b + x3*c + x4*d 처럼 수식으로 나타낼수 있다면 가장 좋겠지만 이게 가능할지 모르겠습니다..).

따라서 위와 같은 질문을 한 것인데 연구원님께서 주신 코드를 활용할 경우 "prediction" 변수가 1개의 값이 아닌 (1개의 output) 4 x 1의 특성을 가진 값으로 나오게 됩니다.

SVM에 대해 이해하고 있지 못하는 점이 많아 진도가 빠르게 안나가네요 ㅠ

도움 주시면 감사드리겠습니다.

[Angelo Yeo]

안녕하세요. 아래와 같이 진행해보시면 좋을 것 같습니다.

inputdata = readmatrix('inputdata.xlsx', Sheet = "Sheet1");
outputdata = readmatrix('outputdata.xlsx', Sheet = "Sheet1");

predictor = inputdata(1:100,1:4);  % 4 input variable
target_variable = outputdata(1:100,1); % 1 output variable

X = predictor;
Y = target_variable;

Mdl = fitrsvm(X,Y,"Standardize",true);
pred_val = predict(Mdl, X);

%% For the new prediction

x_new = [8.9 10.2 42.8 44.8]; % New input variables (4 variables)


s = Mdl.KernelParameters.Scale;
beta = Mdl.Beta;
bias = Mdl.Bias;

x_new_norm = (x_new - Mdl.Mu) ./ Mdl.Sigma ;

x_new_norm/s * beta + bias % compare this and the predict value below

ans = 0.0714

Mdl.predict(x_new) 

ans = 0.0714

SVR에 대한 자세한 설명은 아래 문서에서도 확인하실 수 있습니다.

서포트 벡터 머신 회귀 이해하기 - MATLAB & Simulink - MathWorks 한국

[minhyuk jeung]

위 코드로 실행해보니 해결되었습니다.

감사합니다!! ㅠ