How specify which belong to which cluster using a training data in Fuzzy C-Mean clustering ?

Hi

I am fairly new with Matlab. Sorry for the long text but I hope that everything will be clear.

I am trying to build a fault detection (classification) system by using Fuzzy C-Mean clustering and membership grade. I performed principle component analysis to reduce the number of variable that describe the system to 3 principle components. I have training data for Fault Free condition in addition to 9 more faults (total 10). The system has nine operation conditions in which each operation condition can possibly have any of these faults (1 fault free + 9 faults = 10 in total).

Since I could not manage to do 3D clustering in the first place I started by dividing the process into two steps:

1. Identify the operation condition ( 9 operation conditions) using 2D clustering (using 1st and 2nd principle component).

2. Identify the faults (1 fault free + 9 faults = 10 in total) using 2D clustering (using 1st and 3nd principle components or 2st and 3nd principle components.

In the following script I tried to perform the first step:

    %Read data into matlab, the excel file is attached,

      clear all;

    %%Operation condition classifier
    %  

    [data8 text8 full8]=xlsread('FDD_Training_Clustering.xlsx');

    % seperate data by features
    f8=data8(:,1)';
    % count number of elements in each class
    tlen8=length(f8);

    t_xi8=data8(:,2:11);

    [coeff8,score8,latent8,tsquared8,explained8,mu8] = pca(t_xi8);
    bb8 = score8(:,1);

    x8=t_xi8*coeff8;
    x_3pca=x8(:,1:3); % All principle components
    x_pca1_2=x8(:,1:2); % 1st & 2nd principle components
    x_pca2_3=x8(:,2:3); % 2nd & 3rd principle components
    x_pca1_3=[x8(:,1) x8(:,3)]; % 1st & 3rd principle components

    % Operation condition classifier
    % In the training data each operation condition has 30 points
    ss1=30;ss2=30;ss3=30;ss4=30;ss5=30;ss6=30;ss7=30;ss8=30;ss9=30;

    % Operation condition sizes

    cc_1=1:ss1;
    cc_2=ss1+1:ss1+ss2;
    cc_3=ss1+ss2+1:ss1+ss2+ss3;
    cc_4=ss1+ss2+ss3+1:ss1+ss2+ss3+ss4;
    cc_5=ss1+ss2+ss3+ss4+1:ss1+ss2+ss3+ss4+ss5;
    cc_6=ss1+ss2+ss3+ss4+ss5+1:ss1+ss2+ss3+ss4+ss5+ss6;
    cc_7=ss1+ss2+ss3+ss4+ss5+ss6+1:ss1+ss2+ss3+ss4+ss5+ss6+ss7;
    cc_8=ss1+ss2+ss3+ss4+ss5+ss6+ss7+1:ss1+ss2+ss3+ss4+ss5+ss6+ss7+ss8;
    cc_9=ss1+ss2+ss3+ss4+ss5+ss6+ss7+ss8+1:ss1+ss2+ss3+ss4+ss5+ss6+ss7+ss8+ss9;

    %---------------IMPORTANT INPUT HERE!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    dd=3;% Set d' as needed (d can be 1,2 or 3) Keep it as 3 for now

    aii=x8;

    if dd==1
    % 1D
    x1_1dd = aii(cc_1);x2_1dd = aii(cc_2);x3_1dd = aii(cc_3);x4_1dd = aii(cc_4);x5_1dd = aii(cc_5);x6_1dd = aii(cc_6);x7_1dd = aii(cc_7);x8_1dd = aii(cc_8);x9_1dd = aii(cc_9);
    y1_1dd = ones(1,ss1);y2_1dd = ones(1,ss2);y3_1dd =ones(1,ss3);y4_1dd =ones(1,ss4);y5_1dd =ones(1,ss5);y6_1dd =ones(1,ss6);y7_1dd =ones(1,ss7);y8_1dd =ones(1,ss8);y9_1dd =ones(1,ss9);
    elseif dd==2
    % 2D
    x1_2dd = aii(cc_1,1);x2_2dd = aii(cc_2,1);x3_2dd = aii(cc_3,1);x4_2dd = aii(cc_4,1);x5_2dd = aii(cc_5,1);x6_2dd = aii(cc_6,1);x7_2dd = aii(cc_7,1);x8_2dd = aii(cc_8,1);x9_2dd = aii(cc_9,1);
    y1_2dd = aii(cc_1,2);y2_2dd = aii(cc_2,2);y3_2dd = aii(cc_3,2);y4_2dd = aii(cc_4,2);y5_2dd = aii(cc_5,2);y6_2dd = aii(cc_6,2);y7_2dd = aii(cc_7,2);y8_2dd = aii(cc_8,2);y9_2dd = aii(cc_9,2);
    elseif dd==3
    % 3D
    x1_3dd = aii(cc_1,1);x2_3dd = aii(cc_2,1);x3_3dd = aii(cc_3,1);x4_3dd = aii(cc_4,1);x5_3dd = aii(cc_5,1);x6_3dd = aii(cc_6,1);x7_3dd = aii(cc_7,1);x8_3dd = aii(cc_8,1);x9_3dd = aii(cc_9,1);
    y1_3dd = aii(cc_1,2);y2_3dd = aii(cc_2,2);y3_3dd = aii(cc_3,2);y4_3dd = aii(cc_4,2);y5_3dd = aii(cc_5,2);y6_3dd = aii(cc_6,2);y7_3dd = aii(cc_7,2);y8_3dd = aii(cc_8,2);y9_3dd = aii(cc_9,2);
    z1_3dd = aii(cc_1,3);z2_3dd = aii(cc_2,3);z3_3dd = aii(cc_3,3);z4_3dd = aii(cc_4,3);z5_3dd = aii(cc_5,3);z6_3dd = aii(cc_6,3);z7_3dd = aii(cc_7,3);z8_3dd = aii(cc_8,3);z9_3dd = aii(cc_9,3);
    end

The code below show a visual plot for the 3 principle components in 3d and a 2D plot of the first and second principle components in different colors and marks showing how the cluster should look like:

figure(1) % Shows all principle components
%Each fault in one group
plot3(x1_3dd,y1_3dd,z1_3dd,'or',x2_3dd,y2_3dd,z2_3dd,'xb',x3_3dd,y3_3dd,z3_3dd,'+g',x4_3dd,y4_3dd,z4_3dd,'*k',x5_3dd,y5_3dd,z5_3dd,'sc',x6_3dd,y6_3dd,z6_3dd,'dm',x7_3dd,y7_3dd,z7_3dd,'pm',x8_3dd,y8_3dd,z8_3dd,'hr',x9_3dd,y9_3dd,z9_3dd,'^b')
%Group all faults into only three groups
%plot3(x1_3d,y1_3d,z1_3d,'ob',x2_3d,y2_3d,z2_3d,'xg',x3_3d,y3_3d,z3_3d,'+b',x4_3d,y4_3d,z4_3d,'ob',x5_3d,y5_3d,z5_3d,'or',x6_3d,y6_3d,z6_3d,'xr',x7_3d,y7_3d,z7_3d,'xg',x8_3d,y8_3d,z8_3d,'+b',x9_3d,y9_3d,z9_3d,'+b')
legend('W30B-5','W30B0','W30B5','W40B-5','W40B0','W40B5','W50B-5','W50B0','W50B5')
title('3-D PCA Projection')
xlabel('First PCA Component')
ylabel('Second PCA Component')
zlabel('Third PCA Component')
grid on
axis vis3d

figure(2) % Shows the training data group into the based on each operation condition
%Each fault in one group
plot(x1_3dd,y1_3dd,'or',x2_3dd,y2_3dd,'xb',x3_3dd,y3_3dd,'+g',x4_3dd,y4_3dd,'*k',x5_3dd,y5_3dd,'sc',x6_3dd,y6_3dd,'dm',x7_3dd,y7_3dd,'pm',x8_3dd,y8_3dd,'hr',x9_3dd,y9_3dd,'^b')
%Group all faults into only three groups
%plot3(x1_3d,y1_3d,z1_3d,'ob',x2_3d,y2_3d,z2_3d,'xg',x3_3d,y3_3d,z3_3d,'+b',x4_3d,y4_3d,z4_3d,'ob',x5_3d,y5_3d,z5_3d,'or',x6_3d,y6_3d,z6_3d,'xr',x7_3d,y7_3d,z7_3d,'xg',x8_3d,y8_3d,z8_3d,'+b',x9_3d,y9_3d,z9_3d,'+b')
legend('W30B-5','W30B0','W30B5','W40B-5','W40B0','W40B5','W50B-5','W50B0','W50B5')
title('2-D PCA Projection')
xlabel('First PCA Component')
ylabel('Second PCA Component')
grid on
axis vis3d

Now after specifying the principle components we go to clustering:

%%FCM_Clestering

%Find 9 clusters using fuzzy c-means clustering.
options = [2 1000 0.00001 1];
[centers,U,obj_fcn] = fcm(x_pca1_2,9,options);

figure(4)
plot(x_pca1_2(:,1),x_pca1_2(:,2),'ob')

%Classify each data point into the cluster with the largest membership value.
maxU = max(U);
index1 = find(U(1,:) == maxU);
index2 = find(U(2,:) == maxU);
index3 = find(U(3,:) == maxU);
index4 = find(U(4,:) == maxU);
index5 = find(U(5,:) == maxU);
index6 = find(U(6,:) == maxU);
index7 = find(U(7,:) == maxU);
index8 = find(U(8,:) == maxU);
index9 = find(U(9,:) == maxU);

%Plot the clustered data and cluster centers.
figure(3)
plot(x_pca1_2(index1,1),x_pca1_2(index1,2),'ob')
hold on
plot(x_pca1_2(index2,1),x_pca1_2(index2,2),'or')
plot(x_pca1_2(index3,1),x_pca1_2(index3,2),'ok')
plot(x_pca1_2(index4,1),x_pca1_2(index4,2),'og')
plot(x_pca1_2(index5,1),x_pca1_2(index5,2),'om')
plot(x_pca1_2(index6,1),x_pca1_2(index6,2),'sb')
plot(x_pca1_2(index7,1),x_pca1_2(index7,2),'sr')
plot(x_pca1_2(index8,1),x_pca1_2(index8,2),'sk')
plot(x_pca1_2(index9,1),x_pca1_2(index9,2),'sm')

plot(centers(1,1),centers(1,2),'xb','MarkerSize',15,'LineWidth',3)
plot(centers(2,1),centers(2,2),'xr','MarkerSize',15,'LineWidth',3)
plot(centers(3,1),centers(3,2),'xk','MarkerSize',15,'LineWidth',3)
plot(centers(4,1),centers(4,2),'xg','MarkerSize',15,'LineWidth',3)
plot(centers(5,1),centers(5,2),'xm','MarkerSize',15,'LineWidth',3)
plot(centers(6,1),centers(6,2),'xb','MarkerSize',15,'LineWidth',3)
plot(centers(7,1),centers(7,2),'xr','MarkerSize',15,'LineWidth',3)
plot(centers(8,1),centers(8,2),'xk','MarkerSize',15,'LineWidth',3)
plot(centers(9,1),centers(9,2),'xm','MarkerSize',15,'LineWidth',3)
hold off
grid on

The clusters I receive are different each time!

in the this step I already know which point belong to which cluster but sometimes the clusters I get are not correct. I only need to know the center of each cluster from this training step so I can use them to classify the testing data later.

My questions are as the following:

1. Is there a 3D Fuzzy C-Mean technique I can use for that, that allow me to built 3D clusters with my training data and then use these clusters to classify the testing data input points using the membership grade of each point with respect to each cluster?

2. How can I restrict the points to a specific cluster in the training step?

3. After building the cluster, can I choose a certain shape (preferably circles or ellipses) for the clusters or is it automatic shaping and how can I plot the border of the cluster around it?

4. After building the cluster, can I introduce some kind weighting parameter so that I can enlarge the cluster a bit if needed?

5. Do you recommend any classification technique using the membership grade with respect of each cluster after building the clusters correctly?

I really appreciate your help.

[M.Abuasbeh]

Hi Walter,

I am very sorry if my question caused any inconvenience for anyone.

I would definitely not put it that way if I knew I could cause any inconvenience for anyone.

I appreciate any help I can get.

Thanks in advance!