inversing huge Y Matrix

Hello, I need to get Zbus for some short circuit study. The system is huge consisting of over 9000 buses. From psse, I found the ybus which is quite sparse and inversing it is giving me wrong results. Could anyone please tell me how to inverse such huge matrix to get zbus?

[Steven Lord]

How exactly are Zbus and ybus related? [From your description is sounds like those are commonly used terms/variable names in your area of study, but please explain the relationship for those of us who aren't familiar with that area.]

How are you storing ybus? Is it a full matrix that has only a few non-zero elements (sparsely populated but not sparsely stored) or is it a sparse matrix (sparsely populated and sparsely stored)?

fullWithOnlyFewNonzero = eye(5)

fullWithOnlyFewNonzero = 5x5

1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

sparseWithOnlyFewNonzeros = speye(5)

sparseWithOnlyFewNonzeros = 
   (1,1)        1
   (2,2)        1
   (3,3)        1
   (4,4)        1
   (5,5)        1

whos full* sparse*

  Name                           Size            Bytes  Class     Attributes

  fullWithOnlyFewNonzero         5x5               200  double              
  sparseWithOnlyFewNonzeros      5x5               128  double    sparse    

Generally speaking, as John D'Errico said unless you absolutely positively explicitly need an inverse matrix you should not invert the matrix but should try an alternate approach. For example, if you're trying to solve a system of equations use the backslash operator \. [Yes, when you did this type of exercise in class while you were learning, you may have been told to invert. That's one of the cases where there's a difference between theory and practice.]

Answer 1

[John D'Errico]

As always, I'll start by saying that computing the inverse of a matrix is a bad idea. And of course, you don't say what you think is "wrong", nor do you provide the matrix. (Just attach it to a comment in the form of a .mat file. Since it is sparse as you claim, it won't take much space anyway to store.)

First, you need to understand if the matrix is of full rank. If not, then you cannot find the "right" result, as there are infinitely many solutions, if you would find any solutions at all.

You also need to learn about things like condition numbers. But without seeing the matrix itself, this answer would then turn into a full course on numerical linear algebra. And, while you would surely benefit from that since you are trying to compute the inverse of a matrix, Answers is not the place to teach it. So post the matrix in a .mat file, attached to a comment, and we can see if we can help you more.

[Matthew Worker]

I have the ymatrix in .csv file whose size is 396mb. So I can't attach it here. Since you are telling inversing y matrix to get z matrix is a bad idea, what do you recommend to get z matrix?

[John D'Errico]

Sigh. First, you tell us the matrix is sparse. SO LEARN TO USE SPARSE MATRICES! If it truly is sparse, then it will take far less than that. If it is 0.1% non-zero (a not uncommon measure when you talk about sparse matrices) then that 396 MB file will probably require on the order of 1/2 MB to store.

Next, As I said, you THINK you want to compute the inverse of the matrix. But almost always, even when you think that to be the case, you would be wrong. This comes down to learning some ideas about numerical linear algebra, about how to solve a system of equations in ways that do not force you to compute the inverse of a matrix.

Finally, you talk about the z matrix. We don't know what that means. There is a good chance z is the solution of a set of equations. But I cannot know that, as you are being far too vague. If so, then you might be willing to use tools like linsolve, or the simple backslash. Or you could work with the factors of your Y matrix, perhaps QR, LU, LDL, or Cholesky factors. Most of the time when someone says they need to compute the inverse of a matrix, they don't.

But before you do that, you need to understand about things like the numerical rank and condition number of your matrix, as trying to solve anything, if your matrix is singular may be wrong. (Then you would first need to understand why it is singular, but you still may be able to solve your problem.) And we don't know anything about those quantities, or even the actual size of your matrix.

[Matthew Worker]

I find your answer totally arrogant. I don't know why you are answering with such arrogancy while you can totally choose not answering any question if you don't want. I already checked the rank condition number etc and it is full rank with high condition number. And as I am talking about y matrix, I think it should be clear that Z matrix means impedance matrix. My point was finding z matrix, either by inversing y matrix or some other means. I told y matrix is quite sparse because lots of the elements of the y matrix are zero as it is a huge system and there is no connectivity in many pairs. So basically I was looking for suggestions in case anybody finds any issues like this and how they do compute z matrix in such case.

[Torsten]

And as I am talking about y matrix, I think it should be clear that Z matrix means impedance matrix.

No. For us, y and z are just letters without meaning. So z = inv(y) ? Or how can z be deduced from y ? What do you mean by "My point was finding z matrix, either by inversing y matrix or some other means." What are the "other means" apart from inverting ?

[Stephen23]

"My point was finding z matrix, either by inversing y matrix or some other means."

Then you should be asking about that:

https://xyproblem.info/

If the connections are modeled using systems of equations, then we can help to solve them efficiently, if you describe them. As John D'Errico correctly wrote, there are much better approaches for solving systems of equations than INV (e.g. MRDIVIDE or MLDIVIDE).

"I told y matrix is quite sparse because lots of the elements of the y matrix are zero as it is a huge system and there is no connectivity in many pairs."

You should probably be using SPARSE type. And avoid inverting matrices.

[Matthew Worker]

okay I should have mentioned my question is related to power systems. As I mentioned about buses, I thought it is clear that it is power systems related question. Z and Y are not any random variable. Ybus is admittance matrix and Zbus is impedance matrix.

[Stephen23]

@Matthew Worker: take a look at:

https://www.mathworks.com/help/matlab/ref/mldivide.html

https://www.mathworks.com/help/matlab/ref/mrdivide.html

[John D'Errico]

Use mathematics. Don't just call something an admittance matrix and an impedance matrix, and expect us to know what you are doing with them. HOW ARE Z AND Y RELATED IN TERMS OF MATHEMATICS? Maybe I can do some reading of my own:

http://www.ittc.ku.edu/~jstiles/723/handouts/section_4_2_Impedance_and_Admittance_Matricies_package.pdf

And in there, we see matrices called Z and Y, related by a matrix inverse. Regardless, that you see the inverse of a matrix does not mean you need to compute that INVERSE!!!!! This is perhaps the most common fallacy I see made in the area of linear algebra. What you will do with that inverse is what mattters, because you will literally never just compute the inverse of a matrix just to marvel at it, and say, what a pretty inverse that is!

For example, if you want to compute the inverse of Y, then somewhere you will multiply something by Z, the inverse of Y. We don't know what exactly you will do, but it will be something of that form. And that is part of the problem. What we might tell you to do will depend on that formula. It will also depend on the properties of that matrix.

And that gets us around to the last issue. You said you had some sort of problem with the result. Again, you were far too vague to be useful. Is the matrix singular? Is it merely close to singular, and because it is such a large matrix, it is effectively numerically singular? Have you made a mistake in how you built that matrix? Yeah, right, like that NEVER happens.

Now, I did try multiple times to get you to tell me what the rank of the matrix is. What is the condition number? I've asked that multiple times. Part of that could be resolved if you would post the matrix in a .mat file. And yes, that might require you do so as a sparse matrix. You did say it was sparse!

So, in terms of your matrix being a sparse one, again, you have been far too vague. Just saying there are a lot ot zeros tells us essentially nothing. My guess is, your matrix is not even close to being usefully sparse. But this just means you probably cannot post the matrix itself. If we cannot see the matrix, we cannot see what properties it has. And that makes it more difficult to help you.

If you would, please tell us these things:

size(Y) % What size is Y?
nnz(Y) % is the matrix truly sparse as you claim it to be?
rank(Y) % The numerical rank of Y. This tells us if MATLAB thinks the matrix is singular
cond(Y) % How close is the matrix to singularity?

Better yet, would be to make the matrix a sparse matrix, and then put it in a .mat file.

help sparse

I'm sorry, but just telling us that Z and Y are related to power systems is useless information. You are trying to solve a relatively large problem, but I think you are doing that without any understanding of the linear algebra. You have gotten into the deep end of the pool but have no idea how to swim. And that suggests you should probably be talking with your advisor, or perhaps a faculty member who does understand linear algebra.

[Matthew Worker]

first of all, i actually expected help from someone in power systems area and anybody from that area would have understood what I was talking about. That's why I didn't explain too much. If you don't understand from my question what I am talking about, why do u need to answer to it? you could just skip my question. As you are insisting on knowing more about it, I need Z matrix (impedance matrix) for calculating distance between all the bus pairs. The formula of distance needs impedance whereas Y matrix (admittance matrix) is easy to get from the available software. Also:

Size of Y: 8684 8684

The matrix is sparse.

Rank of Y: 7616

Condition number of Y: 1.032304698587612e+20

Thing is that I didn't beg you to help me. I don't know why you reply like this. You may be very good at your background but that doesn't mean you will have to answer to everybody if you can't talk politely with them. I agree that I should have written my question with some more details, but even if I wrote it so briefly, people who don't understand my question could simply ignore it rather than replying with such insolence.

[Sam Chak]

OP: I have the y-matrix in .csv file whose size is 396mb.

Hi @Matthew Worker,

I understand your frustration. Many of us are willing to assist with issues related to using MATLAB, even if the topic is not our area of expertise. Would you be able to upload the CSV file to cloud storage and share the file via a link?

Additionally, if possible, you can try transferring the CSV data to MATLAB and storing it in the MAT-file format. This way, you can check if the file size is significantly reduced. This example shows you how to save all variables from the workspace in a binary MAT-file.

https://www.mathworks.com/help/matlab/ref/save.html?s_tid=doc_ta#mw_683a1d95-e38c-4ed0-a21d-e3f80945cc7f

[Paul]

Perhaps an alternative approach would be to use the Paperclip icon on the Insert menu to upload a .mat file that contains a representative, but much smaller Ybus, and include the code to show how Zbus is to be derived and the operations on Zbus to find the the distances (it's my less-than-primitive understanding that different distance measures exist).

[Paul]

I have no idea if what follows is related to what @Matthew Worker is looking for.

Suppose we have a complex matrix Y, and Z = inv(Y).

syms Z [3 3]

Define the matrix D as follows:

for ii = 1:3
    for jj = 1:3
        D(ii,jj) = Z(ii,ii) + Z(jj,jj) - Z(ii,jj) - Z(jj,ii);
    end
end
D

D = 

Can D be obtained from Y without expliclity forming Z?

[Walter Roberson]

Size of Y: 8684  8684
The matrix is sparse.
Rank of Y: 7616

The rank of Y is less than the size of Y. The determinent of Y will be 0. You cannot take the inverse of Y.

[Sam Chak]

Hi @Matthew Worker

According to the Nodal Admittance Matrix on Wikipedia, for a three-bus network, the bus admittance matrix (Ybus) can be constructed as described. The bus impedance matrix (Zbus) can then be obtained by taking the inverse of Ybus. Is that what you have been doing for a sparse matrix Ybus?

%% admittances 
y1  = -5i;
y12 = -1.25i;
y13 = -2.5i;
y2  = -2.5i;
y23 = -2.5i;
y3  =  0i;

%% Bus admittance matrix
Y11 = y1 + y12 + y13;
Y12 = -y12;
Y13 = -y13;
Y22 = y2 + y12 + y23;
Y23 = -y23;
Y33 = y3 + y13 + y23;
Ybus= [Y11, Y12, Y13;
       Y12, Y22, Y23;
       Y13, Y23, Y33]

Ybus = 
   0.0000 - 8.7500i   0.0000 + 1.2500i   0.0000 + 2.5000i
   0.0000 + 1.2500i   0.0000 - 6.2500i   0.0000 + 2.5000i
   0.0000 + 2.5000i   0.0000 + 2.5000i   0.0000 - 5.0000i

%% Bus impedance matrix
Zbus= inv(Ybus)

Zbus = 
   0.0000 + 0.1600i   0.0000 + 0.0800i   0.0000 + 0.1200i
   0.0000 + 0.0800i   0.0000 + 0.2400i   0.0000 + 0.1600i
   0.0000 + 0.1200i   0.0000 + 0.1600i   0.0000 + 0.3400i

[Paul]

Hi Sam,

Obviously, I can't say for sure, but I think that's the idea. In this comment I posted the equation for the Thevenin Impedance Distance between nodes. I played around for a little bit, and it wasn't clear to me how to compute the distance matrix from Ybus without first explicitly computing Zbus.