Trying to complete a t-test between two 925x902 matrices to find if the difference between the two is significant to the 5% level

The two matrices are DEM and B where DEM is the standard matrices and B is the simulated one and thus the one the test needs to be carried out on to show how significantly different the two matrices are.

Trying to complete the test on the 5% confidence level

How would i complete this test?

[Star Strider]

That depends on what the data are in the matrices, what you’re testing, and how you want to do the test.

Are you comparing, rows, columns, or something else?

Statistical tests have assumptions (such as normality) so that not all tests are appropriate for a particular problem.

[Todd Pierce]

I am trying to compare the entire matrix,

the matrices are basically elevation data points and i am trying to compare the difference between the two matrices with respect to the elevation data and whether or not they are significant.

[dpb]

On a point-by-point basis you have no degrees of freedom from which to estimate the variance of either population -- and as a simulation, there is no variance in a deterministic model outputs.

As S-S says, isn't an appropriate use of the two-sample t-test.

[Todd Pierce]

what if converted the matrices into a double vector [834350x1] and then ran the test?

DEM=(DEM(:));
B=(B(:));

[h,p,ci,stats]=ttest2(DEM,B)

When i run this i get the following result:

h = 1
p = 0
ci =
   -9.0723
   -8.9748
   
stats = 
  struct with fields:
    tstat: -363.1155
       df: 1668698
       sd: 16.0506
       

Would this not be acceptable, and the results an accurate representation of the datasets in comparison?

[Jeff Miller]

This test is telling you that the average DEM value is about 9 points lower than the average B value, and that a difference of 9 points would virtually never occur by chance (p=0) assuming that the variations among DEM scores and among the B scores are just due to chance. On that basis you can conclude that the DEM and B matrices are definitely not statistically similar.

I presume that the variation among DEM scores is not just due to chance, however, due to real, systematic differences such as row and column effects (and similarly for the variation among B scores). Correcting for these systematic differences will make the p value even smaller (i.e., 0 to even more decimal places), so it is not going to strengthen your intended conclusion (as I understand it) that the matrices are similar.

[dpb]

And is, as the documentation for ttest2 says, a totally inappropriate test to apply to the subject data set -- to wit, "ttest2(x,y) returns a test decision for the null hypothesis that the data in vectors x and y comes from independent random samples from normal distributions with equal means and equal but unknown variances, ..."

There is no random sampling in sight in this problem definition; in fact the points are correlated identically in the grid of locations.

However, the returned test statistic of ~-9 probably illustrates that there is a bias between the model and the observed data; looking at the difference instead and for bias either by location or by magnitude of response could possibly lead to some useful observations.

Visualization of these differences would be a starting point.