## Multivariate Normal Regression Types

### Regressions

Each regression function has a specific operation. This section shows how to use these functions to perform specific types of regressions. To illustrate use of the functions for various regressions, “typical” usage is shown with optional arguments kept to a minimum. For a typical regression, you estimate model parameters and residual covariance matrices with the `mle` functions and estimate the standard errors of model parameters with the `std` functions. The regressions “without missing data” essentially ignore samples with any missing values, and the regressions “with missing data” ignore samples with every value missing.

### Multivariate Normal Regression

Multivariate normal regression, or MVNR, is the “standard” implementation of the regression functions in Financial Toolbox™ software.

### Multivariate Normal Regression Without Missing Data

Estimate parameters using `mvnrmle`:

```[Parameters, Covariance] = mvnrmle(Data, Design); ```

Estimate standard errors using `mvnrstd`:

`StdParameters = mvnrstd(Data, Design, Covariance);`

### Multivariate Normal Regression With Missing Data

Estimate parameters using `ecmmvnrmle`:

```[Parameters, Covariance] = ecmmvnrmle(Data, Design); ```

Estimate standard errors using `ecmmvnrstd`:

`StdParameters = ecmmvnrstd(Data, Design, Covariance);`

### Least-Squares Regression

Least-squares regression, or LSR, sometimes called ordinary least-squares or multiple linear regression, is the simplest linear regression model. It also enjoys the property that, independent of the underlying distribution, it is a best linear unbiased estimator (BLUE).

Given m = `NumSamples` observations, the typical least-squares regression model seeks to minimize the objective function

`$\sum _{k=1}^{m}{\left({Z}_{k}-{H}_{k}b\right)}^{T}\left({Z}_{k}-{H}_{k}b\right),$`

which, within the maximum likelihood framework of the multivariate normal regression routine `mvnrmle`, is equivalent to a single-iteration estimation of just the parameters to obtain `Parameters` with the initial covariance matrix `Covariance` held fixed as the identity matrix. In the case of missing data, however, the internal algorithm to handle missing data requires a separate routine `ecmlsrmle` to do least-squares instead of multivariate normal regression.

### Least-Squares Regression Without Missing Data

Estimate parameters using `mvnrmle`:

```[Parameters, Covariance] = mvnrmle(Data, Design, 1); ```

Estimate standard errors using `mvnrstd`:

`StdParameters = mvnrstd(Data, Design, Covariance);`

### Least-Squares Regression With Missing Data

Estimate parameters using `ecmlsrmle`:

```[Parameters, Covariance] = ecmlsrmle(Data, Design); ```

Estimate standard errors using `ecmmvnrstd`:

`StdParameters = ecmmvnrstd(Data, Design, Covariance);`

### Covariance-Weighted Least Squares

Given m = `NUMSAMPLES` observations, the typical covariance-weighted least squares, or CWLS, regression model seeks to minimize the objective function

`$\sum _{k=1}^{m}{\left({Z}_{k}-{H}_{k}b\right)}^{T}{C}_{0}\left({Z}_{k}-{H}_{k}b\right)$`

with fixed covariance C0.

In most cases, C0 is a diagonal matrix. The inverse matrix $W={C}_{0}^{-1}$ has diagonal elements that can be considered relative “weights” for each series. Thus, CWLS is a form of weighted least squares with the weights applied across series.

### Covariance-Weighted Least Squares Without Missing Data

Estimate parameters using `mvnrmle`:

```[Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0); ```

Estimate standard errors using `mvnrstd`:

`StdParameters = mvnrstd(Data, Design, Covariance);`

### Covariance-Weighted Least Squares With Missing Data

Estimate parameters using `ecmlsrmle`:

```[Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0); ```

Estimate standard errors using `ecmmvnrstd`:

`StdParameters = ecmmvnrstd(Data, Design, Covariance);`

### Feasible Generalized Least Squares

An ad hoc form of least squares that has surprisingly good properties for misspecified or nonnormal models is known as feasible generalized least squares, or FGLS. The basic procedure is to do least-squares regression and then to do covariance-weighted least-squares regression with the resultant residual covariance from the first regression.

### Feasible Generalized Least Squares Without Missing Data

Estimate parameters using `mvnrmle`:

```[Parameters, Covariance] = mvnrmle(Data, Design, 2, 0, 0); ```

or (to illustrate the FGLS process explicitly)

```[Parameters, Covar0] = mvnrmle(Data, Design, 1); [Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0); ```

Estimate standard errors using `mvnrstd`:

`StdParameters = mvnrstd(Data, Design, Covariance);`

### Feasible Generalized Least Squares With Missing Data

Estimate parameters using `ecmlsrmle`:

```[Parameters, Covar0] = ecmlsrmle(Data, Design); [Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0); ```

Estimate standard errors using `ecmmvnrstd`:

`StdParameters = ecmmvnrstd(Data, Design, Covariance);`

### Seemingly Unrelated Regression

Given a multivariate normal regression model in standard form with a `Data` matrix and a `Design` array, it is possible to convert the problem into a seemingly unrelated regression (SUR) problem by a simple transformation of the `Design` array. The main idea of SUR is that instead of having a common parameter vector over all data series, you have a separate parameter vector associated with each separate series or with distinct groups of series that, nevertheless, share a common residual covariance. It is this ability to aggregate and disaggregate series and to perform comparative tests on each design that is the power of SUR.

To make the transformation, use the function `convert2sur`, which converts a standard-form design array into an equivalent design array to do SUR with a specified mapping of the series into `NUMGROUPS` groups. The regression functions are used in the usual manner, but with the SUR design array instead of the original design array. Instead of having `NUMPARAMS` elements, the SUR output parameter vector has `NUMGROUPS` of stacked parameter estimates, where the first `NUMPARAMS` elements of `Parameters` contain parameter estimates associated with the first group of series, the next `NUMPARAMS` elements of `Parameters` contain parameter estimates associated with the second group of series, and so on. If the model has only one series, for example, `NUMSERIES` = `1`, then the SUR design array is the same as the original design array since SUR requires two or more series to generate distinct parameter estimates.

Given `NUMPARAMS` parameters and `NUMGROUPS` groups with a parameter vector (`Parameters`) with ```NUMGROUPS * NUMPARAMS``` elements from any of the regression routines, the following MATLAB® code fragment shows how to print a table of SUR parameter estimates with rows that correspond to each parameter and columns that correspond to each group or series:

```fprintf(1,'Seemingly Unrelated Regression Parameter Estimates\n'); fprintf(1,' %7s ',' '); fprintf(1,' Group(%3d) ',1:NumGroups); fprintf(1,'\n'); for i = 1:NumParams fprintf(1,' %7d ',i); ii = i; for j = 1:NumGroups fprintf(1,'%12g ',Param(ii)); ii = ii + NumParams; end fprintf(1,'\n'); end fprintf(1,'\n');```

### Seemingly Unrelated Regression Without Missing Data

Form a SUR design using `convert2sur`:

```DesignSUR = convert2sur(Design, Group); ```

Estimate parameters using `mvnrmle`:

```[Parameters, Covariance] = mvnrmle(Data, DesignSUR); ```

Estimate standard errors using `mvnrstd`:

`StdParameters = mvnrstd(Data, DesignSUR, Covariance);`

### Seemingly Unrelated Regression With Missing Data

Form a SUR design using `convert2sur`:

```DesignSUR = convert2sur(Design, Group); ```

Estimate parameters using `ecmmvnrmle`:

```[Parameters, Covariance] = ecmmvnrmle(Data, DesignSUR); ```

Estimate standard errors using `ecmmvnrstd`:

`StdParameters = ecmmvnrstd(Data, DesignSUR, Covariance);`

### Mean and Covariance Parameter Estimation

Without missing data, you can estimate the mean of your `Data` with the function `mean` and the covariance with the function `cov`. Nevertheless, the function `ecmnmle` does this for you if it detects an absence of missing values. Otherwise, it uses the ECM algorithm to handle missing values.

Estimate parameters using `ecmnmle`:

```[Mean, Covariance] = ecmnmle(Data); ```

Estimate standard errors using `ecmnstd`:

```StdMean = ecmnstd(Data, Mean, Covariance); ```