panel ols with unbalanced data

I'm able to run an ols panel regression with balanced data, that is for every cross section j I have the same number of observations t in the time dimension. My problem is that now cross section j=1 has a different number of observation in the time dimension than cross section j=2. How can I write a general code using mvregress in order resolve this problem?

Answer 1

[Hang Qian]

Hi Alberto,

For an unbalanced panel data set, one may consider padding NaNs in the response variables for those cross-sections with fewer observations in the time dimension. For example, at j=1 there are 2 observations, at j=2 there is only one observation. By artificially creating a second equation with fake regressors but NaN in the response variable at j=2, an unbalance panel becomes a balanced one. MVREGRESS uses Expectation-Maximization (EM) to maximize the log likelihood function. The EM algorithm is friendly to missing values. I think RVREGRESS will work as usual in the presence of NaNs.

Regards,

Hang Qian

[Alberto]

Hi Hang, Thank you for you answer. My understanding is that because of MVREGRESS is friendly to missing value the only thing I have to do is to create balanced matries in the time series and cross section dimension by adding "NaN" for those cross sections that have less data on the time series dimension. Correct?

[Hang Qian]

Yes, you are right. MVREGRESS does not have any indicator variable for indexing the unbalanced panel data, so the workaround is to make the data artificially balanced. EM algorithm can use the conditional mean to make an educated guess (i.e., impute) on the missing values.

[Alberto]

Following your advice I've balanced my data in the time series dimension with "NaN" for the dependent variable Y (the Xs have no missing data). But what I get is a message error inside the function MVREGRESS that the covariance matrix "is not positive-definite". Do you think it is due to the fact that the ratio NaN to real data is 75% to 25% in the Y matrix?