How do I make my neural network consistent?

Hi,

I have a DSGE model, which I want to solve using neural network for my dynamic equations. First, I solved a simpler version of my model and everything was great. But when I tried to change slightly the model imposing zero lower bound (ZLB) (for neural network this change one of my input data) and then solve the model using the solution of simpler model as an initial step it fails and I got this kind of things. What can I change to make my neural network more consistent?

Answer 1

[Prasanna]

Hi Vahagn,

To improve the consistency of your neural network, when solving a DSGE model with a zero lower bound, you can perform the following steps:

• If your simpler model worked well, but the more complex one fails, consider increasing the depth or width of your neural network. More layers or neurons can help capture the additional complexity introduced by the ZLB.
• Implement techniques like dropout or L2 regularization to prevent overfitting
• Experiment with different learning rates and batch sizes. Smaller learning rates and batch sizes help the model converge and learn better under new constraints.
• If your model is failing to converge, consider modifying the loss function to better reflect the objectives under the ZLB. For instance, incorporating penalties for violating the ZLB could guide the model towards feasible solutions.
• If the neural network approach continues to struggle, consider hybrid methods that combine traditional DSGE solution techniques with neural networks. For example, using methods like 'Dynare' or 'Gensys' to solve the model first and then fine-tuning with a neural network.

For more information on the above, refer the following documentations:

• First order perturbation: https://mutschler.eu/dynare/perturbation/first-order-theory/
• Analyze linearized DSGE models: https://www.mathworks.com/help/econ/analyze-linearized-dynamic-stochastic-general-equilibrium-models.html

Hope this helps!