question regarding https://ww​w.mathwork​s.com/help​/deeplearn​ing/ug/sol​ve-odes-us​ing-a-neur​al-network​.html

In the above mentioned demonstration, when I change the ODE (modelLoss.m) I don't get anything close to the actual solution. Why is that? It seems it only works for the specific ODE in the example.

Answer 1

[Manikanta Aditya]

Hi @Christos,

The issue you're encountering is likely due to the specific configuration and training of the neural network in the example. The modelLoss.m function and the network architecture are tailored to solve the particular ODE given in the example. When you change the ODE, the network might not be properly configured to handle the new equation.

• The loss function in modelLoss.m is designed for the specific ODE in the example. You need to modify the loss function to match the new ODE. Ensure that the loss function correctly penalizes deviations from the new ODE and its initial conditions.
• Generate training data that is suitable for the new ODE. The range and distribution of the training data should cover the domain of the new ODE.
• The neural network architecture might need adjustments to better fit the new ODE. Experiment with different network architectures, such as the number of layers and neurons, to improve the network's ability to approximate the solution.
• Adjust the training parameters, such as the learning rate, number of epochs, and batch size, to ensure the network converges to a good solution for the new ODE.

I hope this helps.

[Christos]

Thank you for your answer! The only way to adjust the network/parameters in order to get a better solution is to use an ODE for which an exact solution is known. What happens when the exact solution is not available? How does one know what are "good" parameters/architecture for such a problem?

[Manikanta Aditya]

@Christos, The challenge of adjusting network parameters and architecture without an exact solution is indeed significant.

• Use cross-validation techniques to evaluate different network configurations. This helps in assessing the model's performance and selecting the best parameters.
• Experiment with different architectures and training parameters. Iteratively refine your model based on validation performance.
• Perform systematic searches over a range of hyperparameters to find the optimal combination. These methods can help explore the hyperparameter space effectively.
• Utilize Bayesian optimization to find the best hyperparameters. This method builds a probabilistic model of the objective function and selects the most promising hyperparameters to evaluate.
• Apply regularization techniques like dropout, L2 regularization, or early stopping to prevent overfitting and improve generalization.
• If you have a neural network that works well for a similar ODE, you can use transfer learning. Start with the pre-trained network and fine-tune it on your new ODE.

Refer to the following references which can help you find some answers to your queries:

• Selecting the Best Architecture for Artificial Neural Networks - Fritz ai
• 3.5: How to choose a neural network's hyper-parameters? - Engineering LibreTexts