MATLAB Answers

LSTM padding and masking

35 views (last 30 days)
Ao Du
Ao Du on 10 Dec 2020
Commented: Yildirim Kocoglu on 16 Jan 2021
I am solving a sequence-to-sequence classification problem based on LSTM using Matlab 2020b. The sequences have varaible length so padding within each minibatch is needed. However, I am not sure if Matlab automatically do the masking when calculating the crossentroy loss as well as the training/validation accuracy. From the training plot, the reported accuracy (around 70%) is much lower than those manually calculated by using checkpoints (where I get around 90% accuracy). I suspect although Matlab 2020b supports sequence padding and validation data in LSTM, it still did not offer the option of masking to reduce the influence caused by padding. Any insights?

  0 Comments

Sign in to comment.

Answers (1)

Aditya Patil
Aditya Patil on 22 Dec 2020
Currently, masking is not supported in MATLAB. I have brought the request to the notice of concerned people.
As a workaround, you can sort the inputs so that the amount of padding required is minimized. You may also set the minibatch size to 1, so that no padding is required.

  1 Comment

Yildirim Kocoglu
Yildirim Kocoglu on 16 Jan 2021
Thank you! I was really curious about this as well since it can be done in python. I really hope they can add this feature.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by