positionEmbeddingLayer
Description
A position embedding layer maps sequential or spatial indices to vectors. Use this layer in transformer neural networks to encode information about data positions in a sequence or image.
Creation
Syntax
Description
creates a position embedding layer and sets the layer
= positionEmbeddingLayer(outputSize
,maxPosition
)OutputSize
and
MaxPosition
properties.
creates a position embedding layer and sets the layer
= positionEmbeddingLayer(outputSize
,maxPosition
,Name=Value
)PositionDimension
, Name
, Parameters and Initialization, and Learning Rate and Regularization properties using one or
more name-value arguments.
Properties
Examples
Algorithms
References
[1] Gehring, Jonas, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. "Convolutional Sequence to Sequence Learning." In Proceedings of the 34th International Conference on Machine Learning - Volume 70, 1243–52. ICML’17. Sydney, NSW, Australia: JMLR.org, 2017
[2] Glorot, Xavier, and Yoshua Bengio. "Understanding the Difficulty of Training Deep Feedforward Neural Networks." In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 249–356. Sardinia, Italy: AISTATS, 2010. https://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf
[3] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification." In 2015 IEEE International Conference on Computer Vision (ICCV), 1026–34. Santiago, Chile: IEEE, 2015. https://doi.org/10.1109/ICCV.2015.123
Extended Capabilities
Version History
Introduced in R2023b