• torch.nn.Embedding是一个简单的查询表，它存储固定字典和大小的嵌入项。此模块通常用于存储词嵌入并使用索引检索它们。模块的输入是一个索引列表，输出是相应的词embeddings。num_embeddings 表示单词的总数目， embedding_dim 表示词嵌入的维度，词嵌入就相当于是一个大矩阵，矩阵的每一行表示一个单词

torch.nn.Embedding(num_embeddings, embedding_dim, 
padding_idx=None, max_norm=None, norm_type=2.0, 
scale_grad_by_freq=False, sparse=False, _weight=None)

输入输出大小

代码

>>> embedding=nn.Embedding(12,3)输入一个批次2个样本 每一个样本4个值
>>> input = torch.LongTensor([[1,2,4,5],[4,3,2,9]])input形状大小
>>>> input.shape
torch.Size([2, 4])输出
>>>embedding_output=embedding（input）
>>> embedding_output
tensor([[[ 0.5579, -0.0065,  0.6583],[ 1.4306,  0.9306, -1.5018],[ 1.2549,  0.0773, -0.5379],[-0.0705, -0.1410,  0.8684]],[[ 1.2549,  0.0773, -0.5379],[ 0.1238,  0.7192, -1.4713],[ 1.4306,  0.9306, -1.5018],[ 1.2244,  1.2920,  1.5501]]], grad_fn=<EmbeddingBackward>)输出形状大小
>>> embedding_output.size()
torch.Size([2, 4, 3])词向量访问>>> embedding(input)
tensor([[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]]], grad_fn=<EmbeddingBackward>)
>>> embedding_output=embedding(input)
>>> embedding_output[0]
tensor([[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]], grad_fn=<SelectBackward>)>>> embedding_output[0][0]
tensor([ 0.1910, -0.0026, -1.4159], grad_fn=<SelectBackward>)>>> embedding_output.data
tensor([[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]],[[ 0.1910, -0.0026, -1.4159],[ 0.6250, -0.6104,  0.0703]]])nn.Embedding加入参数padding_idx
padding_idx---如果加入该参数，则在输出遇到索引时，用padding_idx(初始化为零)的嵌入向量填充输出--padding_idx=0
>>> embedding = nn.Embedding(10, 3, padding_idx=0)
>>> input = torch.LongTensor([[1,2,3,4,5,0]])
>>> input
tensor([[1, 2, 3, 4, 5, 0]])
>>> embedding(input)
tensor([[[-1.8560, -0.1569,  1.6875],[-0.5221, -0.4287, -0.8269],[-0.1110, -0.3052, -0.6554],[ 0.4090, -0.0836,  1.3469],[-0.4474,  0.9179, -0.7385],[ 0.0000,  0.0000,  0.0000]]], grad_fn=<EmbeddingBackward>)--padding_idx=5>>> embedding = nn.Embedding(10, 3, padding_idx=5)
>>> embedding(input)
tensor([[[-0.5237, -1.0189, -1.1322],[-0.3238,  1.0482,  0.3465],[ 0.2802,  0.9517,  0.4519],[-1.2358,  1.0914, -0.3249],[ 0.0000,  0.0000,  0.0000],[-1.0445,  0.5659,  0.2044]]], grad_fn=<EmbeddingBackward>)