How padded sequences given as packed sequences are dealt by RNN in pytorch?

Wasi Ahmad Source

In pytorch, we can give a packed sequence as an input to the RNN. From official doc, input of an RNN can be as follows.

input (seq_len, batch, input_size): tensor containing the features of the input sequence. The input can also be a packed variable length sequence.

Example

packed = torch.nn.utils.rnn.pack_padded_sequence(embedded, input_lengths)
outputs, hidden = self.rnn(packed, hidden)
outputs, output_lengths = torch.nn.utils.rnn.pad_packed_sequence(outputs)

Here, embedded is the embedded representation of a batch input.

My question is, how the computation is carried out for packed sequences in RNN? How the hidden states are computed for padded sequences in a batch through packed representation?

pytorch

Answers

answered 11 months ago jsl #1

Based on the answer by matthew_zeng of this relevent question: output are not calculated for the padded elements, and hidden will be the hidden state after the last valid input.

comments powered by Disqus