Why RNN need two bias vectors?

Wasi Ahmad Source

In pytorch RNN implementation, there are two biases, b_ih and b_hh. Why is this? Is it different from using one bias? If yes, how? Will it affect performance or efficiency?

pytorch

Answers

answered 11 months ago Haha TTpro #1

The formular in Pytorch Document in RNN is self-explained. That is b_ih and b_hh in the equation.

rnn

You may think that b_ih is bias for input (which pair with w_ih, weight for input) and b_hh is bias for hidden (pair with w_hh, weight for hidden)

comments powered by Disqus