Adam: A Method for Stochastic Optimization on ShortScience.org

arxiv.org
scholar.google.com

Adam: A Method for Stochastic Optimization
Kingma, Diederik P. and Ba, Jimmy
arXiv e-Print archive - 2014 via Local Bibsonomy
Keywords: dblp

Summaries/Notes 2

[link] Summary by Joseph Paul Cohen 8 years ago

Adam is like RMSProp with momentum. The (simplified) update [[Stanford CS231n]](https://cs231n.github.io/neural-networks-3/#ada) looks as follows:

```
m = beta1*m + (1-beta1)*dx
v = beta2*v + (1-beta2)*(dx**2)
x += - learning_rate * m / (np.sqrt(v) + eps)
```

Your comment:

Write your summary here (You can use $\LaTeX$ and markdown syntax):

Anon Private