NMF aims to find two matrices W and H, such that V=W H.There are two cost functions as follows:
Least-squares error: $||V – WH||^2$
Divergence: $D(V || WH)$
However, when we try to optimize the the functions $||V – WH||^2$ and $D(V || WH)$. They are convex in W only or H only, and they are not convex in both variables together.
To solve this problem, the authors build a new update rules called "multiplicative versus additive update rules" and $||V – WH||^2$ and $D(V || WH)$ are non-increasing under the update rules.
Multiplicative versus additive update rules: $H_{\alpha \mu}\leftarrow H_{\alpha \mu} + \eta_{\alpha \mu}\bigg[\sum\limits_{i} W_{i\alpha}\frac{V_{i\mu}}{(WH)_{i\mu}}-\sum\limits_{i} W_{i\alpha}\bigg]$