yongzhuang

NMF aims to find two matrices W and H, such that V=W H.There are two cost functions as follows:

Least-squares error: $||V – WH||^2$

Divergence: $D(V || WH)$

However, when we try to optimize the  the functions $||V – WH||^2$ and  $D(V || WH)$. They are convex in W only or H only, and they are not convex in both variables together. 
To solve this problem, the authors build a new update rules called "multiplicative versus additive update rules"  and  $||V – WH||^2$ and $D(V || WH)$  are non-increasing under the update rules. 

Multiplicative versus additive update rules: $H_{\alpha \mu}\leftarrow H_{\alpha \mu} + \eta_{\alpha \mu}\bigg[\sum\limits_{i} W_{i\alpha}\frac{V_{i\mu}}{(WH)_{i\mu}}-\sum\limits_{i} W_{i\alpha}\bigg]$