The paper suggests using a differentiable function which can smoothly interpolate between multiplicative and additive gates in neural networks.  It is an intriguing idea and the paper is well written.  The mathematical ideas introduced are perhaps not novel (a cursory search seems to indicate that Abel's functional equation with f=exp is called the tetration equation and its solution called the iterated logarithm), but their use in machine learning seem to be.