Neural Decision Tree Towards Fully Functional Neural Graph.

Though the traditional algorithms could be embedded into neural architectures with the proposed principle of [H. Xiao, Hungarian layer: Logics empowered neural architecture, arXiv: 1712.02555], the variables that only occur in the condition of branch could not be updated as a special case. To tackle this issue, we multiply the conditioned branches with Dirac symbol (i.e., 1(x>0)), then approximate Dirac symbol with the continuous functions (e.g., 1 - e(-alpha vertical bar x vertical bar)). In this way, the gradients of condition-specific variables could be worked out in the back-propagation process, approximately, making a fully functional neural graph. Within our novel principle, we propose the neural decision tree (NDT), which takes simplified neural networks as decision function in each branch and employs complex neural networks to generate the output in each leaf. Extensive experiments verify our theoretical analysis and demonstrate the effectiveness of our model.