Neural Factorization Machines for Predictive Analytics

25 May, 2020 by NExT

Patent Name: Neural Factorization Machines for Predictive Analytics

Filing No: 10201704120Q (SG)

Filing Date: 19 May 2017

Country to be Filed: Singapore

Description: Many predictive tasks of web applications need to model categorical variables, such as user IDs and demographics like genders and occupations. To apply standard machine learning techniques, these categorical predictors are always converted to a set of binary features via one-hot encoding, making the resultant feature vector highly sparse. To learn from such sparse data effectively, it is crucial to account for the interactions between features. Factorization Machines (FMs) are a popular solution for efficiently using the second-order feature interactions. However, FM models feature interactions in a linear way, which can be insufficient for capturing the non-linear and complex inherent structure of real-world data. While deep neural networks have recently been applied to learn non-linear feature interactions in industry, such as the Wide&Deep by Google and DeepCross by Microsoft, the deep structure meanwhile makes them difficult to train. In this paper, we propose a novel model Neural Factorization Machine (NFM) for prediction under sparse settings. NFM seamlessly combines the linearity of FM in modelling second-order feature interactions and the non-linearity of neural network in modelling higher-order feature interactions. Conceptually, NFM is more expressive than FM since FM can be seen as a special case of NFM without hidden layers. Empirical results on two regression tasks show that with one hidden layer only, NFM significantly outperforms FM with a 7.3% relative improvement. Compared to the recent deep learning methods Wide&Deep and DeepCross, our NFM uses a shallower structure but offers better performance, being much easier to train and tune in practice.

Patents

QLive: Low Latency Live Streaming Solution

25 May, 2020

Advanced Conversational Recommender System

25 May, 2020

Multi-Channel Graph Neural Network (MuGNN)

25 May, 2020

Food Recognition Enhanced Using Privileged Information

25 May, 2020

Knowledge Enhanced Translation-based User Preference Model (KTUP)

25 May, 2020

A Contextual Relation Networks for Mixed-Dish Recognition

25 May, 2020

Tree Enhanced Embedding Model Predictive Analysis Methods and Systems

25 May, 2020

Machine Learning Using Partial Order Hypergraphs

25 May, 2020

Video Visual Relation Detection

25 May, 2020

Object Trajectory Proposal

25 May, 2020

Neural Factorization Machines for Predictive Analytics

25 May, 2020

Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

22 May, 2020

Back