∞-AE model's implementation in JAX. Kernel-only method outperforms complicated SoTA models with a closed-form solution and a single hyper-parameter.

Overview

Infinite Recommendation Networks (∞-AE)

This repository contains the implementation of ∞-AE from the paper "Infinite Recommendation Networks: A Data-Centric Approach" [arXiv] where we leverage the NTK of an infinitely-wide autoencoder for implicit-feedback recommendation. Notably, ∞-AE:

  • Is easy to implement (<50 lines of relevant code)
  • Has a closed-form solution
  • Has only a single hyper-parameter, $\lambda$
  • Even though simplistic, outperforms all complicated SoTA models

The paper also proposes Distill-CF: how to use ∞-AE for data distillation to create terse, high-fidelity, and synthetic data summaries for model training. We provide Distill-CF's code in a separate GitHub repository.

If you find any module of this repository helpful for your own research, please consider citing the below under-review paper. Thanks!

@article{sachdeva2022b,
  title={Infinite Recommendation Networks: A Data-Centric Approach},
  author={Sachdeva, Noveen and Dhaliwal, Mehak Preet and Wu, Carole-Jean and McAuley, Julian},
  journal={arXiv preprint arXiv:2206.02626},
  year={2022}
}

Code Author: Noveen Sachdeva ([email protected])


Setup

Environment Setup

pip install -r requirements.txt

Data Setup

Once you've correctly setup the python environment, the following script will download the ML-1M dataset and preprocess it for usage:

./prep.sh

How to train ∞-AE?

  • Edit the hyper_params.py file which lists all config parameters of ∞-AE.
  • Finally, type the following command to train and evaluate ∞-AE:
CUDA_VISIBLE_DEVICES=0 python main.py

Results sneak-peak

Below are the [email protected] results for the datasets used in the paper:

Dataset PopRec MF NeuMF MVAE LightGCN EASE ∞-AE
Amazon Magazine 8.42 13.1 13.6 12.18 22.57 22.84 23.06
MovieLens-1M 13.84 25.65 24.44 22.14 28.85 29.88 32.82
Douban 11.63 13.21 13.33 16.17 16.68 19.48 24.94
Netflix 12.34 12.04 11.48 20.85 Timed out 26.83 30.59*

Note: ∞-AE's results on the Netflix dataset (marked with a *) are obtained by training only on 5% of the total users. Note however, all other methods are trained on the full dataset.


MIT License

You might also like...

A Python/Windows port scanner that scans for open and closed ports on IP addresses.

A Python/Windows port scanner that scans for open and closed ports on IP addresses.

DeLevo Port Scanner If you found this application interesting or useful, please consider starring the repo above. Usage Download Python from here if y

Apr 18, 2022

Removes closed captions from .srt subtitles

Removes closed captions from .srt subtitles

.srt CC remover English subtitles are sometimes only available as the "CC" versions that include descriptions such as (LOUD EXPLOSION) to help hearing

Apr 7, 2022

This is the implementation of the paper AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models

This is the implementation of the paper AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models

AdaMix (Mixture-of-Adapter) This is the implementation of the paper AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models

Nov 26, 2022

DHGE: Dual-view Hyper-Relational Knowledge Graph Embedding for Link Prediction and Entity Typing.

DHGE-master DHGE: Dual-view Hyper-Relational Knowledge Graph Embedding for Link Prediction and Entity Typing. Requirements This project should work fi

Oct 12, 2022

code and links for the "write code" module for Hyper Island, 2022

Resources and code for "Writing Code" (Hyper Island) Slides Learning to code This and that FizzBuzz, much more than you ever wanted to know Hacker cul

Mar 11, 2022

Algo trading with strategy customization, genetic algorithm for hyper params optimizing, and backtesting.

Algo-Trading-with-Genetic-Algorithm Algo trading with strategy customization, genetic algorithm for hyper params optimizing, and backtesting. Run prep

Oct 15, 2022

Training ImageNet / CIFAR models with sota strategies and fancy techniques such as ViT, KD, NAS, Rep, etc.

Image Classification SOTA Image Classification SOTA is an image classification toolbox based on PyTorch. Updates March 24, 2022 Support training strat

Nov 22, 2022

SOTA Google's Perceiver-AR Music Transformer Implementations and Models

SOTA Google's Perceiver-AR Music Transformer Implementations and Models

Perceiver Music Transformer SOTA Google's Perceiver-AR Music Transformer Implementations and Models Multi-Instrumental Version Solo Piano Version Usef

Nov 7, 2022

Multi-framework implementation of Deep Kernel Shaping and Tailored Activation Transformations, which are methods that modify neural network models (and their initializations) to make them easier to train.

Official Python package for Deep Kernel Shaping (DKS) and Tailored Activation Transformations (TAT) This Python package implements the activation func

Nov 28, 2022
Owner
Noveen Sachdeva
CS PhD Student | Machine Learning Researcher
Noveen Sachdeva
Build password manager app using tkinter, by which we will be able to store passwords of websites and generate complicated passwords.

Build password manager app using tkinter, by which we will be able to store passwords of websites and generate complicated passwords. In this project also applied error handling, storing and usage of JSON data by which enabled a search button.

Arjunan K 0 Aug 27, 2022
Conatins the Python Script and .exe file for a overly complicated digital clock

ShortOnTime Conatins the Python Script and .exe file for a overly complicated digital clock Usage If you just want the clock and not the code then dow

null 10 Oct 26, 2022
Source code of KDD'22 paper: Learning to Rotate: Quaternion Transformer for Complicated Periodical Time Series Forecasting

Quatformer (KDD 2022 paper) Weiqi Chen, Wenwei Wang, Bingqing Peng, Qingsong Wen, Tian Zhou, Liang Sun, "Learning to Rotate: Quaternion Transformer fo

null 9 Nov 11, 2022
A Django based contact form, which has a simple form which is connected to a SQL-Lite database and has features to be added .

This is a Django based contact form, which has a simple form which is connected to a SQL-Lite database and has features to be added . It has an admin Interface to view the entires in the DB. Now, we can update the message board through the admin panel. NOTE: Not a production grade!

Nalin Rajendran 1 Sep 9, 2022
CLOOB training (JAX) and inference (JAX and PyTorch)

cloob-training Pretrained models There are two pretrained CLOOB models in this repo at the moment, a 16 epoch and a 32 epoch ViT-B/16 checkpoint train

Katherine Crowson 63 Nov 14, 2022
This module provides and interface between JAX and Pint to allow JAX to support operations with units.

JAX + Units Built with JAX and Pint! This module provides and interface between JAX and Pint to allow JAX to support operations with units. The propag

Dan Foreman-Mackey 27 Aug 10, 2022
Baselax (Baselines + JAX) provides stable-baselines-style implementations of reinforcement learning (RL) algorithms with Google JAX framework.

Status: in development, no practical use. Baselax Baselax (Baselines + jax) provides stable-baselines-style implementations of reinforcement learning

sicer 3 Sep 7, 2022
SBX: Stable Baselines Jax (SB3 + Jax)

Stable Baselines Jax (SB3 + Jax = SBX) Proof of concept version of Stable-Baselines3 in Jax. Implemented algorithms: Truncated Quantile Critics (TQC)

Antonin RAFFIN 75 Nov 23, 2022