RecSys 2020: Highlights of a Special Conference

The 14th ACM Conference on Recommender Systems was special in many ways: a fully virtual conference that did an amazing job to keep social interaction alive – even with many people being far away location- and time-wise. And on the other hand: never before have biases and fairness been so prevalent and heavily addressed. This blogpost guides you through the conference, provides useful links to presentations, materials and papers. So, enjoy the conference (again).

Planned to take place in Rio de Janeiro (marking South America’s hosting debut) this year’s RecSys ran from September 22nd to 26th of which the first 3 days main conference took place, followed by two days filled with tutorials and workshops. Day 0—as always—was reserved for the doctoral symposium. The overall South and Latin American RecSys experience was spearheaded by the the Latin American School on Recommender Systems in October 2019.

RecSys 2020 in a Few Numbers

The conference took place fully virtual. Live paper, poster, and tutorial sessions were running twice 11h apart from each other to accommodate presenters and participants from all time zones across the globe. However, the conference remains single-track allowing everyone to see and participate in everything.

• 1200+ participants: 64% from industry and 36% from academia, 21% students
• Main Conference: 3 Keynotes, 9 Paper Sessions, 12 Social Sessions, 3 Poster Sessions
• 6 tutorials, 12 workshops
• 39/218 Long Paper Submissions accepted (18%)—new submission record!
• 26/128 Short Paper Submissions accepted (20%)
• 2/7 Papers accepted (29%) for newly introduced Reproducibility Track
• 25% of Submissions coming form the USA
• Topic Focus shifted from Algorithms (47% vs. 26% in 2020) to Applications (17% vs. 39% in 2020)
• 10/20 Industrial Contributions accepted
• 46.6km Total Distance run by Linus Dietz making him the recipient of the Best Runner Award

Keynotes on Manipulation, Biases and Conversational AI Agents

Each of the main conference days was opened by a keynote speech:

Filippo Menczer: 4 Reasons why Social Media Make us Vulnerable to Manipulation

On day 1 Filippo Menczer from the Observatory on Social Media at Indiana University talked about „4 Reasons why Social Media Make us Vulnerable to Manipulation“ (find the same talk on YouTube). Considering the social media echoes of recent elections in the US or the current Corona pandemic he presented interesting simulations and analyses outlining the four reasons as follows:

1. Manipulation
2. Platform Bias
3. Information Overload
4. Echo Chambers

With illustrations of content virality, in particular fake news propagation, and social bot behavior the talk was intriguing and quite specific. He also showed the correlation of quality and popularity under different settings of user attention and information overload along with certain information sharing patterns. Menczer also shared various interesting tools created in his lab. In his summary, he claimed that „the interplay of cognitive, social, and algorithmic biases makes us vulnerable to misinformation“ and that „social bots exploit these vulnerabilities“. It was a great reflection of this year’s many RecSys contributions that involved research on biases. Definitely worth a laugh was his comment on adult’s critical thinking competence:

Kids have less critical thinking than adults and—believe me—adults don’t have much of it.

Ricardo Baeza-Yates: Bias in Search and Recommender Systems

The keynote on day 2 perfectly followed the content of the first day. Professor Ricardo Baeza-Yates from Northeastern University presented „Bias in Search and Recommender Systems“ (very similar talk here). He structured the complex interplay of different kinds of biases that permeate the personalized web where search and recommender systems are the dominating forces. With overarching biases like activity, algorithmic and cognitive biases he drilled down on the intricacies arising with some vicious feedback loops from this complex interplay.

Fortunately, he also touched on de-biasing techniques, the relevance of different goals like diversity, serendipity, or novelty next to sole accuracy. He also made the case of more exploration without compromising long-term business goals. It was a warning but still encouraging talk.

Michelle Zhou: ‚You Really Get Me’—Conversational AI Agents That Can Truly Understand and Help Users

The last day of the main conference was opened by Michelle Zhou, co-founder and CEO of Juji Inc., an AI startup located in the Silicon Valley that works on responsible and empathetic Artificial Intelligence agents. With a set of live demos Michelle covered the technical advances of her framework:

• evidence-based personality inference
• model-based conversation generation.

There was also a discussion on real-world applications of these conversational agents. Unfortunately, there is no similar talk online, but you may find this recent interview by Conversational Components with her interesting.

Best Papers

The conference was again awarding prizes to the best long and short papers as well as to the best reviewers.

The best long paper award was given to „Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations“ by Tang et al. from Tencent for their approach that now improves the Tencent video recommender system. Their solution addresses negative transfer which describes the unfavorable compromising between multiple loosely correlated or even conflicted tasks in a recommender system. The authors describe the resulting phenomenon in state-of-the-art MTL models as seesaw phenomenon where some tasks are improved while sacrificing others. They propose a shared learning structure to improve shared learning efficiency and thus alleviate the seesaw phenomenon and negative transfer. With industrial and public benchmark datasets they provide evidence of their approach’s superiority compared to prior approaches.

„Exploiting Performance Estimates for Augmenting Recommendation Ensembles“ by Gustavo Penha and Rodrygo L. T. Santos received the best runner-up long paper award. In their work, they propose a personalized method to combine the base estimators of a recommender ensemble for achieving significantly higher accuracy in state-of-the-art ensemble recommender results. Hereto, they exploit historical user feedback to generate performance estimates which serve as personalized weights for the base recommenders when generating the ensemble recommendations.

Finally, the best short paper was „ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation“ by Mi et al. The authors present an approach for continuous updates of session-based recommenders alleviating the risk of catastrophical forgetting. Therefore, they determine and use a small set of representative historical sequence data (the exemplars) for replay when training on new data along with distillation loss.

Additional papers that piqued my interest and which are next on my reading list:

• Afchar and Hennequin (Deezer Research): Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction
• Sato et al. (Fuji Xerox): Unbiased Learning for the Causal Effect of Recommendation
• Huang et al.: Keeping Dataset Biases out of the Simulation: A Debiased Simulator for Reinforcement Learning based Recommender Systems
• Schnabel et al. (Microsoft): Debiasing Item-to-Item Recommendations With Small Annotated Datasets
• Rendle et al. (Google Research): Neural Collaborative Filtering vs. Matrix Factorization Revisited
• Li et al.: Cascading Hybrid Bandits: Online Learning to Rank for Relevance and Diversity
• Goldenberg et al. (Booking.com): Free Lunch! Retrospective Uplift Modeling for Dynamic Promotions Recommendation within ROI Constraints
• Aridor et al.: Deconstructing the Filter Bubble – User Decision-Making and Recommender Systems
• Saito: Doubly Robust Estimator for Ranking Metrics with Post-Click Conversions
• Wang et al.: Causal Inference for Recommender Systems
• Guo et al. (Twitter): Deep Bayesian Bandits: Exploring in Online Personalized Recommendations

Dominant Topics: Biases, Fairness, Causality, Bandits and Reinforcement Learning

In my personal view, the outcry of recent years regarding the narrow-minded focus on accuracy was heard by the community. Acknowledging biases and developing de-biasing techniques, looking beyond correlation and trying to model causal effects as well as addressing fairness and accountability are among the dominant topics of the conference. I believe that the RecSys research community is much more aware of these topics than general society gives it credit for. However, my view is biased towards what I saw at the conference and not what happens in general behind each system. But there is also evidence that addressing these issues drives beneficial long-term business goals and is therefore grounded in industry’s own interest.

This blogpost cannot summarize all the developments and tendencies, but it should serve as an entry point for those that couldn’t attend or those that want to recap. Writing about recap and thinking about perspectives there is a very informative and well-crafted overview by Justin Basilico from Netflix: „Recent Trends in Personalization at Netflix“

The trends he identifies for Netflix seem to nicely reflect great parts of the overall conference which is why I strongly recommend to have a look at the slides:

• Causality
• Bandits
• Reinforcement Learning
• Objectives
• Fairness
• Experience Personalization

Tutorials and Workshops to Practice Skills and Intensify Exchange

The fourth conference day was packed with 6 tutorials on the following advanced topics for RecSys practice:

• Conversational Recommendation Systems: Video
• Feature Engineering for Recommender Systems (Nvidia rapids.ai): Video and Code
• Counteracting Bias and Increasing Fairness in Search and Recommender Systems: Video, Slides and References
• Introduction to Bandits in Recommender Systems: Video and Code
• Bayesian Value Based Recommendation: A Modelling based Alternative to Proxy and Counterfactual Policy based Recommendation (Criteo): Video and Code
• Adversarial Learning for Recommendation: Applications for Security and Generative Tasks – Concept to Code: Video, Slides and Code

The last two days also offered a broad range of 12 interesting workshops to intensify specific topics in RecSys research:

• CARS: Workshop on Context-Aware Recommender Systems
• ComplexRec: Workshop on Recommendation in Complex Environments
• FAccTRec: Workshop on Responsible Recommendation
• fashionXrecsys: Workshop on Recommender Systems in Fashion and Retail
• HealthRecSys: Workshop on Health Recommender Systems
• ImpactRS: Workshop on the Impact of Recommender Systems
• IntRS: Joint Workshop on Interfaces and Human Decision Making for Recommender Systems
• OHARS: Workshop on Online Misinformation- and Harm-Aware Recommender Systems
• ORSUM: Workshop on Online Recommender Systems and User Modeling
• PodRecs: Workshop on Podcast Recommendations
• REVEAL: Workshop on Bandit and Reinforcement Learning from User Interactions
• RecSys Challenge 2020 Workshop

Please checkout the official RecSys website to be linked to your dedicated workshop.

Similar to last year, the REVEAL workshop attracted the most attention with more than 900 participants being around. You should definitely check it out. There was also the release of Open Bandit Pipeline – a python library for bandit algorithms and off-policy evaluation that was considered as one of the highlights of this workshop.

With respect to the RecSys Challenge 2020 Workshop have a look at the blogpost of the winning team from NVIDIA describing their solution here.

Social

The social aspect of the conference was by far the most challenging due to the fully virtual setting and the fact that social events play a crucial role at RecSys conferences. But, in retrospective, the virtual surrogate exceeded all my expectations! The Whova event app provided a simple, yet powerful single-entry point: checking on the agenda, connecting with peers, starting discussions, watching presentations, etc. worked like a charm when using the (Web-)App. However, this was just the beginning. The main driver of remote personal connection and joy was gather.town – a Pokémon-like 2D virtual environment with small avatars that you can navigate. Simply running into a bunch of other avatars with the participants names attached launches an instant video-conference as an overlay including all people within your neighborhood. Poster-sessions with interactive pdf-overlays and connecting to the presenter when stepping on their poster area rug, bumping into someone and having a small chat or taking place at a table for a private conversation, or inviting people to your house or—more realistic concerning Rio de Janeiro—beach, everything really brought a sense of connecting with your fellows or meeting new ones. It was a blast. This little tool really sparked joy for people that would have loved to meet in person because it almost felt like it.

This made it also far more enjoyable to launch our remote Karaoke session from it, attend a Cocktail Mixing Session or simply listen to some live music. Dear organizing committee: you did a fantastic job!

Further Useful Links

• All papers are freely available in the Conference Proceedings
• RecSys 2020 Challenge by Twitter
• Google Brain releases TensorFlow Recommenders
• Re-release of Spotify Million Playlist Dataset on AIcrowd
• Further upcoming material will follow here – stay tuned!

RecSys 2021 in Amsterdam

Packed with all these information and nice experiences the global RecSys community is looking forward to its 15th gathering in 2021. Personally, I hope that the corona pandemic will be well controlled then to allow for a presence event. If so, the global RecSys community will meet from September 27th to October 1st in Amsterdam, at the Amsterdam Conference Center to be precise—the former building of the Amsterdam Stock Exchange. Therefore, its slogan comes as no surprise: „A place to meet and exchange“. See you there hopefully.