Skip to navigation Skip to main content

Day 1

Registration & Breakfast

Chairperson Opening Remarks

Chairperson Opening Remarks 

Judith Lee Associate Professor and Dept. Chair, Operations & IT Management Golden Gate University

Making AI Interpretable with Generative Adversarial Networks

Many of our experiences in daily life, from music recommendations to medical diagnosis, are influenced by AI and machine learning. Complex modeling algorithms outperform simpler models in predictive accuracy but at the expense of intepretability. In our talk, we share a framework that can generate explanations for the output of any type of machine learning model, from simpler models to morecomplex models such as ensembles and neural networks.

Juan Hernandez Senior Data Scientist Square

Machine Thinking

This talk focusses on the why and what of machine learning, as opposed to how of ML. Ashish will present his experiences with deploying multiple ML apps in production and talk about how to think and structure machine learning problems.

Ashish Bansal Senior Director, Data Science Capital One

Sprez.za·tu·ra

Sprezzatura, the art of becoming invisible. Within Netflix, personalization is a key differentiator, helping members to quickly discover new content that matches their taste. Done well, it creates an immersive user experience, however when the recommendation is out of tune, it is immediately noticed by our members. During the presentation, we focus on several of the algorithmic challenges related to the launch of new Netflix originals in the service, and go over concepts such as causality, incrementality and explore-exploit strategies.

Roelof van Zwol Director, Product Innovation Netflix

Networking Coffee Break

Software Engineering Best Practices for Building AI-enabled applications

Advancements in computational power, and algorithms has brought Artificial Intelligence (AI) to the forefront in the past decade, bringing AI out from 'AI winter'. Many companies and developers are actively exploring how AI can help their businesses, be it as chatbots in customer support scenarios, or doctors' assistants in hospitals, or legal research assistants in legal domain, or as marketing manager assistants in marketing, or as real-time face detection applications in security domain. Building high quality and scalable AI models or services takes specific kind of discipline, methodology and tools. While some of these processes, methods and tools overlap with traditional software engineering practices, several are new to AI domain. In this talk, I will share the best practices for building AI-enabled applications from personal experiences from running AI Operations team at IBM's Watson division.

Rama Akkiraju Director, Distinguished Engineer IBM

Creating Customer-Facing Experiences by Machine Learning

Airbnb is a global platform that connects travelers and hosts from over 191 countries. In this talk, we will present how we approach personalization of travelers’ booking experience using Machine Learning. We will start from the cold start problem when the data is limited. We will then show how personalized features are used to accommodate wide differences in our traveler & host attributes.  We will then discuss how we deploy models in production with real-time features.

Kapil Gupta Data Science Lead Airbnb

Connections between Machine Learning Models and Statistical Experiments

Uber Eats CRM team in EMEA starts email campaigns to encourage order momentum early in the lifecycle. The experimenters on the CRM team plan to run a campaign with 10 different email subject lines and find out the best subject line in terms of the open rate and the number of open emails. The business question is that how should we decide what percentage of impressions to allocate to each line while running an experiment? Previously, the experimenters run A/B experiments to find out the most effective subject lines. During this new campaign, they try to adapt to intermediate results of their tests, optimizing what they earn while learning about their lines. The classic A/B tests are not designed to solve such problems. Aligning with Dara’s Priority in 2018 on automation to lower costs, we researched automating the experiment to increase the return for statistical experiments. This document introduces a new experiment method called multi-armed bandit experiment (MAB) and we used the EMEA CRM campaign to show a significant (p-value<0.01) improvement in customer engagement compared to classic AB tests. MAB groups received more than thousands of open emails than the AB test. We could have saved more than 30% of the experiment time.The bandits experiment as well as the continuous experiment framework has been reviewed by and presented to Uber Data leadership and Uber AI labs.

Jeremy Gu Senior Data Scientist Uber

Lunch

Deep learning in medicine: Applications to disease diagnostics and DNA sequencing

Deep learning is an exciting class of statistical machine learning that is powering products across Google from speech recognition to image understanding. We will introduce deep learning, covering both what it is and why it's so exciting. We describe our recent research at Google applying deep learning in biotechnology, including automated detection of diabetic retinopathy from images of the retina, predicting cardiovascular risk factors from those same eye scans, and creating a deep-learning-based genome mutation detector for high-throughput DNA sequencing data.

Ryan Poplin Machine Learning Engineer Google

Turning Clicks into Purchases: Revenue Optimization for Product Search in E-Commerce

In recent years, product search engines have emerged as a key factor for online businesses. According to a recent survey, over 55% of online customers begin their online shopping journey by searching on an E-Commerce (EC) website like Amazon as opposed to a generic web search engine like Google. Information retrieval research to date has been focused on optimizing search ranking algorithms for web documents while little attention has been paid to product search. There are several intrinsic differences between web search and product search that make the direct application of traditional search ranking algorithms to EC search platforms difficult. In this talk, I will talk about a novel learning framework for EC product search called LETORIF. In this framework, we utilize implicit user feedback signals and jointly model the different stages of the shopping journey to optimize for EC sales revenue.

Liangjie Hong Head of Data Science Etsy

Transforming a mission critical eCommerce platform by building AI powered chatBOTs, leveraging deep learning algorithms

Use cases for leveraging AI have been well documented for consumer facing eCommerce platforms but there has been less progress in incorporating AI in large B2B eCommerce platforms with very complex high tech products. Cisco’s eCommerce platform is responsible for buying & selling all of Cisco’s products, including hardware, software, services & subscriptions. It is a custom-built mission critical platform which accounts for $40+ billion of Cisco’s revenue annually. Cisco is under going a transformation to move it’s hardware revenue as recurring subscriptions based revenue. Cisco is also looking at additional avenues for increasing revenue by providing additional platforms for Direct/SMB customers who are not well versed with how to order Cisco’s complex products. This presents a number of challenges for our users to understand the ever-changing user experience.We have been leveraging AI & Machine Learning to provide a guided experience to our users to bring them onboard in this transformational journey. We have built multi-lingual AI powered chatBOTs leveraging Google’s Cloud platform. We have also leveraged machine learning algorithms to find popular products by category, items sold together and products offering best discounts which can change dynamically. BOT also has the capability to understand the sentiments of the user & route it to a live agent if user is unhappy.The BOT will also be used on the Enterprise platform to predict whether a customer quote will be approved using deep learnings algorithms and TensorFlow. This will help sales account managers get guidance on optimal discounts for customers and help with supply change planning and forecasting.

Dharmesh Panchmatia Senior Director, Engineering Cisco

Networking Coffee Break

Deep Learning Applications to Online Payment Fraud Detection

The talk will cover some applications and use-cases of deep neural network architectures applied to the problem of payments fraud detection. With the multi-fold objectives such as maximizing fraud catch rate, approving the good user volume reliably and quickly, the underlying problem formulation and considerations applicable to large-scale online payment transaction data: such as dimensionality reduction, sparsity, high cardinality and temporality will be covered. Covering an assortment of deep learning methodologies applied to each problem formulation, some empirical comparisons and results will be presented.Lastly, some high level aspects of run-time performance benchmarking as applicable to training/inferencing processes and model deployment at PayPal will be presented.

Nitin Sharma Distinguished Data Scientist PayPal

Product Recommendation at Scale

Square offers a suite of products to empower various types and sizes of sellers to run and grow their businesses. How do product data scientists at Square leverage machine learning to help sellers discover the most relevant products? Corin Qi, a Square data science lead will talk about how her team developed a common framework to accomplish this goal at scale. 

Corin Qi Data Science Lead Square

Just Like a DNA String: Videogame Player Segmentation through Frequent Sequence Mining

Developed in cooperation with Alessandro Canossa, Senior User Researcher at Massive Entertainment - a Ubisoft StudioTraditional segmentation strategies rely on clustering based on aggregated characteristics. Major risk is that difference between segments can be minimal and not descriptive: by aggregating data, one represents users as a combination of disconnected features.We want to account for the context in which each action is undertaken by users. User behavior can be represented as the sequence of actions performed.We did this for a video game (Tom Clancy’s The Division) and looked for clusters based on the sequences of players’ actions.This type of analysis has been pioneered by scientists to investigate chromosome sequences and it can help in getting a deeper insight into user behavior and offer new horizons for predictive analytics.

Sasha Makarovych Game Data Analyst Massive Entertainment - a Ubisoft Studio

Networking Drinks Reception

Day 2

Registration & Breakfast

Chairperson Opening Remarks

Chairperson Opening Remarks 

Judith Lee Associate Professor and Dept. Chair, Operations & IT Management Golden Gate University

When Machine Learning takes on Signal Processing

This talk describes my journey of leading audio and vision researchers and domain experts from using signal processing as their primary tool to using machine learning and deep learning as their first option. This change was not just about using new technology, but also about changing the mindset and reframed the way R&D is done. I also plan to summarize the reasons behind the shift, what worked/what didn't and lessons learned in hopes to guide and inform other teams planning to go through a similar transition.

Vivek Kumar Director, Advanced Technology Group Dolby Laboratories

Leveraging Causal Inference for Demand Estimation

In any retail or e-commerce setting, demand estimation is of paramount importance. Unfortunately, it is also very challenging to estimate the true demand when there is under-supply. This talk will present a causal inference based approach to estimating demand. Causal inference principles will be developed from the ground-up. These principles will then be used to develop demand estimation models. 

Ganesh Krishnan Data Scientist Instacart

Deep Learning at Freebird

Ethan is the CEO and co-founder of Freebird. Ethan is an experienced travel industry professional who is passionate about making travel better for consumers. Ethan worked in Corporate Development at Expedia Inc from 2010-2014, where he developed deep experience and relationships across the $1T global travel industry, leading commercial deals, high-profile strategic projects, and M&A totaling over $350M. Prior to that, Ethan graduated from Brown and worked in private equity, and then left the MBA program at Harvard in order to launch Freebird.

Sam Zimmerman Co-Founder and CTO Freebird, Inc.

Networking Coffee Break

The History (and Future) of Machine Learning at Reddit!

The History (and Future) of Machine Learning at Reddit! Background on Reddit's history Overview of ML efforts of our past Anti-spam, anti-cheating, bot detection, vote manipulation prevention, etc... Future plans for ML at Reddit Personalization, rankings, recommendations...

Luis Bitencourt-Emilio Principal Director, Engineering Reddit

Panel: Attaining Buy-in From Management

This panel will discuss the process, tips, and tricks to gain buy-in from management at any level. Here you will be able to ask questions to a board of senior management, and understand the thinking behind the process of attaining buy-in from the powers that be.Panelists:Nick Caldwell, VP Engineering, RedditSami Ghoche, Senior Software Engineer, LinkedInLiangjie Hong, Head of Data Science, Etsy

Big Data Expert Industry Leader Innovation Enterprise

COTA: Improving Uber Customer Care with NLP & Deep Learning

Uber’s Customer Obsession Ticket Assistant (COTA) is a collection of Natural Language Processing (NLP) models that help ensure fast and accurate resolution of customer support tickets. The ML models assist agents by automatically recommending the most relevant issue label and response. Some stats: Our models cover over 90% of all support tickets and have shown via A/B experiments to reduce average handle time by more than 10% while maintaining or increasing customer satisfaction. In addition to productionizing these models to score real time within Uber’s ML Platform (Michelangelo), we’ve explored migrating to a number of Deep Learning architectures. These new approaches offer up to a 20% improvement in model accuracy and will be discussed during the talk.

Hugh Williams Data Science Manager - Applied Machine Learning Uber

Lunch

Modeling Customer Archetypes

How do you synthesize behavioral signals to target the 'best' customers? This talk will cover how to combine supervised and unsupervised machine learning models to create a more discerning targeting model and reduce dimensionality.

Inna Kaler Data Scientist Square

Cloud Training and Embedded Inference for IoT/Robotics

We describe a workflow and best practices for training deep-learning models efficiently in the cloud, and deploying them on embedded devices. This has many applications in computer vision, speech recognition, and natural language processing. We also discuss a class of model architectures and techniques useful when performing inference with very limited memory and computation.

Mark Palatucci Head of Cloud AI & Machine Learning Anki

Growing the LinkedIn content ecosystem with machine learning.

How content is created and distributed at Linkedin. The role of follows and follow recommendations. (4 types of follow recommendations)3 major problems to address:- Personalization requires operating at a very large scale: matching members to other members is a quadratic problem on the order of 10^17 pairs.- Two-step optimization problem: follow recommendations have to optimize for likelihood of a follow edge being created, but also have to optimize for actual engagement that can result from this edge over time. Added complication of different models that rank content once it’s generated that are always changing.- Dynamic follow recommendations: How to be responsive to real-time actions that the member is taking, while using resources efficiently and operating at a massive scale.

Sami Ghoche Senior Software Engineer LinkedIn

Networking Coffee Break

Risks of Machine Learning

Despite their broad applications, there is still no comprehensive theoretical understanding of learning algorithms and their performance. One of the key reasons is that the field of machine learning (ML) has historically been linked with statistics and probability theory; resulting in black-box models. The hyper-parameters tuning, random initialization, statistical assumptions/simplifications, Ad-hoc tweaks/tricks, overfitting, biasness, computational instability, slow convergence, lack of optimality guarantee, and the need for huge observations for learning are just a few challenges of such models. In black-box models, running diagnostics to figure out what is the problem is going to be a sophisticated task. The literature around the ML has practically stagnated and out of more than half a century of ML research, there are only a few that are making substantial progresses and the rest are just about fine-tuning the existing methods for different datasets.

Nima Safaei Associate Director, Network Analytics Scotiabank

Navigating the road to Production

As the practice of Machine Learning and AI comes increasingly into focus, more and more practical applications are met with an ever-increasing set of tools and techniques.  The AI/ML-driven success of some firms at have led the tech industry and beyond to seek the same benefits at scale—hoping to capture some of the same benefits.  But what happens when you hire talented people, develop a promising ML or AI solution and are ready to deploy it into a new or existing product? The inherent complexity of the Data Science development lifecycle can throw traditional software engineering practices and deployment cycles for a loop.  This talk will provide the audience with some tools to help bridge the gap.  We’ll explore automated data pipeline management, deployment testing, dynamic/automated model parameter adjustment, MLaaS APIs and data science collaboration & revision tools.

Justin Norman Senior. Manager, Applied Machine Learning Fitbit