STEDI framework to evaluate AB metrics.

This framework provides a holistic way to assess if an experiment metric is good, and helps identify areas of improvement.

Somit Gupta is a Senior Data Scientist for Microsoft’s Analysis and Experimentation team. He helps MSN, edge browser, Office and Windows to innovate faster with trustworthy experimentation. He helps the team run experiments at scale: defining the OEC for the product, design & monitoring of experiments, and interpretation of results to make a ship decision. He co-lectured a tutorial on A/B Testing at KDD 2017. Previously he was a product manager for Windows and Skype. Somit received a master’s degree in Computer Algebra from University of Waterloo in 2011. Prior to that he received a bachelor’s degree in Computer Engineering from National Institute of Technology, Surathkal, India.