Material Detail

This video was recorded at 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Paris 2009. The web provides an unprecedented opportunity to evaluate ideas quickly using controlled experiments, also called randomized experiments, A/B tests (and their generalizations), split tests, and MultiVariable Tests (MVT). Controlled experiments embody the best scientific design for establishing a causal relationship between changes and their influence on user-observable behavior. Data Mining and Knowledge Discovery techniques can then be used to analyze the data from such experiments. The tutorial will provide a survey and practical guide to running controlled experiments based on the recently published survey article in the Data Mining and Knowledge Discovery Journal, co-authored with the two of the tutorial co-presenters Controlled Experiments on the Web: Survey and Practical Guide, and based on the book "Always Be Testing" co-authored by the 3rd tutorial co-presenter Always Be Testing: The Complete Guide to Google Website Optimizer. The book includes use of industry tools, such as Google Website Optimizer and recently ranked #2 on Amazon's sales rank for computers/e-commerce books. The tutorial includes multiple real-world examples of actual controlled experiments (many with surprising results), a review the theory and the statistics used to plan and analyze such experiments, and a discussion of the limitations and pitfalls that might face experimenters. Demos will be shown of some tools that support controlled experiments.

Disciplines:

• Science and Technology  / Computer Science