Material Detail

Predictive Modelling in the Wild: Success Factors in Data Mining Competitions and Real-World Applications

Predictive Modelling in the Wild: Success Factors in Data Mining Competitions and Real-World Applications

This video was recorded at 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Paris 2009. In this tutorial, we give our perspective on the keys to success in application of predictive modeling to competitions like KDD Cup and real-life business intelligence projects. We argue that these two modes of applying predictive modeling share many similarities, but have also some important differences. We discuss the main success factors in predictive modeling: domain understanding, statistical acumen, and appropriate algorithmic approaches. We describe our relevant experiences in the context of three recent predictive modeling competitions where our team has had success (KDD Cup 2007 and 2008 and INFORMS DM challenge 2008) and two case studies of projects we have led at IBM Research. We also survey some of the recurring challenges and complexities in practical predictive modeling applications. One key issue is information leakage, and we discuss its definition, influence, detection and avoidance. We consider leakage to be the silent killer of many predictive modeling projects, and we demonstrate its impact on the competitions, and discuss the challenges in addressing it in the real-life projects. Other challenges include framing real-life modeling objectives into predictive modeling, and usefully applying relational learning concepts when modeling "real-life" complex, relational datasets.

Quality

  • User Rating
  • Comments
  • Learning Exercises
  • Bookmark Collections
  • Course ePortfolios
  • Accessibility Info

More about this material

Browse...

Disciplines with similar materials as Predictive Modelling in the Wild: Success Factors in Data Mining Competitions and Real-World Applications
People who viewed this also viewed

Comments

Log in to participate in the discussions or sign up if you are not already a MERLOT member.