With the proliferation of AI infused technologies and experiences over the past several years, we are often faced with the question of whether these AI implanted technologies are more successful and productive for our users and organizations. This is especially relevant whenever a new AI technology surfaces, we often wonder is it better than what existed beforehand? This session will cover how to measure the success of your AI products and experiences and benchmark them against prior solutions. The researchers will discuss how they measured the success of new AI products including chatbots for IT support and Gen AI tools that leverage large language models and can perform tasks for users. Our talk will discuss these case studies of new AI products and the metrics we used to measure their productivity enhancements and success.