page23
3
Setting the Scene
I began writing this book in July 2023, a week after Snowflake Summit ended. My head
was full of ideas, buzzing with the prospect of writing this book to impart my perspective
and available wisdom on performance tuning Snowflake to a wider audience. What
struck me was that, in just four years, Snowflake had transitioned from the cloud data
warehouse of choice to a much richer and hard-to-define platform encompassing a wide
variety of tooling, data formats, and capabilities.
Within this book I do not dive into the ever-expanding Snowflake product
capabilities, instead preferring to focus on what some describe as the “black art” of
performance tuning. By now, plenty of organizations have both ported applications to
Snowflake and/or developed applications on Snowflake from scratch. The time is right
for a book on Snowflake performance tuning to extract maximum value from these
investments.
It would be too easy to cover what has already been described at an overview level
by many vendors, some of whom are offering solutions that treat the symptoms and not
the root cause. Conversations supported by Microsoft PowerPoint is one thing; practical
techniques supported by hard and fast empirical evidence is entirely another. I prefer to
demonstrate pragmatic approaches to resolving performance issues while developing
tools to both educate and deliver a firm foundation for you to later build upon.
Snowflake is designed from the ground up to deliver optimal query performance
with minimal user intervention. The “out-of-the-box” developer and user experience is
truly exceptional, delivering astounding results for both data warehousing applications
and, increasingly, much wider use cases including AI/ML applications.
In contrast to a provision-based model where you are constrained by your deployed
infrastructure, Snowflake implements a consumption-based model: you pay for
what you consume. Typically, provision-based infrastructure is idle for an average
of 70 percent to 80 percent of the time, with occasional activity or, more commonly,
overloaded activity peaks. In contrast, consumption-based models scale according to
demand, providing performance elastically.
But this flexibility comes at a price: scalability and performance cost real money. you
must therefore reconsider your approach in a consumption-based model and focus on
reducing cost wherever possible. Costs are incurred when you execute code where you
consume CPU and memory. In Snowflake parlance, CPU and memory are encapsulated
within warehouses. You also incur costs for storage on a per-terabyte basis. At the time
ChaptEr 1
tuning thE SnowFlakE Data ClouD