page54
35
2
3
5
8
0 1 2
4
6
10
16
8
12
20
32
Execuon Cycle
CPUs
Figure 2-3.
SIMD example
Compression
Snowflake query execution also makes use of compression; however, again available
details are sparse. Compression may refer to in-memory compressed data, which is
decompressed “on the fly” or to data stored in CPU caches, local SSD, and remote
storage.
I know Snowflake applies a combination of compression methods including some
tailored to the specific data type used. You can also be confident Snowflake provides the
best compression available as part of their service and continually strives to improve
their service.
Vectorization
The query execution engine is also vectorized, handling batches of a few thousand rows
at a time. The actual batch size is unspecified, but given the propensity for programmers
to prefer factors of 2, and speculation on my part, it may be in the range of 2,048 to 8,192
rows, and these rows will be in columnar format.
Query execution may also spill result sets to both local storage (SSD) and remote
storage where result sets exceed CPU cache and allocated memory.
Chapter 2
the Query Optimizer