page131
114
Micro-partition Metadata
Snowflake stores data in an internal, compressed, columnar format. You also know from
Chapter 1 that Snowflake captures and maintains the following statistics for each micro-
partition stored in the Cloud Services layer:
•
Table and micro-partition
•
Row count
•
Size in bytes (including compression information)
•
File reference
•
Table version
•
Clustering
•
Total number of micro-partitions
•
Micro-partition overlap values
•
Micro-partition depth
•
Column
•
Max/min value range
•
The number of distinct values
•
NULL count
•
Subcolumn
•
Statistics for common paths in semi-structured data
Referencing the metadata held for each micro-partition, the optimizer is able to
rapidly identify only those micro-partitions holding the data required to satisfy the query
results and excludes (or prunes) irrelevant micro-partitions.
Because the relevant metadata is known for each micro-partition, the contents
can be compressed as the optimizer does not need to interrogate micro-partitions
interactively to identify matching content. You also know each micro-partition
contains up to 16 MB of data compressed using proprietary compression algorithms.
Uncompressed, each micro-partition holds between 50 MB and 500 MB of data, and
compression is optimized according to the column’s data type.
ChaPTer 4
MiCro-ParTiTionS