page276
364
Replication
Replication data volumes are impacted by your approach to data ingestion and curation.
As covered in Chapter 8, parallelization has the potential to minimize replicated data
through minimizing the number of micro-partitions changed for a given object. You will
observe that data replication costs can far exceed curation costs; therefore, you must
retain a tight focus on all parts of the data distribution.
Chapter 9 discussed how entitlements can be implemented, offering two different
implementation patterns. The “all-or-nothing” approach leads to a high probability of
data being replicated but with significant portions unused by clients. The bespoke client-
centric approach of prefiltering data prior to presentation for client consumption may
reduce replicated data sets, but at additional provider curation and refresh cost.
Snowflake replication costs are always paid by the data provider regardless of the
data transfer mechanism.
From a producer perspective, you should always be mindful of your client
consumption costs. From a consumer perspective, you should insist your consumption
costs are minimized. Producers should facilitate their consumers by enabling multiple
accounts to ingest the same share. Where data products are subject to user license limits,
I suggest enabling multiple accounts to consume the same share offers additional sales
opportunities due to higher usage and increased data product license requirements.
Multiplatform Distribution
Snowflake does not exist in isolation as the sole data marketplace. Figure 10-3 illustrates
one possible ecosystem where Snowflake is used to curate and master data products and
then distribute to multiple distribution venues.
ChapTer 10
OpTimizing perfOrmanCe