page184
270
Foundational Information
In this section, I will describe the basic concepts relating to a typical application design
used within previous chapters while expanding the application scope to distribute data
globally.
For this investigation let’s will assume an additional requirement to distribute our
offerings globally, as indicated by Figure 8-1.
Figure 8-1.
Example global application perspective
The example focuses on batch inbound data, while the same principles apply to
inbound streamed data. The velocity is likely to be higher, and the volume of each
streamed data set will be lower. The net effect of ingesting streamed data will be higher
micro-partition churn. To mitigate against performance impact, you might adopt a
halfway house of consolidating streamed data into batches before applying in bulk.
What follows is a broad outline of every application at a very high level. Data
is ingested on the left, and products are consumed on the right. Extending our
consumption model by distributing data across marketplaces, regions, cloud service
providers (CSPs), and disparate platforms presents further challenges.
IYou offer this information as a broad analog for all application data flows along with
indicative Data Manipulation Language (DML) actions. The information supplied is not
intended to suggest there is a single right way to construct applications.
Let’s investigate the outline function of each application container individually.
Data Products
Throughout this chapter I use the term “data product” to refer to data curated into a
product or feature resulting in a value-added component that a client consumes. Data
products may be distributed free of charge or commercialized in some manner. In
essence, a data product is “something” a client wants to consume.
Chapter 8
parallelization