page201
287
Figure 8-13.
Warehouse full processing unit consumption
As I identified previously, splitting the
MERGE
statement into its component DML
operations will not work as the outcome serializes the workflow. You need a different
approach: segment the target core table.
To parallel process a data load, you must consider these factors:
•
How to shard the target core table into physical partitioned tables
•
Number of warehouse concurrent processing units required per feed
•
Orchestrating physical partition loads
•
Impact of partition load completion versus full table load
•
Denormalizing physical partitions to represent the physical
core table
Figure 8-14 shows a high-level design to implement parallel processing of a
single feed noting there will be “n” segmented tables according to the segmentation
key chosen.
Chapter 8
parallelization