page216
302
Now populate the raw or staging table
stg_customer_order
using the known base
data for each DML operation from earlier.
USE WAREHOUSE IDENTIFIER ( $tpc_warehouse_m );
INSERT OVERWRITE INTO stg_customer_order
SELECT *
FROM base_customer_order_insert
UNION
SELECT *
FROM base_customer_order_update
UNION
SELECT *
FROM base_customer_order_delete;
With the test data configured and the target core table reconfigured as five
segmented tables, you can do the following:
USE WAREHOUSE IDENTIFIER ( $tpc_warehouse_xs );
SELECT count(1)
FROM part_customer_order_asia;
For this data set, you see 300,094,996 records; your record count will differ. Make a
note of this number.
SELECT count(1)
FROM stg_customer_order
WHERE region_name = 'ASIA';
For this data set, you see 60,417 records; your record count will differ. Make a note of
this number.
Stream Interaction
Streams are an ideal mechanism for identifying change data capture on a base object,
but their usage can be confusing. Let’s examine use cases for integrating streams into our
data pipeline.
Chapter 8
parallelization