page223
309
SELECT timestampdiff ( second, current_timestamp, scheduled_time ) AS
next_run,
scheduled_time,
current_timestamp,
name,
state
FROM TABLE ( information_schema.task_history())
ORDER BY completed_time DESC;
Figure 8-21 shows example output for scheduled tasks.
Figure 8-21.
Stream output
Purging a Stream
After successfully merging all raw or staged data, you must remove the data in
preparation for the next load. You may perform the
TRUNCATE
just before loading new
data; preserving raw or staged data until just prior to the next load is good practice in
the event you need to investigate the most recent data load. However,
TRUNCATE
has a
side effect of also removing load metadata; you can find information at
https://docs.
snowflake.com/en/sql-reference/sql/truncate-table#usage-notes
.
In this example you use
TRUNCATE
, which is not the only option for clearing a staging
table. You can choose
INSERT OVERWRITE
instead. Regardless of the method chosen,
interaction with our choice of data load operator must be tested to ensure the stream
correctly expresses the desired outcome.
TRUNCATE TABLE stg_customer_order;
For further information, please learn more about the
COPY
command:
https://docs.
snowflake.com/en/sql-reference/sql/copy-into-table?utm_source=legacy&utm_
medium=serp&utm_term=copy
and for Snowpipe:
https://docs.snowflake.com/en/
user-guide/data-load-snowpipe-intro?utm_source=legacy&utm_medium=serp&utm_
term=snowpipe
.
Chapter 8
parallelization