Time windows helper classes and functions
[Deprecated] WindowedDataFrame approach
Imports
import os
from pyspark.sql import functions as f
Create widgets for passing in timestamp
dbutils.widgets.text("timestamp", "2020-12-12")
dbutils.widgets.text("target", "no target")
Initialise target store
%run ../init/target_store
Define time windows
time_windows=[14, 30, 90]
Load silver data
df_web_visits = spark.read.table(
f"{os.environ['READ_ENV']}.odap_digi_sdm_l2.web_visits"
)
Add timestamp column to the dataframe by joining the target store
target_store = spark.read.table("target_store")
df = df_web_visits.join(target_store, on="customer_id").filter(
f.col("visit_timestamp") <= f.col("timestamp")
)
Create a custom time_widowed function using visit_timestamp
as your time_column