Updating "Attributes" in Local File upload , after data Ingestion

Question

Step 1: On day 1, I created Source data flow via Localfileupload
Step 2: Uploaded .csv with 10 attributes into Dataset : Dataset1
Step 3: On day 2, I wanted to remove 2 attributes from 10
 
Thus question, could i do it ?

Avinash_Gupta_ · Accepted Answer

@navin_kumar0601

Generally speaking, AEP is an additive platform which means there are limited options for deleting data once it has been ingested. From the UI, you can delete a batch of data from the underlying data lake, but that will not remove the Profile data. You can also remove the entire dataset using the UI, which will remove all the data from the data lake and related Profiles. However, that will remove all data associated with the dataset (think of it as dropping a table in SQL). You can delete the dataset and re ingest the data by removing the 2 attributes.

Anil_Umachigi · Answer

@navin_kumar0601 When you say remove it all depends on from where? and also if the data set is a time series or Record (Attributes) data.

Record data: If record data type, you can overwrite those attributes on day 2 if that work for you? else you can remove the entire profile using privacy API. Another option is to drop the entire dataset thus deleting all data.

Time series data: This is time stamped data, no two rows will be same because of time stamp and hence there is not really any way to remove or update data in this case. Your only option is

Delete the batch which ingested the data ( will also delete other rows of data), if enabled for profile this method will not remove it from profile
Delete dataset to remove all data & which will remove all of the profile related data as well

Hope that helps

Anil

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded