Duplicates removal using Fusion? | Community
Skip to main content
V
viovi
New Participant
October 31, 2023
Question

Duplicates removal using Fusion?

  • October 31, 2023
  • 2 replies
  • 1593 views

Hello,

 

Is there a way to remove duplicates coming from Parse CSV?

CSV file is coming from another system, so we do not have access to it before it's been received by Fusion. We need to make sure that we have only 1 line per employee, and exclude any duplicates, if they happen to be there before the scenario processes it further.

Are there any examples how to implement this?

 

Could you please advise?

Thank you!

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

2 replies

ChrisStephens
ChrisStephens
New Participant
November 2, 2023

As an alternative approach, you could place an array aggregator right after the parse CSV, and group it by whatever key you need to ensure is only processed once.

    V
    vioviAuthor
    New Participant
    November 2, 2023

    Thank you, Chris! I tried that but for some reason the other values (except for the Group by field) were empty and did not carry over values to the following modules (I have more than 15 columns in CSV file total). I guess I'm just missing something there.

    ChrisStephens
    ChrisStephens
    New Participant
    November 2, 2023

    You have to select which fields to pass thru, did you do that?

    lgaertner
    lgaertner
    New Participant
    November 1, 2023

    Hello Viktoriia,

     

    Without knowing the structure of your csv file, I would use this workflow in Fusion.

    As I do not have a corresponding csv file, I used a data store with contact names in the column name:

     

    1. use the module Set variable with the variable value emptyarray, which you find under keywords on the array functions tab

    2. parse your csv records into an Array aggregator 

    3. add another Set variable module with the variable name from step 1 and the value map(2. <Array[]>; data.name)

    4. add another Set variable module with the variable name from step 1 and the value deduplicate(<3. [])

     

    Regards

    Lars

      V
      vioviAuthor
      New Participant
      November 1, 2023

      Thank you, Lars, I'll try this approach.

       

      Terms & ConditionsAccessibility statement

      Loading footer...