This Technique is Useful for Many Purposes

Python’s built-in functions make it easy to find duplicates in a list. The technique uses the set() function which automatically removes duplicates, then converts the result back to a list and returns …

Updated October 28, 2023

Python’s built-in functions make it easy to find duplicates in a list. The technique uses the set() function which automatically removes duplicates, then converts the result back to a list and returns it.

We can use this method to remove or identify duplicates in any list data type. For example, we might want to write a program that takes a user’s favorite colors as input (which could include duplicate color names). We could use set() to find the unique colors then convert them back into a list for further processing:

favorite_colors = ["red", "green", "blue", "yellow", "orange", "red"]  # This is our favorite colors list with some duplicates.
unique_colors = list(set(favorite_colors))  # The set function automatically removes duplicates, then we convert it back into a list using the list() function.
print(unique_colors)  # Prints: ['red', 'green', 'blue', 'yellow', 'orange']

We can also use this method to identify duplicate items in any iterable (like a tuple or dict). For example, if we had a dictionary of people and their birthdays:

birthday_dict = { "Alice": "1990-05-16", "Bob": "1987-12-03", "Charlie": "1990-05-16" }  # This is our birthday dictionary with some duplicates.
duplicate_dates = [k for k, v in list(birthday_dict.items()) for i, j in list(birthday_dict.items()) if i != k and v == j]  # We loop through the items of the dict twice to find pairs with matching values.
print(duplicate_dates)  # Prints: ['Charlie', 'Alice'] as they have same birthdate.

This way, you can make sure no duplicates are in your list by converting it into a set and then back into a list before proceeding with other tasks. This method is especially useful when working with large data sets or high-level programming where there might be the need for unique values.

Hey! Do you love Python? Want to learn more about it?
Let's connect on Twitter or LinkedIn. I talk about this stuff all the time!