Easy Way to Find Duplicates in a List Using Sets in Python
Finding duplicates in a list can be a bit of a pain. However, it’s a piece of cake once you use the built-in set data structure in Python because sets only contain unique elements. Here is a simple wa …
Finding duplicates in a list can be a bit of a pain. However, it’s a piece of cake once you use the built-in set data structure in Python because sets only contain unique elements. Here is a simple way to detect and remove duplicates from a list by converting the list into a set, then back into a list.
# Python program to demonstrate conversion
# of list to Set and vice versa
# List with duplicate values
list1 = [10,20,30,40,50]
# Converting list to set
set1 = set(list1)
print("The set after converting list is :", set1)
# Back conversion of set to list
list2 = (list(set1))
print("The list after back conversion of set :", list2)
In this example, the duplicate values are not present in the converted set. This is because sets only contain unique elements by definition.
Now, let’s consider a situation where you have a list with duplicated data and you want to find out which items appear more than once in the list. Here we will use collections.Counter():
# Python program to demonstrate Counter
import collections
# List with duplicate values
list1 = [10,20,30,40,50,40]
# Converting list to Counter object
counter_obj = collections.Counter(list1)
print("The counter after converting list is :", counter_obj)
# To check the items that appear more than once in a list
for item, frequency in counter_obj.items():
if frequency > 1:
print('Duplicate value:', item)
In this example, you can see that we have two “40” elements in our original list and that’s why it shows up as a duplicate when we use collections.Counter().
AI Is Changing Software Development. This Is How Pros Use It.
Written for working developers, Coding with AI goes beyond hype to show how AI fits into real production workflows. Learn how to integrate AI into Python projects, avoid hallucinations, refactor safely, generate tests and docs, and reclaim hours of development time—using techniques tested in real-world projects.