When you filter a queryset across a ManyToMany relation or a reverse relation, Django can return the same record several times — the list shows duplicates, pagination counts the wrong number of pages, and count() reports the wrong total. The cause is the JOIN, which produces one row for every related record that matches the filter. The cleanest fix is to add .distinct() at the end of the queryset.

The problem

A familiar situation: a book has several authors, and you want to list every book by An or Binh.

# Lấy mọi cuốn sách của An hoặc Bình
books = Book.objects.filter(authors__name__in=["An", "Binh"])
print(books.count())  # 3 — sai, chỉ có 2 cuốn sách
for book in books:
    print(book.title)
# Django cơ bản
# Django cơ bản   <- trùng!
# Django nâng cao

The result is 3 rows even though there are only 2 books: “Django cơ bản” appears twice because it matches both authors in the filter. The error quietly spreads: len() and count() both report 3, pagination computes one page too many, and the UI shows the same book twice. It gets more dangerous when you use this result to total money or compute statistics — the numbers are wrong and no error is raised.

The solution

Just add .distinct() at the end of the queryset. Django translates it to SELECT DISTINCT, so the database removes duplicate rows before returning the result, and you get exactly the 2 books.

books = Book.objects.filter(
    authors__name__in=["An", "Binh"]
).distinct()
print(books.count())  # 2 — đúng

distinct() is also handy when you only need a list of unique values — for example, the names of every author who has published a book. Combine values_list() with distinct() and let the database do the work instead of loading everything and filtering with set() in Python.

# Danh sách tên tác giả duy nhất
Author.objects.values_list("name", flat=True).distinct()
# <QuerySet ['An', 'Binh']>

One important note: distinct() removes rows that are completely identical, not records that are merely “similar”. If two books share the same title but have different IDs, both still appear — that is genuinely duplicated data, which needs a unique constraint rather than distinct().

Applying it to your project

Review the querysets in your project that filter across tags, categories, authors, or any many-to-many relation, and add .distinct() at the end. The fastest check is to print count() before and after adding it: if the number drops, you have just fixed a real duplication bug. Make it a team rule: every queryset that filters across a many-to-many relation gets distinct().

Related posts

FAQ

Does distinct() slow the query down?

Not noticeably: the database handles DISTINCT inside the SQL statement, usually far faster than loading all the duplicate records into Python and filtering them with set(). Only when a table has millions of rows and lacks a suitable index do you need to measure again with django-debug-toolbar.

Does distinct(‘column_name’) work on MySQL and SQLite?

No. The distinct() form with a column name (DISTINCT ON) exists only on PostgreSQL; running it on MySQL or SQLite raises a NotSupportedError. If your project uses several databases, keep distinct() without arguments so the code behaves the same everywhere.

How is distinct() different from using set() in Python?

distinct() filters duplicates right at the database, returning only the rows you need. Using set() means you have already loaded all the duplicates into memory before filtering — it costs RAM and is slower on large data. The general rule: filter as early as possible, ideally at the database layer.

I added distinct() but the results are still duplicated — what now?

Check whether you use annotate() or values() with several columns — then each row already differs in the computed column, so DISTINCT cannot remove it. The fix is to call distinct() before annotate, or to call distinct() on exactly the column you need.

💬 Have you tried this in your own project? If you hit an error or know a cleaner way, leave a comment below — I read them all and will fold good ideas into the post.

Sources