Ask Your Question
0

how to remove duplicate elements in a list

asked 2010-09-17 09:23:50 +0100

sriram gravatar image

updated 2015-01-14 12:06:34 +0100

FrédéricC gravatar image

From a list L = ([1,2,3,4],[3,5,6,7],[7,8,9,10],[1,2,3.4]) remove duplicate sets here [1,2,3,4] has occurred twice and put only one making L = ([1,2,3,4],[3,5,6,7],[7,8,9,10])

edit retag flag offensive close merge delete

Comments

L is a tuple. For a list, replace the outer parens with square brackets.

ccanonc gravatar imageccanonc ( 2010-09-17 23:45:59 +0100 )edit

Please change your tags. "urgentX" is more annoying than I am.

ccanonc gravatar imageccanonc ( 2010-09-17 23:46:58 +0100 )edit

4 Answers

Sort by » oldest newest most voted
2

answered 2010-09-21 11:26:42 +0100

Volker Braun gravatar image

Sage has a uniq() function already. However, it requires that the data is hashable, so you can uniq-ify tuples and frozensets but not lists and sets:

sage: L = [(1, 2, 3, 4), (3, 5, 6, 7), (7, 8, 9, 10), (1, 2, 3, 4)]
sage: uniq(L)
[(1, 2, 3, 4), (3, 5, 6, 7), (7, 8, 9, 10)]
edit flag offensive delete link more
0

answered 2010-09-17 10:14:37 +0100

jsrn gravatar image

If you're lazy and don't care about ordering in L:

sage: list(set(L))

Again, all of this is pure Python, so you might benefit from reading a tutorial on writing Python code.

edit flag offensive delete link more
0

answered 2010-09-17 10:10:06 +0100

mvngu gravatar image

Maybe this might help:

sage: L = [[1,2,3,4], [3,5,6,7], [7,8,9,10], [1,2,3,4]]
sage: seen = []
sage: for e in L:
....:     if e in seen:
....:         continue
....:     seen.append(e)
....:     
sage: seen
[[1, 2, 3, 4], [3, 5, 6, 7], [7, 8, 9, 10]]
edit flag offensive delete link more
0

answered 2010-09-17 11:02:14 +0100

niles gravatar image

The page fastest way to uniquify a list in python addresses this; there, it's claimed that the following function is the "best" (fast, and maintains order):

def f5(seq, idfun=None): 
    # order preserving
    if idfun is None:
        def idfun(x): return x
    seen = {}
    result = []
    for item in seq:
        marker = idfun(item)
        # in old Python versions:
        # if seen.has_key(marker)
        # but in new ones:
        if marker in seen: continue
        seen[marker] = 1
        result.append(item)
    return result
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2010-09-17 09:23:50 +0100

Seen: 8,587 times

Last updated: Sep 21 '10