How to remove duplicate dictionaries from a list in Python? -

i have list of dictionaries sorted specific key. each dictionary contains 32 elements, , there on 4000 dictionaries in list. need code work through list , return new list duplicates removed.

the methods these links:

• removing duplicates in lists
• how remove duplicates list in whilst preserving order?

don't me because dictionary unhashable.

any thoughts? if need more info, comment , add info.

edit:

a duplicate dictionary 2 dictionaries same values list[dictionary][key].

---

ok, here detailed explanation need it.

i have list of dictionaries thus:

[ {     "id" : "0001",     "organization" : "solarusa",     "matchcode" : "solarusa, street, somewhere state, whatev zip",     "owner" : "timothy black",    }, {     "id" : "0002",     "organization" : "solarusa",     "matchcode" : "solarusa, street, somewhere state, whatev zip",     "owner" : "johen wilheim",    }, {     "id" : "0003",     "organization" : "zapotec",     "matchcode" : "zapotec, street, somewhere state, whatev zip",     "owner" : "simeon yurrigan",    } ] 

of list, first , second dictionaries duplicates because matchcodes identical.

now list sorted following code:

# sort_by "matchcode" def sort( list_to_be_sorted, sort_by ):     return sorted(list_to_be_sorted, key=lambda k: k[sort_by]) 

so have neat list of dictionaries sorted matchcode. need iterate on list, accessing list[dictionary][key] , deleting duplicates when 2 key values match.

just can use tuple hashable equivalent list, can use frozenset hashable equivalent dict. trick need pass d.items() rather d constructor.

>>> d = {'a': 1, 'b': 2} >>> s = frozenset(d.items()) >>> hash(s) -7588994739874264648 >>> dict(s) == d true 

and can use favorite of solutions you've seen. dump them set, or use orderedset or unique_everseen recipe if need preserve order, etc. example:

>>> unique_sets = set(frozenset(d.items()) d in list_of_dicts) >>> unique_dicts = [dict(s) s in unique_sets] 

or, preserving order , using key value:

>>> sets = (frozenset(d.items()) d in list_of_dicts) >>> unique_sets = unique_everseen(sets, key=operator.itemgetter(key)) >>> unique_dicts = [dict(s) s in unique_sets] 

of course if have lists or dicts nested within, have convert recursively, list of lists.