The Invent with Python Blog

Mon 05 February 2018

Python Tuples are Immutable, Except When They're Mutable

Posted by Al Sweigart in python   

Prologue

'When I use a word,' Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean — neither more nor less.'

'The question is,' said Alice, 'whether you can make words mean so many different things.'

'The question is,' said Humpty Dumpty, 'which is to be master — that's all.'

Are Tuples Mutable or Immutable?

In Python, tuples are immutable, and "immutable" means the value cannot change. These are well-known, basic facts of Python. While tuples are more than just immutable lists (as Chapter 2 of Luciano Ramalho's excellent "Fluent Python" explains), there is a bit of ambiguity here. Luciano has written up a great blog post on this topic as well.

Before we can get a nuanced answer to "Are tuples mutable or immutable?", we need some background information.

Some Background Information

According to the Python data model, "objects are Python's abstraction for data, and all data in a Python program is represented by objects or by relations between objects". Every value in Python is an object, including integers, floats, and Booleans. In Java, these are "primitive data types" and considered separate from "objects". Not so, in Python. Every value in Python is an object, and Ned Batchelder's fantastic PyCon 2015 talk Facts and Myths About Python Names and Values" goes into more detail about this.

So not only is the datetime.datetime(2018, 2, 4, 19, 38, 54, 798338) datetime object an object, but the integer 42 is an object and the Boolean True is an object.

All Python objects have three things: a value, a type, and an identity. This is a bit confusing, because we often casually say, for example, "the value 42", although 42 is also called an object which itself has a value. But never mind that, let's continue with our 42 example. Enter the following into the interactive shell:

>>> spam = 42
>>> spam
42
>>> type(spam)
<class 'int'>
>>> id(spam)
1594282736

The variable spam refers to an object that has a value of 42, a type of int, and an identity of 1594282736. An identity is a unique integer, created when the object is created, and never changes for the lifetime of the object. An object's type also cannot change. Only the value of an object may change.

Let's try changing an object's value by entering the following into the interactive shell:

>>> spam = 42
>>> spam = 99

You may think you've changed the object's value from 42 to 99, but you haven't. All you've done is made spam refer to a new object. You can confirm this by calling the id() function and noticing spam refers to a completely new object:

>>> spam = 42
>>> id(spam)
1594282736
>>> spam = 99
>>> id(spam)
1594284560

Integers (and floats, Booleans, strings, frozen sets, and bytes) are immutable; their value doesn't change. Lists (and dictionaries, sets, arrays, and bytearrays), on the other hand, are mutable. This can lead to a common Python gotcha:

>>> spam = ['dogs', 'cats']
>>> eggs = spam  # copies the reference, not the list

>>> spam
['dogs', 'cats']
>>> eggs
['dogs', 'cats']

>>> spam.append('moose') # modifies the list referred by spam
>>> spam
['dogs', 'cats', 'moose']
>>> eggs
['dogs', 'cats', 'moose']

The reason eggs has changed even though we only appended a value to spam is because spam and eggs refer to the same object. The eggs = spam line made a copy of the reference, not the object. (You can use the copy module's copy() or deepcopy() functions if you want to copy a list object.)

The glossary in Python's official documentation says this about "immutable" (emphasis mine):

"An object with a fixed value. Immutable objects include numbers, strings and tuples. Such an object cannot be altered. A new object has to be created if a different value has to be stored."

The official Python documentation (and every other book, tutorial, and StackOverflow answer I've found) describes tuples as immutable. Conversely, the glossary says this about "mutable":

"Mutable objects can change their value but keep their id(). See also immutable."

Let's move on to discuss how value and identity affect the == and is operators.

== vs is

The == equality operator compares values, while the is operator compares identities. You can consider x is y to be shorthand for id(x) == id(y). Enter the following into the interactive shell:

>>> spam = ['dogs', 'cats']
>>> id(spam)
41335048
>>> eggs = spam
>>> id(eggs)
41335048
>>> id(spam) == id(eggs)
True
>>> spam is eggs  # spam and eggs are the same object
True
>>> spam == eggs  # spam and eggs have the same value, naturally
True
>>> spam == spam  # Just like spam and spam are the same object and have the same value, naturally
True

>>> bacon = ['dogs', 'cats']
>>> spam == bacon  # spam and bacon have the same value
True
>>> id(bacon)
40654152
>>> id(spam) == id(bacon)
False
>>> spam is bacon  # spam and bacon are different objects
False

Two different objects may share the same value, but they will never share the same identity.

Hashability

According to the glossary in the official Python documentation, "An object is hashable if it has a hash value which never changes during its lifetime", that is, if the object is immutable. (There are a couple other requirements concerning the __hash__() and __eq__() special methods, but that's beyond the scope of this post.)

A hash is an integer that depends on an object's value, and objects with the same value always have the same hash. (Objects with different values will occasionally have the same hash too. This is called a hash collision.) While id() will return an integer based on an object's identity, the hash() function will return an integer (the object's hash) based on the hashable object's value:

>>> hash('dogs')
-4064183437113369969

>>> hash(True)
1

>>> spam = ('hello', 'goodbye')
>>> eggs = ('hello', 'goodbye')
>>> spam == eggs  # spam and eggs have the same value
True
>>> spam is eggs  # spam and eggs are different objects with different identities
False
>>> hash(spam)
3746884561951861327
>>> hash(eggs)
3746884561951861327
>>> hash(spam) == hash(eggs)  # spam and eggs have the same hash
True

Immutable objects can be hashable, mutable objects can't be hashable. This is important to know, because (for reasons beyond the scope of this post) only hashable objects can be used as keys in a dictionary or as items in a set. Since hashes are based on values and only immutable objects can be hashable, this means that hashes will never change during the object's lifetime.

In the interactive shell, try to create a dictionary with immutable objects for keys by entering the following:

>>> spam = {'dogs': 42, True: 'hello', ('a', 'b', 'c'): ['hello']}
>>> spam.keys()
dict_keys(['dogs', True, ('a', 'b', 'c')])

All the keys in spam are immutable, hashable objects. If you try to call hash() on a mutable object (such as a list), or try to use a mutable object for a dictionary key, you'll get an error:

>>> spam = {['hello', 'world']: 42}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

>>> d = {'a': 1}
>>> spam = {d: 42}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'

Tuples, being immutable objects, can be used as dictionary keys:

>>> spam = {('a', 'b', 'c'): 'hello'}

...or can they?:

>>> spam = {('a', 'b', [1, 2, 3]): 'hello'}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

It seems that if a tuple contains a mutable object, it can't be hashed. (Raymond Hettinger explains, with lots of context, why immutable tuples can contain mutable values.) This fits with what we know: immutable objects can be hashable, but this doesn't necessarily mean they're always hashable. And remember, the hash is derived from the object's value.

This is an interesting corner case: a tuple (which should be immutable) that contains a mutable list cannot be hashed. This is because the hash of the tuple depends on the tuple's value, but if that list's value can change, that means the tuple's value can change and therefore the hash can change during the tuple's lifetime. But if we go back to the glossary definition of hashable, a hash is never supposed to change during the object's lifetime.

Does this mean that the value of tuples can change? Are tuples mutable?

So, Are Tuples Mutable or Immutable?

Before we can finally answer this question, we should ask, "Is mutability a property of data types or of objects?"

(Update: Our BDFL generally thinks it's of data types.)

Python programmers often say things like "strings are immutable, lists are mutable", which makes us think that mutability is a property of types: all string objects are immutable, all list objects are mutable, etc. And in general, I agree.

But in the previous section, we saw how some tuples are hashable (implying they're immutable) but some other tuples aren't hashable (implying they're mutable).

Let's go back to the official Python documentation's definition of immutable and mutable: "An object with a fixed value" and "Mutable objects can change their value", respectively.

So perhaps mutability is a property of objects, and some tuples (that contain only immutable objects) are immutable and some other tuples (that contain one or more mutable objects) are mutable. But every Pythonista I've run into says and continues to say that tuples are immutable, even if they're unhashable. Why?

In one sense, tuples are immutable because the objects in a tuple cannot be deleted or replaced by new objects. Just like spam = 42; spam = 99 doesn't change the 42 object in spam; it replaces it with an entirely new object, 99. If we use the interactive shell to look at a tuple that contains a list:

>>> spam = ('dogs', 'cats', [1, 2, 3])
>>> id(spam[0]), id(spam[1]), id(spam[2])
(41506216, 41355896, 40740488)

The same objects will always be in this tuple, and they will always have the same identities in the same order: 41506216, 41355896, and 40740488. Tuples are immutable.

However in another sense, tuples are mutable because their values can be changed. Enter the following into the interactive shell:

>>> a = ('dogs', 'cats', [1, 2, 3])
>>> b = ('dogs', 'cats', [1, 2, 3])
>>> a == b
True
>>> a is b
False

In this example, the tuples referred to by a and b have equal values (according to ==) but are different objects (according to is). Let's change the list in a's tuple:

>>> a[2].append(99)
>>> a
('dogs', 'cats', [1, 2, 3, 99])
>>> a == b
False

We have changed the a's value. We must have, because a is no longer equal to b and we didn't change b's value. Tuples are mutable.

Epilogue

'The question is,' said Humpty Dumpty, 'which is to be master — that's all.'

Humans are the masters of words, not the other way around. Words are invented by humans to convey ideas to other humans. Personally, I'm going to continue to say that tuples are immutable, because this is more accurate than not and the most helpful term in most contexts.

However, at the same time we should also be aware that from our definitions of "mutable" and "value", we can see that tuples' values can sometimes change, i.e. mutate. It isn't technically wrong to say that tuples can be mutable, though it will certainly raise eyebrows and require more explanation.

Thank you for sitting through my, what turned out to be lengthy, explanation on tuples and mutability. I certainly learned a lot in writing this post, and hope I conveyed it to you as well.

Comments