The Invent with Python Blog

Writings from the author of Automate the Boring Stuff.

Sun 24 November 2019

Type Hints for Busy Python Programmers

Posted by Al Sweigart in python   

"Okay, I know that type annotations or type hints or whatever-"

Yes, "type annotation" and "type hint" mean the same thing.

"Right. I know what they are: they let you specify data types for your variables and function parameters & return values, which lets you catch certain kinds of bugs early and save you from lots of debugging and headache. I'm in. How do I use them?"

There's two parts. The first part is you add type hints to your source code. This involves the typing module that's a part of the Python Standard Library as of verison 3.5. The second part is you install the mypy module to do the actual checking.

"Wait, why are there two modules? That sounds complicated."

The first module, typing, lets you specify different data types. But the Python interpreter doesn't care if they're correct. The variable's actual value doesn't have to match the variable's type; the code will still run just fine.

"Doesn't that defeat the purpose?"

Nah, remember that in Python type hints are optional. If you're writing a quick script and don't care, you don't have to use type hints. The Python interpreter itself completely ignores type hints. No type checking happens at runtime. So, technically, type hints aren't "optional static typing" because "static typing" implies that variables always store values of a certain type. Python is still a dynamically typed language even with type hints. Type hints are just hints that third-party tools like Mypy can use to point out potential bugs.

It's the second module, mypy, that is the tool that actually checks that your source code is valid and all the types match. Mypy is made by a third-party, which is why we use two separate modules. You don't have to use Mypy. You could instead use any third-party module that does the checking that Mypy does.

"So what could I use instead of Mypy?"

Pyright, from Microsoft. Pyre, from Facebook. Pytype, from Google. They're all free and should work the same, but most people seem to use Mypy.

"So typing comes with Python 3.5 and later. How do I install Mypy?"

Run python -m pip install mypy. (Use python3 on macOS and Linux.) Add --user if you get error messages about permissions: python -m pip install --user mypy.

If the Python interpreter doesn't care about type hints, then how do I check if my source code matches the declared types?

Mypy installs a command line tool in Python's Scripts folder when you install Mypy. So you run mypy from the command line (not from the Python interactive shell) and if your code is valid, it prints nothing. If it's not, it will display errors. I generally use the python -m mypy style of running it.

But even easier is to get a plugin for whatever editor or IDE you use. This way, your editor runs Mypy in the background as you type code and can near-instantly show you any mistakes you make. For example, for Sublime Text, you just install the SublimeLinter and SublimeLinter-contrib-mypy using Package Control. Then, as you type Python code in Sublime Text, it displays any errors that Mypy finds automatically.

"When you say type hints in Python are optional, do you mean half my variables could use type hints and the other half doesn't have to?"

Yes. Python calls this gradual typing. If you want to add type hints to your Python code, you don't have do it all at once. It's like unit test coverage: you could just test some of your functions.

"Okay, I'll install Mypy and set up my editor to use it. Done. So what are all the types I use when writing type hints in my code?"

The Mypy site has a great cheat sheet. Basically, you use the same names as those type conversion functions, like int(), str(), and so on. That's because those are technically the class names for integers and strings. Just like you call, 10, 31) to create an object of the date class, int("42") creates an integer object of the int class. In Python, "class" and "type" and "data type" are the exact same thing.

"Okay. So how do I write type hints in my code?"

After the variable name, add a colon and the type name. Here's an example, with comments explaining each one:

numberOfTacos: int = 42  # This variable's type int declares it to be an int variable.
tacosEaten: int  # This declares tacosEaten to be an int variable, but doesn't set an initial value. It's not set to None: using it will result in NameError.

# Type hints without an initial value are handy when there's no one place where an initial value is set, like here:
if numberOfTacos < 10:
    tacosEaten = numberOfTacos
    tacosEaten = 10

piesEaten: float = 3.14
myCatsName: str = 'Zophie'
isBestCat: bool = True
someBytes: bytes = b'HELLO'
randomValues: list = ['Zophie', 99, 3.1415]  # Note that the list can contain values of any type.
goodCats: tuple = ('Zophie', 'Pooka', 'Simon', 'Fat-tail', 'Footfoot')
fruitCount: dict = {'apples': 5, 'tomatoes': 7}
barbers: set = set(['Alice', 'Bob', 'Carol'])
shavers: frozenset = frozenset(['Alice', 'Bob', 'Carol'])

"Cool. So what happens when I write code that sets a variable to the wrong type of value?"

In Python, nothing. When you run the Python interpreter to run your program, it doesn't care about type hints. It just sets the variable to the value. But, before that point, when you're writing the source code in an editor that runs Mypy in the background, Mypy will report to the editor that there's a problem and the editor will show it to you as you're typing.

numberOfTacos: int = 42  # The numberOfTacos variable should only have ints.
numberOfTacos = 'bleventeen'  # Your editor will display a warning on this line. But Pyhton will still run this code just fine.

"What about the values inside of the list, tuple, dictionary, and other container types? Can I specify type hints for those?"

Yes, and you do this adding square brackets with the item types inside. But cats: list[str] = ['Zophie', 'Pooka'] isn't valid Python code. (The reasons why this style couldn't work is too long to go into here.) The solution that the Python core devs came up with is to have analogous classes in the typing module. Here's an example:

from typing import List, Tuple, Dict, Set, Frozenset

groceryList: List[str] = ['bread', 'eggs', 'tofu']
cityHall: Tuple[float, float, str] = (37.779, -122.419, 'San Francisco')
fruitCount: Dict[str, int] = {'apples': 5, 'tomatoes': 7}
barbers: Set[str] = set(['Alice', 'Bob', 'Carol'])
shavers: Frozenset[str] = frozenset(['Alice', 'Bob', 'Carol'])

"What if I want to have a list that can contain multiple types of values for its items?"

Use the Union type from the typing module. For example:

from typing import Union
favoriteLettersAndNumbers: List[Union[int, str]] = [42, 'x', 86]

You can also use Union to specify variables that can have more than one type:

from typing import Union
serialNumber: Union[int, str] = 42
serialNumber = '42b'

"Okay. So if I have a variable that contains integers, but could also contain None, should I use, uh, Union[int, NoneType]?

You can use Optional instead, which means the same thing. And technically, NoneType isn't a defined type the way int or str are. So you can't say type(None) == NoneType the way you can say type(42) == int. Instead, we'd just use None like in this example:

from typing import Union, Optional

# These two have the same type hint, but `Optional[int]` is easier to write and read:
eggs: Union[int, None] = 42
bacon: Optional[int] = 42

But here's the thing: you should avoid using "null" values like None at all. Computer scientist and inventer of the null value Tony Hoare called it his Billion Dollar Mistake because it's often the cause of bugs. I vaguely remember some study that said NullReferenceException was by itself the cause of something like 30% of all Java application crashes. Kotlin, a language that is a sort of modernized Java, by default doesn't let you set variables to null. You can still use None if you need to, but don't use it without forethought.

"What if I don't care what type of a value a variable has. Should I just not use type hints?"

No, remember that "explicit is better than implicit." You should explicitly use the Any type for variables that can be of any type:

from typing import Any
spam: Any = 42
spam = 'hello'  # This is fine. `spam` can be set to any type.

Can I do more specific checks? Like, specify that a variable should be an int, but never a negative integer?

Nope. Type hints are only for types, not for values. I'd recommend subclass the class or, since you can't subclass int, you create your own non-negative integer class. Anyway, remember that type hints only work before runtime. They don't do runtime checking of values. You should use assert or even just if statements for that.

"What about classes I write or other types like or the match objects returned from'[aeiou]', 'hello')?

You just use the class name as the type. Remember, in Python the terms type, data type, and class all mean the same thing. Also note that self isn't written with a type hint:

class Cat:
    def __init__(self, name: str, color: str, weightkg: float) -> None: = name
        self.color = color
        self.weightkg = weightkg

def getZophieClone() -> Cat:
    return Cat('Zophie', 'gray', 4.9)

"What about weird types, like functions or sequences or mappings or iterables?"

The typing module has Callable, Sequence, Mapping, Iterable, and others. You can find them in the documentation.

"Alright, cool. So how about functions? How do I write type hints for parameters and return values?"

These are all done in the def statement. The return value's type hint is specified after a -> arrow. For example:

from typing import Optional, List

def getVowelsFromWord(word: str, maxVowels: Optional[int] = None) -> List[str]:
    vowels: List[str] = []
    for letter in word:
        if maxVowels is not None and len(vowels) >= maxVowels:
            break  # Break early since we've reached the maxmimum.

        if letter.upper() in 'AEIOU':
            vowels.append(letter) # Record this vowel letter.
    return vowels  # Return all the vowels fown in word.

In our getVowels() function, it has two parameters, word (which can be a string) and maxVowels (which can be an integer or None). The function itself returns a list of strings. A tool like Mypy can verify that the return statement returns the vowels variable, which has the type hint List[str], which in turn matches the function's return type hint.

"What about type hints in Python 2? Won't those colons cause syntax errors?"

Yes, but you can put type hints inside comments if you're writing Python 2 (or pre-Python 3.5) compatible code:

from typing import List # Required even for comment-style type hints.

spam = 42 # type: int
def sayHello():
    # type: () -> None
    """The docstring comes after the type hint comment."""

def addTwoNumbers(listOfNumbers):
    # type: (List[float]) -> None
    listOfNumbers[0] += listOfNumbers.pop()

Note that you'll still need to import objects from typing even with the comment-style of type hints.

"I've started using type hints, but I've come across a warning that doesn't really apply to my code. How do I get Mypy to ignore that one line?"

You can add a comment # type: ignore to the end of that line:

from typing import Any

def raiseMyCustomException(val: Any):
    # Some code here...
    raise Exception('message here: ' + str(val))

def getLengthIfString(val: Any) -> int:
    # Some code here...

    # This function call always raises an exception, but mypy thinks
    # gives the error "missing return statement", so we tell mypy to ignore it:
    raiseMyCustomException(val)  # type: ignore

"Cool! Now I know enough to get started. Where can I learn more about type hints?"

Check out the official Python documentation or the Mypy documentation, starting with the cheat sheet. There was also a well-received talk at PyCon 2019 on type hints by Carl Meyer.

Learn to program for free with my books for beginners:

Sign up for my "Automate the Boring Stuff with Python" online course with this discount link.