Types in Python

mypy

pip install mypy

Changing types

Even without any additional work, running mypy on an existing code-base can reveal locations that might need fixing.

For example it can point out places where the content of a variable changes type. Python accepts this, and in some places this type of flexibility might have advantages, but it can also lead to confusion for the maintainer of this code.


x = 23
print(x)

x = "Python"
print(x)

x = ["a", "b"]
print(x)

python simple.py works without complaining.

mypy simple.py reports the following:

simple.py:5: error: Incompatible types in assignment (expression has type "str", variable has type "int")
simple.py:8: error: Incompatible types in assignment (expression has type "List[str]", variable has type "int")
Found 2 errors in 1 file (checked 1 source file)

Changing types when reading a number

A quite common case in the real-world when you read in something that is supposed to be a number. In terms of the Python type-system the input is always a string. Even if it looks like a number. We then need to convert it to int() or to float() to use them as such.

People will often reuse the same variable to first hold the string and then the number. This is ok with Python, but might be confusingt to the reader.


num = input("type in an integer: ")
print(num)
print(type(num).__name__)   # str

num = int(num)
print(num)
print(type(num).__name__)   # int

mypy input.py will print the following:

input.py:6: error: Incompatible types in assignment (expression has type "int", variable has type "str")
Found 1 error in 1 file (checked 1 source file)

Types of variables


x :int = 0

x = 2
print(x)

x = "hello"
print(x)

python variables.py

2
hello

mypy variables.py

variables.py:7: error: Incompatible types in assignment (expression has type "str", variable has type "int")
Found 1 error in 1 file (checked 1 source file)

Types of function parameters


def add(a :int, b :int) -> int:
    return a+b

print(add(2, 3))
print(add("Foo", "Bar"))

5
FooBar

function.py:6: error: Argument 1 to "add" has incompatible type "str"; expected "int"
function.py:6: error: Argument 2 to "add" has incompatible type "str"; expected "int"
Found 2 errors in 1 file (checked 1 source file)

Types function returns None or bool

-> bool means the function returns a boolean. Either True or False.

-> None means the function returns None. Explicitely, or implicitely.

def f() -> bool:
    return True

def g() -> None:
    return True


def h() -> None:
    return None

def x() -> None:
    return

def z() -> None:
    pass

function_bool.py:5: error: No return value expected
Found 1 error in 1 file (checked 1 source file)

Types used properly


def add(a :int, b :int) -> int:
    return a+b

print(add(2, 3))

x :int = 0

x = 2
print(x)

5
2

Success: no issues found in 1 source file

TODO: mypy

Complex data structures?
My types/classes?
Allow None (or not) for a variable.

from typing import Generator

def numbers(n: int) -> Generator[int, None, None]:
    return ( x for x in range(n))

print(list(numbers(10)))

from typing import List

def numbers(n: int) -> List[int]:

    return list(range(n))

print(numbers(10))

from typing import IO
import typing

fh1: IO = open("data.txt")

fh2: typing.IO = open("data.txt")

def f() -> int:
    file: Op[str] = None
    return 42


def func(param: int | str) -> int:
    if isinstance(param, int):
        return param
    elif isinstance(param, str):
        return len(param)

# In this case mypy understands that the if and elif cover all the valid options and does not complain about missing `return`
# If we remove the `elif` part we can see the mypy complaint.
# In another similar case in real code I saw it comlaining even when all the possibilities were handled.


class A:
    pass
class B:
    pass

def func2(param: A | B) -> int:
    if isinstance(param, A):
        return 1
    elif isinstance(param, B):
        return 2

Case studies

tskit

I created an issue to add type-annotation.
Sent a pull-request.
- Add mypy to pre-commit
- Add mypy to GitHub Actions.
- Add .mypy.ini excluding all the current errors.

It was rejected based on a decision made in 2020.

I should have searched for this more, before putting in the work.

Opened an issue to add mypy and type-annotation.
Created a pull-request.

linkchecker

I found no type annotation in the linkchecker project.

Opened an issue to add mypy and type-annotation
Created a pull-request to setup mypy in the CI and add configuration file.

veusz

I found no type-annotation in the veusz project.

Opened an issue to add mypy to the CI and add type-annotation
Created a pull-request
mypy was complaining about some missing type stubs. So I installed h5py-stubs. That made mypy complain about other things that lead me opening an issue about the try/except block around import h5py which might be a left-over from early development when h5py was not a hard requirement of the project.
I have another branch from which I am going to send a PR once the first one is accepted.

Process

Run mypy .. Instead of . you might want to run it on a subfolder. This will generate lots of errors.

Edit the `.mypy.ini`

Globally disable the errors to make mypy pass.

[mypy]
disable_error_code = assignment, ...

Disable certain errors by filename (without the .py) or by folder (with *).

[mypy-project/submodule.*]
disable_error_code = assignment, import-not-found ...

This can be done only for modules (where we have __init__.py). All the other files can be excluded using regexes:

[mypy]
exclude = (?x)(
    ^tests/runstests.py$
    | Documents/manual-source/conf.py
    | examples/myexample.py
    | setup.py
  )

One-by-one remove the entries from the global disable_error_code and disabled them in a more file-specific level. Keep making this more and more specific.
Take one of the excluded files, remove it from the exclude list and fix the issues.
Remove one of the entries of the disable_error_code from one of the files or modules and fix the code.
It is not a very good idea to disable name-defined, but sometimes it is necessary. It is better to add # type: ignore[name-defined] to the few lines where it complains and then enable it again. Otherwise we might easily add incorrect type-names.

Python other chapters