4.0. Typing

What is programming typing?

Typing is the practice of assigning a specific data type (e.g., string, integer, boolean) to variables, function arguments, and return values. This discipline governs how data is used, ensuring that operations are performed on compatible types.

Programming languages approach typing in different ways:

Static Typing: Types are checked at compile time, before the code is run. This catches errors early in the development cycle. Examples include Java, C++, and Rust.
Dynamic Typing: Types are checked at runtime, as the code executes. This offers flexibility but can lead to unexpected errors. Python, Ruby, and JavaScript are dynamically typed.
Gradual Typing: A hybrid approach that allows developers to introduce static type checking into a dynamically typed language. Python (version 3.5+) supports this through optional type hints, offering the best of both worlds.

Additionally, type systems can be categorized by their strictness:

Strong Typing: The language enforces strict rules about how types can interact. For example, you cannot add an integer to a string without explicit conversion (str(1) + "s"). Python is strongly typed.
Weak Typing: The language automatically converts types when performing operations (a process called type coercion), which can sometimes lead to unpredictable results (e.g., in some languages, 1 + "s" might result in "1s").

Why is typing useful in programs?

Adopting a typing discipline, especially in large-scale MLOps projects, delivers significant advantages:

Fewer Bugs: Catching type-related errors before runtime prevents a common class of bugs that can be difficult to track down in production.
Improved Readability: Type annotations act as a form of documentation, making it immediately clear what kind of data a function expects and returns.
Better IDE Support: Modern IDEs like VS Code leverage type hints to provide intelligent code completion, error highlighting, and safe refactoring.
Enhanced Collaboration: A clear type system ensures that all team members understand the data structures and interfaces, reducing integration friction.
Greater Confidence: Writing typed code provides a safety net, giving developers more confidence when modifying or extending the codebase.

While adding types requires an initial effort, the long-term payoff in code quality, maintainability, and reliability is substantial.

What is the relation between Python and typing?

Python is a dynamically and strongly typed language. Historically, developers did not need to declare variable types. However, this flexibility can become a liability in complex applications.

Since Python 3.5, the language officially supports gradual typing through type hints. These are optional annotations that specify the expected types. It's crucial to understand that the Python interpreter itself does not enforce these hints at runtime; they are primarily for static analysis tools.

Consider this simple function without type hints:

def print_n_times(message, n):
    for _ in range(n):
        print(message)

With type hints, the function's intent becomes much clearer:

def print_n_times(message: str, n: int) -> None:
    for _ in range(n):
        print(message)

The annotation -> None explicitly states that the function does not return a value. Using type hints is a best practice for modern Python development, as it significantly improves code clarity and robustness.

To learn more, consult the official typing module documentation and the type hints cheat sheet. Type hints follow the same standard syntax regardless of which type checker you use to enforce them.

How can you add types to a dataframe?

Yes, by using a data validation library like Pandera. Pandera provides a flexible and intuitive API to define schemas for dataframe-like objects, making data processing pipelines more robust and readable.

Key features of Pandera include:

Unified Schema Definition: Define a schema once and use it to validate various dataframe libraries, including pandas, Dask, Modin, and PySpark.
Type and Property Checks: Enforce column data types, value ranges, and other properties.
Advanced Validation: Perform complex statistical validations, such as hypothesis testing.
Seamless Integration: Use function decorators to integrate data validation directly into your pipelines.
Pydantic-Style Models: Define schemas using a class-based API that feels familiar to Pydantic users.
Data Synthesis: Generate synthetic data from a schema for property-based testing.

Here is an example of a Pandera schema for validating input data in an MLOps project:

import pandera as pa
import pandera.typing as papd
import pandera.typing.common as padt

class InputsSchema(pa.DataFrameModel):
    """Schema for the project inputs."""

    instant: papd.Index[padt.UInt32] = pa.Field(ge=0, check_name=True)
    dteday: papd.Series[padt.DateTime] = pa.Field()
    season: papd.Series[padt.UInt8] = pa.Field(isin=[1, 2, 3, 4])
    yr: papd.Series[padt.UInt8] = pa.Field(ge=0, le=1)
    mnth: papd.Series[padt.UInt8] = pa.Field(ge=1, le=12)
    hr: papd.Series[padt.UInt8] = pa.Field(ge=0, le=23)
    holiday: papd.Series[padt.Bool] = pa.Field()
    weekday: papd.Series[padt.UInt8] = pa.Field(ge=0, le=6)
    workingday: papd.Series[padt.Bool] = pa.Field()
    weathersit: papd.Series[padt.UInt8] = pa.Field(ge=1, le=4)
    temp: papd.Series[padt.Float16] = pa.Field(ge=0, le=1)
    atemp: papd.Series[padt.Float16] = pa.Field(ge=0, le=1)
    hum: papd.Series[padt.Float16] = pa.Field(ge=0, le=1)
    windspeed: papd.Series[padt.Float16] = pa.Field(ge=0, le=1)
    casual: papd.Series[padt.UInt32] = pa.Field(ge=0)
    registered: papd.Series[padt.UInt32] = pa.Field(ge=0)

How can you improve type safety in classes?

While Python's built-in @dataclass is useful, Pydantic takes class-based data modeling to the next level by performing data validation and serialization at runtime.

Why use Pydantic?

Type Enforcement: Pydantic validates data against your type hints and raises clear errors if the data is invalid.
High Performance: The core validation logic is written in Rust, making it extremely fast.
JSON Schema Support: Automatically generate JSON Schemas from your models for seamless API integration.
Strict and Lax Modes: Choose between strict type checking or allowing Pydantic to coerce data into the correct type.
Broad Compatibility: Works with many standard library types, including dataclasses and TypedDict.
Rich Ecosystem: Integrates with popular libraries like FastAPI, Typer, and SQLModel.

Here is an example of using Pydantic to define a configuration object in an MLOps codebase:

import pydantic as pdt

class GridCVSearcher(pdt.BaseModel):
    """Grid searcher with cross-fold validation for better model performance metrics."""

    n_jobs: int | None = None
    refit: bool = True
    verbose: int = 3
    error_score: str | float = "raise"
    return_train_score: bool = False

How can you check types in Python?

The recommended tool for static type checking is ty, a new type checker from Astral (the makers of uv and Ruff). Like its siblings, ty is written in Rust and is exceptionally fast, so it can run on every save and on every commit without slowing you down. It can be run from the command line or integrated directly into your editor through its language server.

# Install ty into your "check" dependency group
uv add --group check ty

# Type-check your source and test code
uv run ty check

By default, ty check checks the whole project; you can also pass explicit paths (e.g. uv run ty check src/ tests/).

Note: ty is still pre-1.0 and moving fast. Pin it to a compatible range (e.g. ty>=0.0.56,<0.1) rather than an open-ended version, and treat it as a local and CI check that you upgrade deliberately, since new releases can add or refine diagnostics.

Several other type checkers exist, and you will encounter them in older projects:

Mypy: The original and most mature Python type checker, with a broad plugin ecosystem.
pyright: Developed by Microsoft and serves as the engine for Pylance in VS Code.
pyre-check: A performant type checker developed by Meta.
pytype: A static type analyzer from Google that can infer types for untyped code.

All of these tools read the same standard type hints, so you can adopt annotations today and swap checkers later with little friction.

How can you configure ty for your project?

You configure ty by adding a [tool.ty] section to your pyproject.toml file, so every developer and CI run share the same rules. Remember to add the .ty_cache/ directory to your .gitignore file.

ty infers the target Python version from requires-python, but you can set it explicitly. You can also tune individual diagnostics: because ty is pre-1.0, it does not yet fully model the dynamic typing of some machine learning libraries (Pydantic, Pandera, MLflow). Relaxing the categories they trigger keeps ty focused on catching real errors, such as undefined names and bad calls.

[tool.ty.environment]
# Target Python version for type resolution.
python-version = "3.14"

# ty (pre-1.0) does not model every dynamic ML library yet; relax the
# categories they trigger so ty still gates real errors.
[tool.ty.rules]
invalid-argument-type = "ignore"  # dynamic ML/MLflow call sites
invalid-type-form = "ignore"      # pandera typed Series / dynamic type forms
unresolved-attribute = "ignore"   # dynamic attributes on MLflow/pandera objects

If you need to bypass a diagnostic for a specific line, add a # ty: ignore comment with the rule name:

dataset = mlflow.data.from_pandas(df)  # ty: ignore[unresolved-attribute]

Run uv run ty explain rule <name> to understand a diagnostic before silencing it. For all options, refer to the ty documentation and its rules reference.

What are the best practices for typing in Python?

Apply the 80/20 Rule: Prioritize adding types where they provide the most value, such as in function signatures and data model definitions.
Master the typing Module: Become familiar with the tools in the typing module, like List, Dict, Optional, Union, and Protocol.
Use Implicit Typing Where Appropriate: You don't need to annotate every single variable. A type checker can often infer the type from its assignment (e.g., x = 5 is clearly an int).
Use typing.Any as a Last Resort: Avoid using Any whenever possible, as it effectively disables type checking for that variable.
Integrate Type Checking into CI/CD: Run ty as part of your continuous integration pipeline to catch type errors before they reach production.