Skip to content

3.5. Documentations

What is software documentation?

Software documentation is the collection of written text, illustrations, and code commentary that accompanies a software project. It serves as a guide for everyone involved, from end-users to developers. Its purpose is to explain what the software does, how it works, and how to interact with it, ensuring the project is understandable, usable, and maintainable.

Why is documentation essential?

In any software project, especially in MLOps, documentation is not a "nice-to-have"—it's a cornerstone of success. Here’s why:

  • Clarity and Onboarding: It provides a single source of truth, helping new team members (and your future self) quickly understand the project's architecture, code, and processes.
  • Collaboration: MLOps involves diverse teams (data science, engineering, operations). Clear documentation ensures everyone speaks the same language and understands their role in the project lifecycle.
  • Maintainability and Scalability: Well-documented code and systems are easier to debug, update, and scale. It reduces dependencies on individual "heroes" who hold all the knowledge.
  • Adoption and Impact: For your models and tools to be used, others must understand how to interact with them. Good documentation details accepted inputs, expected outputs, and potential failure modes, encouraging wider adoption.
  • Quality and Reproducibility: The act of documenting forces you to clarify your thinking, often revealing design flaws or bugs. In MLOps, it's also critical for reproducing experiments and model results.

How do you document Python code with docstrings?

The primary way to embed documentation directly within Python code is by using docstrings. These are string literals that appear as the first statement in a module, function, class, or method definition. Tools can then automatically extract these to generate API documentation.

  • Module Docstrings: Placed at the top of a file, they describe the module's purpose and contents.
"""Defines trainable machine learning models and their components."""
  • Function and Method Docstrings: They explain what the function does, its arguments, what it returns, and any exceptions it might raise.
def parse_config_file(path: str) -> Config:
    """Parse a configuration file from a given path.

    Args:
        path: The local path to the configuration file.

    Returns:
        The parsed representation of the configuration file.
    """
    return oc.OmegaConf.load(path)
  • Class Docstrings: These describe the class's purpose, its attributes, and its methods.
class ParquetReader(Reader):
    """Reads a pandas DataFrame from a Parquet file.

    Attributes:
        path: The local path to the Parquet dataset.
    """

    path: str

While docstrings are essential for documenting your code's API, they should be complemented by external documentation (like READMEs or a full documentation site) for higher-level guides, tutorials, and architectural overviews.

What are the best tools, formats, and conventions?

Choosing the right combination of tools, formats, and conventions is key to efficient and effective documentation.

Tools

  • MkDocs: A fast, simple static site generator perfect for project documentation. It uses Markdown and is easy to configure. This course uses it!
  • Sphinx: The powerhouse of Python documentation. It's incredibly robust and feature-rich, using reStructuredText by default. It has a steeper learning curve but is the standard for many large projects.
  • pdoc: A lightweight tool that auto-generates API documentation from your project's docstrings with minimal configuration.

Docstring Formats & Conventions

  • Google Style: Simple, readable, and easy to write. It's an excellent choice for most projects due to its clarity.
  • Numpy Style: More structured and verbose than Google style, it's particularly good for scientific and numerical computing projects where detailed parameter descriptions are crucial.
  • reStructuredText (reST): The most feature-complete format, used natively by Sphinx. It offers powerful constructs like cross-referencing but is more complex than Markdown-based styles.

Automation

  • Docstring Generation: IDE extensions like autoDocstring for VS Code can automatically generate docstring templates for your functions and classes, saving time and ensuring consistency.
  • API Documentation Generation: You can generate your API documentation from the command line. For example, using pdoc with Google-style docstrings:
$ uv run pdoc --docformat=google --output-directory=docs/api/ src/your_package

How should you structure documentation?

A great way to structure technical documentation is the Diátaxis framework. It proposes that all documentation serves one of four purposes and should be organized accordingly: Tutorials, How-To Guides, Reference, and Explanation.

The Diátaxis Framework, showing four quadrants: Tutorials, How-to guides, Reference, and Explanation, organized by their purpose from practical to theoretical and from study-oriented to work-oriented.

  • Tutorials are learning-oriented lessons that take a user by the hand through a series of steps to complete a project.
  • How-To Guides are goal-oriented steps that solve a specific problem, like "How to deploy a model to a staging environment."
  • Reference material is information-oriented, providing technical descriptions of the machinery, such as API documentation generated from docstrings.
  • Explanation material is understanding-oriented, clarifying and illuminating a particular topic, like "Why we chose a serverless architecture."

Adopting this structure helps users find exactly what they need, whether they are trying to learn, accomplish a task, get a technical detail, or deepen their understanding.

What makes MLOps documentation unique?

While MLOps projects share documentation needs with traditional software, they have unique components that require special attention:

  • Data Documentation: This is crucial for reproducibility and debugging. It should include:

    • Data Dictionaries: Descriptions of each feature (e.g., age, purchase_amount).
    • Data Schema: The expected data types, formats, and constraints (e.g., age is an integer between 0 and 120).
    • Lineage and Versioning: Where the data came from, how it was transformed, and which version was used for training a specific model.

  • Model Documentation: Often captured in Model Cards, this provides a comprehensive overview of a model's capabilities and limitations. Key elements include:

    • Architecture: The type of model and its structure (e.g., ResNet-50, DistilBERT).
    • Performance Metrics: Evaluation results on different datasets and data segments (e.g., accuracy, F1-score, MAE).
    • Intended Use & Limitations: Where the model excels and, just as importantly, where it might fail or exhibit bias.

  • Experiment Documentation: To ensure scientific rigor, every experiment should be documented. This is often handled by tools like MLflow or DVC, but the process and key findings should be summarized. Document:

    • Parameters: Hyperparameters, feature engineering steps.
    • Metrics: The results of the experiment.
    • Artifacts: Links to the trained model, visualizations, and logs.

  • Pipeline Documentation: The CI/CD and ML pipelines that automate your process must be documented, showing how a code change or new data triggers training, evaluation, and deployment.

What are best practices for documentation?

  1. Write for Your Audience: Tailor the language and depth of detail to the intended reader. A data scientist needs different information than an operations engineer.
  2. Keep It Updated, Automatically: Outdated documentation is worse than no documentation. Integrate documentation updates into your development workflow and automate generation and deployment wherever possible.
  3. Examples Are Essential: Provide clear, copy-pasteable code examples for common use cases. Show how to call your API, run your training script, or interpret a model's output.
  4. Document the "Why": Don't just explain what the code does; explain why it was designed that way. What trade-offs were made? What alternatives were considered? This is invaluable for future maintainers.
  5. Create a Contribution Guide: If you want others to contribute, tell them how. Provide clear guidelines on setting up the development environment, running tests, and submitting pull requests.
  6. Establish a Feedback Loop: Make it easy for users to report issues or suggest improvements for the documentation, for example, by linking to your project's issue tracker.
  7. Use Visuals: Diagrams of your architecture, ML pipelines, or model performance charts can often communicate complex ideas more effectively than text alone.
  8. Version Your Docs: Just like your code and models, your documentation should be versioned. This ensures users can find the documentation that corresponds to the specific version of the software they are using.

Additional Resources