Created by: gwideman, Nov 5, 2013 12:52 am
Revised by: gwideman, Dec 5, 2013 6:39 pm (21 revisions)

2013-11-06. Definitely a work in progress. Or at least, unfinished :-)

Meta-overview

Original intent

This article (and subsidiary pages), started as an attempt to create a coherent explanation of the Python packaging and deployment landscape, for myself and colleagues.
  • How to package and deploy a Python program or library to other users.
    • In that process, what variants are there, and why?
  • For development, what implications there are for project structure, layout and workflow

Effort frustrated!

My effort was frustrated by a number of factors, including:
  • The ongoing state of flux of the Python packaging and deployment landscape.
    • Including the transition from Python 2.x to 3.x
    • The interim status of packaging tools
  • The incomplete and often out-of-date and conflicting status of Python packaging and deployment documentation and advice (official and elsewhere).
    • Notably, the widespread omission of:
      • Explanations for certain fundamentals
      • Enumeration of different scenarios which call for different packaging/deployment approaches.

Effort retargeted

Consequently, this article and subsidiary pages have become:
  • Partly a model of what I think should be in the docs
  • Partly a collection of information distilled from existing docs and automatically extracted from source code
  • Partly explanations for some areas that I feel are currently glossed over in existing docs.
  • Partly critique noting particular problem areas in existing docs.
I hope these pages can serve as a collection of suggestions for improving the official docs, primarily those maintained by the Python Packaging Authority (PyPA). (See issues filed on the PyPA issues site: XXXX).

Outline of packaging/deployment topics

These are the topics that I suggest a developer needs at hand in order to proceed:
Part 1: Background
  • Scope/Overview: Overview of the issues within the domain of "Python packaging, deployment and installation".
  • History and ongoing developments
    • So that the developer can navigate the abundance of information which is outdated or otherwise not applicable.
  • The salient tools and formats
    • Packaging/deployment/installation tools: Plain old scripts and zips, distutils, setuptools, pip, zc buildout?, others?
    • Distribution package formats: plain zips (possibly with __main__.py), "sdist", "bdist", eggs, wheel, other?
Part 2: Use cases; choice of tools/formats
  • Use cases. Variations in project structure and deployment situations which motivate differing packaging and deployment requirements.
  • Distribution package format features and distinctions
  • Packaging/deployment/installation tool features and distinctions
      • Tools vs formats
      • Tools vs key features
Part 3: How to operate the tools
  • The setup.py/setup( ) business.
    • This detail is crucial for making sense out of existing docs.
  • The existing official docs
  • Comprehensive docs for the main tools
    • And their arguments and options
Part 4: Top issues with existing docs