Created by: gwideman, Dec 4, 2013 5:12 pm
Revised by: gwideman, Dec 4, 2013 5:12 pm (1 revisions)

.
  • Work in progress
  • Part of a series which starts here.

Top things to know


Top things to know in order to make sense of Python packaging-related topics, given current state of docs (2013-10-25)

1. Setuptools is really an addon for Distutils.


So you have to know something about Distutils as a starting point.

2. Distutils, in brief


Core services for packaging and installing Python project is provided by Distutils, which is one of the Python core libraries.

Distutils official documentation:

  • Installing Python Modules
  • Distributing Python Modules

which do cover useful topics, but are some ten to twelve years old, thus thoroughly suspect, and use terminology that's not current.

Also:

  • The Python core library docs, like any other library, [LINK]

...but this would be a relatively indirect way of learning how to perform packaging and install tasks (and see obtuseness notes below)

setup()


The primary way of using Distutils capabilities is through its setup() function. The setup() function accepts quite a number of different arguments, but it also digests (indeed requires) command line arguments. It is thus intended to be called from a wrapper script like this:

[setup.py()]

So the overarching idea is that the "user interface" for Distutils packaging and installation is a script, customarily named setup.py. When setup.py is invoked on the command line, the user must supply a "command" argument specifying the function which setup() should perform.

[command line]

The idea is ... [diagram]

  • args to setup() = metadata
  • command-line args to setup.py = command telling task to perform

So a canonical process would be:

[command...] to create a package, which includes within it a setup.py file...

[command] to install run that setup.pt and install that package (or somesuch)

Some obtusenesses


  • The single setup() function is used for all manner of distibution-related activities (not just "setting something up"), on both the developer end and the target user's end. It would have been more helpful if this command had been named "command()"
  • When people talk about using the Distutils install command, they mean calling some setup.py script with a particular command-line command argument.
  • Because the inputs to the setup() function come partly from it's arguments and partly from command-line arguments, the documentation suffers from the problem that these two types of arguments are, by rote, documented independently of each other. So the library docs tell about the arguments to the setup() function, disregarding the all-important command-line "command" argument, while docs about using setup.py tell about the command argument and neglect documenting which setup() arguments pertain to which commands. (LINKS)

[Expand]

3. Setuptools, in brief


Setuptools is not part of the Python core library, and must be installed separately.

To employ Setuptools again requires a setup.py script calling setup(), like this:

[script]

Note the change to import setuptools, and not distutils. The inferences that the reader might draw are

  • that setuptools is a replacement for distutils,
  • that setup() is a function supplied by setuptools,
  • and that it's setuptools docs that are now the salient documentation for what to do.

All of these suppositions are false. In actuality:

  • By importing setuptools, setup.py causes setuptools to import distutils and interleave some additional functionality to enhance base distutils.
  • The setup() call ostensibly to setuptools is passed right though to distutils, which is now able to recognize additional commands added by setuptools.
  • All the original functionality provided by setuptools is still available, and is often the functionality that needs to be called, despite the presence of setuptools
    • It's not clear to me which, if any, of the existing distutils commands are modified by setuptools.

So, to get up to speed on "using setuptools" requires first reading up on distutils functions, then understanding what setuptools adds.

4. What kind of packages do Distutils and setuptools produce?


[Track this down by examining all the command and argument variants. Sigh]

5. Using an installer instead of just setup.py


  • Why?
  • How does this even work?
  • Fetches the package in the first place?
  • Resolves dependencies? And fetches them?

6. Setuptools and eggs


One of the features introduced by setuptools is the "egg" Python package format. The basic egg format specifies (I think... need to check details) a particular naming convention, and a particular set of metadata.(Do eggs include a setup.py?)

Eggs can be created with setuptools command XXXX.

This allows installations to be able to do X and Y that base Distutils packages can't.

Since the recommendation for installation is to use pip, and pip supposedly can't install eggs, the implication is that eggs are not a recommended way to proceed and we should not create them. Although:

(a) pip documentation in some places says pip can install eggs, and in other places says pip cannot.

(b) pip itself is delivered as an egg.

7. EasyInstall


If we do have to install an egg then we can use setuptools EasyInstall tool apparent can install eggs.(OK, but why is this better than just setup.py? Dependencies? Anything else?)

However, EasyInstall is deprecated because of

  • [criticisms]
  • Too magical
  • Messes with... something... sys.path?

8. setup.py now considered bad practice


Although providing setup.py with a package gives the user a uniform way to install packages, it is now widely thought to be a bad idea that the user has to run an executable provided by the package in order to perform the installation. Better that the package should provide metadata as a data file (ie: not as arguments to setup() in setup.py), and then the user uses a known trustworthy tool to read that metadata and perform the installation.

  • This is a benefit with pip? (or does pip run the setup.py script?)
  • This is a direction supported by wheel format? Only supported by wheel, or does it apply to other formats?