Created by: gwideman, Oct 8, 2013 9:08 pm
Revised by: gwideman, Oct 23, 2013 8:37 pm (19 revisions)


In an effort to discover the true behavior of Windows Python Launcher, this page examines the main source code for py.exe, launcher.c, at
The source code interacts with the following items, which affects its decisions:
  • builtin_virtual_paths: A list of specific paths to python, hard-coded in launcher.c, which may appear in the shebang line of a script, which Launcher treats specially.
  • builtin_prefixes: A list of specific directories, hard-coded in launcher.c, which Launcher removes from a script's shebang line, if present.
  • commands: A list of key-value pairs, defining "custom virtual shebang-line commands". Launcher reads these from the global and/or user's py.ini file, [command] section.
    • Launcher may also find commands in Windows PATH. See later for details.
  • installed_pythons: A list of INSTALLED_PYTHON records corresponding to all python installations found recorded in the registry.
    • INSTALLED_PYTHON records:
      • version: 4 characters, eg: 2.7 or 3.3
        • Registry doesn't seem to record, and Launcher doesn't discover, the third version digit
      • bits: 32 or 64
      • executable: Path to python.exe for this version
    • Sorted from highest-numbered version to lowest, with 64-bit before 32-bit of the same version
      • Sort order for multiple X.Y.* releases? Sort order seems not to be defined.
  • magic_values: Hardcoded into Launcher. Translates the "magic number" values in first two bytes of compiled python file into python version.

launcher.c source code steps

launcher.c source code version: a6c53b8
Decision locations
py.exe args
Error exit
Run (invoke1) python with args: ""
If no python installed

-X [script]
-X.Y [script]
Check that arg is valid -X or -X.Y format and X and Y are valid digits.
If yes, locate_python("X.Y"), run (invoke1) python with Args: rest-CL
If not X.Y format, fall through...
If X.Y format, but no such version installed

Show py help. Run (invoke1) python with same arguments (so it shows help).
If no python installed

script [args]
-arg(s) script [args]
scriptpath = find first arg not prefixed with '-'.
maybe_handle_shebang(scriptpath, all-of-py's args)

script is compiled
Get magic number (first 2 bytes) of file. find_by_magic() translates to X.Y, then locate_python().
Run python (invoke2) with all-of-py's args.
No, but maybe it should?
If find_by_magic() fails, then resulting path through maybe_handle_shebang() code is haphazard.
Starts with one of the builtin virtual paths:
  • "/usr/bin/python"
  • "/usr/local/bin/python"
  • "python"
Isolate "version suffix" (if any) joined to end of virtual path. Example: "/usr/bin/python3.2" --> "3.2". Use version suffix (even if "") in locate_python(suffix). Run python (invoke5). Args: ???
Fails if "version characters" are invalid X.Y, or python version unavailable.
Precludes custom version paths (ie: custom virtual shebang-line commands) based on these same directories, whose command name starts with "python". Eg: "/usr/bin/pythonABC" or even just "pythonABC".
parse_shebang, maybe_handle_shebang/
Is exactly builtin virtual path using env:
  • "/usr/bin/env python"
find_on_path("python"), and run it (invoke4). Args: ???
If no python on PATH, fall through...

Unix behavior in this case is to run env, with the first arg as the program for env to find on the PATH, and then run. Does not need to be specifically plain "python"
parse_shebang, maybe_handle_shebang/
Starts with builtin virtual path using env:
  • "/usr/bin/env python"
Like: "/usr/bin/env python3.2"
Proceeds as for other /builtin-virtual-paths/python cases. Ie: Uses locate_python(suffix), (not find_on_path()). Run python (invoke5).
Fails if "version characters" are invalid X.Y, or python version unavailable.
Odd: 'env python' uses PATH, whereas 'env python2' uses locate_python().
parse_shebang, maybe_handle_shebang
Not a builtin virtual path
Take first "argument" in shebang-line, remove built-in prefix if present, look up result:
  • in PATH
  • in custom commands defined in py.ini [commands] section
If found, run command (invoke3). Args: ???
(no error)
Odd: command that happens to be on PATH cannot be overridden by command definition in py.ini.
parse_shebang, maybe_handle_shebang, process

Present, but not a builtin, custom or PATH-findable command
locate_python(""); invoke1.
No python installed


  • rest-CL: Rest of py.exe command-line, following parameter(s) that Launcher processed itself.

locate_python algorithm

document here

py.ini configuration file format

  • Launcher does not interpret the default settings in multiple iterations. That is to say, specifying 'python=3', and then 'python3=3.2', results in Launcher interpreting a shebang-line containing just 'python' as a request for version 3, not for version 3.2 specifically.
    • And somewhere Launcher's doc is incorrect on this point
  • Does Launcher care about slash vs backslash???

Hard-coded Lists

  /usr/bin/env python

Potential surprises

To be verified:
  • Default py command behaves differently than python command.
    When there is no py configuration (ie: no py.ini [default] sections and no corresponding environment variables) and no shebang-line, issuing command-line 'py script' or just plain 'script' will use locate_python's algorithm, which defaults to selecting version=2. In particular, this scenario uses locate_python instead of using PATH, in contrast to 'python script', which does use PATH.
  • Version specifier X.Y-32 but no X.Y-64.
    • A request not specifying -32 means accept latest of either, and choose 64 if both 32 and 64 of same X.Y available
    • You can specify "latest 32", but not "latest 64".
    • So if you have installed 3.1-64 and 3.3-32, there's no way for a script to request and get the 64-bit python.
    • The implication is that scripts ought not have dependencies on 64-bit.
  • Version specificity is limited to X.Y, (not X.Y.Z)
    • In py.exe command line, py.ini [defaults], and shebang-line, versions can only be specified to X.Y, not X.Y.Z
    • locate_python algorithm doesn't appear to necessarily choose highest installed version within X.Y range. (request 3.2 might select 3.2.1 instead of 3.2.2)
    • This area of concern turns out not to be an issue because
      • The msi installer uses registry keys HKLM\SOFTWARE\(Wow6432Node)\Python\PythonCore\X.Y for each installation. So only stores a single set of data for each X., and apparently only allows a single X.Y version to be installed at a time.
        • Presumably the idea is that X.Y.n+1 is primarily a bug-fix relative to X.Y, and consequently should replace it, rather than both being available and selectable on the same machine.
        • (Not sure of installer algorithm when an attempt is made to install a lower-numbered version when a higher-numbered one is already installed.)
        • Additional same-X.Y Python directory trees could be put in place, and even pointed-to by PATH, and used. However, the msi-created registry keys would not reflect these other Python "installations", and they would not be found by Launcher's locate_python algorithm (though they might be found by Launcher's find_on_path algorithm).
  • There's no way for script or command line to request a version >= X.Y. To me this seems like the commonest scenario, where the script programmer knows that the script uses a python feature introduced in version X.Y. The script doesn't need version precisely X.Y, but it does need X.Y or above. (Probably limited to within the version 2 or the version 3 family.)
  • Custom virtual commands can't start with the word 'python'.
  • Custom virtual commands can't override same-named executables on PATH (including ones that could be found by virtue of PATHEXT).
  • py.exe only became available in 3.3.something. Installing a version of 3.x < 3.3 causes HKCR\.py --> HKCR\Python.File\shell\open\command and friends to point direct to \some\install\of\python, not to py.exe.
  • Launcher documentation page says: "If PY_PYTHON=3 and PY_PYTHON3=3.1, the commands python and python3 will both use specifically 3.1". This is mistaken. The effect of these two variables is separate, not cumulative.

Issues in handling of shebang line


/user/bin/env python
Launcher treats '.../env python' like Unix would, by using PATH
/user/bin/env python2
But odd that Launcher does not use PATH when processing '.../env pythonX.Y'. (Unlike under Unix).
/user/bin/env Specialpython
Algorithm transforms this to just 'Specialpython'. Look up on PATH, or failing that, custom command
But then again, '.../env arg_doesnt_start_with_python' does cause Launcher to perform a PATH search (and if that fails, looks for custom command).
/user/bin/env pythonSpecial
Fails validate_version
Thus pre-empts custom virtual shebang-line commands that begin with 'python'.
Fails validate_version
Attempts to define specific custom command for X.Y.Z would fail if they begin with 'python'
Fails validate_version


Code clarifications

  • It would be very helpful if comments were attached to all functions describing the expected input args and result values.
    • And for result values within a function to be named according to the meaning of the result.
  • A number of variables are named in ways that are either misleadingly or burdensomely ambiguous:
Function or variable
better name
Several scopes:
command, cmdline

The word "command" is ambiguous because in this code it could refer to:
  • User's command line which invoked py
  • Launcher Virtual command: One of several unix-appropriate paths to python which py.exe understands as an instruction to use its locate_python algorithm. The list of these Virtual Command paths is hard-coded into py.exe
  • Custom command: A command defined by the user in one of the py.ini files. To be invoked in a script's shebang line
  • The command parsed from a script's shebang line
  • The command being assembled by py.exe to execute python
Because of this overloading of the word "command", all instances of 'command' or 'cmd' could be made more distinct
maybe_handle_shebang: is_virt
This variable does not signify "is virtual command" -- a shebang line may contain a virtual command and this value ends up either True or False. Instead, this value signifies whether the shebang line contains a built-in virtual command.
In this code 'locate' is used in two different ways: (a) "get info from registry and put it into the list of available pythons" and (2) "get an available version of python from the list". I suggests "survey" for the functions which read the registry and build the list, and "find" or "lookup" for those which read the list.

(no name change)