Here’s a small Python pop-quiz:

  1. If I do pip install pillow, which module do I need to import to actually use the pillow package?
  2. Which Python package on PyPI provides the bs4 module?

If your answers are PIL, and “It depends”, you would be correct! The reason for this discrepancy is that there is no relationship whatsoever between the name of a Python project on PyPI, and the module(s) that one imports in Python itself (the import name) that that project provides. This also implies that one package on PyPI can offer any number of importable modules, and, conversely, that multiple PyPI packages can offer the same importable module1. These two “features” can be a source of confusion, as illustrated by the following situation:

  • user installs the package foo, which provides the foo importable module
  • user forgets about it, and installs the package foo-nightly, which also provides the foo importable module
  • user is confused why the foo package is misbehaving or is completely broken2

The above behavior is technically a longstanding bug in pip (has been present for at least 10 years), and in fact, even uv, a modern replacement for pip, does the same, though it actually has the courtesy of informing the user that the new package will clobber an existing one (at least as of version 0.8.7).

Can the Python ecosystem do better?

The PEP 794 proposal, which was recently accepted, seeks to somewhat ameliorate the situation. The TL;DR of the proposal is that developers will now be able to specify the importable names of modules that their project provides under the project.import-names entry in the pyproject.toml file.

A simple example of an entry in the pyproject.toml file is:

[project]
# other fields
import-names = ["neuron"]

NOTE: while support for PEP 794 did land into the packaging package (source), a stable release of packaging on PyPI that includes it is not yet available, and consequently, no build tool (and hence no package) actually supports it just yet.

Unfortunately, since maintaining backwards compatibility is paramount, this is a completely optional field, but, if widely adopted, could markedly improve the status quo. The problem is that the metadata is under the project table, which means that the build backend is in charge of parsing it, and therefore, all build backends would need to implement it. Since the acceptance of PEP 517, there has been a proliferation of build backends, including, but not limited to:

  • setuptools
  • poetry
  • flit
  • hatch
  • scikit-build-core
  • meson-python

The fact that (at least!) all of the above need to implement support for PEP 794 presents a significant barrier to adoption; nevertheless, since the PEP also introduces an Import-Name field in the project metadata, it would theoretically allow package installers (like pip and uv) to detect conflicts when installing packages which provide the same import names. Furthermore, while not mandated by the PEP, the build backends could also perform validation at the end of the build process to make sure wheels/sdists actually contain only those import names which they explicitly declare3. Finally, since PyPI itself performs validation of the metadata fields(source), it could also report any discrepancies between the declared and actual import names at the upload step (though due to the optional nature of the PEP, this would probably be a rather controversial change).

Overall, PEP 794 presents a step in the right direction for Python packaging, but its impact ultimately depends on how widely the ecosystem adopts it.

Footnotes

  1. this always trips me up; the “right” package to install in case of Beautiful Soup is beautifulsoup4. However, there are also bs4 (which installs just fine, it is only when you go to the actual project page do you realize it’s not the right one) and beautifulsoup (which only has an sdist available, and upon build failure actually points you towards the beautifulsoup4 package) ↩

  2. since a package I’ve worked on, the NEURON simulator, has both a neuron and a neuron-nightly project on PyPI, I can confirm that this has happened to me on multiple occasions during testing of some features ↩

  3. during the rewrite of the build system for the NEURON simulator, I encountered situations where the wheel contained spurious directories, and having this kind of validation step at my disposal would’ve greatly helped since I wouldn’t have needed to dig into the wheel itself to see if everything was packaged correctly ↩