Why might Python's `from` form of an import statement bind a module name?

Question

I have a Python project with the following structure:

testapp/
├── __init__.py
├── api
│?? ├── __init__.py
│?? └── utils.py
└── utils.py

All of the modules are empty except testapp/api/__init__.py which has the following code:

from testapp import utils

print "a", utils

from testapp.api.utils import x

print "b", utils

and testapp/api/utils.py which defines x:

x = 1

Now from the root I import testapp.api:

$ export PYTHONPATH=$PYTHONPATH:.
$ python -c "import testapp.api"
a <module 'testapp.utils' from 'testapp/utils.pyc'>
b <module 'testapp.api.utils' from 'testapp/api/utils.pyc'>

The result of the import surprises me, because it shows that the second import statement has overwritten utils. Yet the docs state that the from statement will not bind a module name:

The from form does not bind the module name: it goes through the list of identifiers, looks each one of them up in the module found in step (1), and binds the name in the local namespace to the object thus found.

And indeed, when in a terminal I use a from ... import ... statement, no module names are introduced:

>>> from os.path import abspath
>>> path
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'path' is not defined

I suspect this has to do with Python, at the time of the second import statement, trying to import testapp.api.utils which refers to testapp.utils and failing but I'm not certain.

What is happening here?

Answer 1

From the import system documentation:

When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module’s namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule. Let’s say you have the following directory structure:

spam/
    __init__.py
    foo.py
    bar.py

and spam/__init__.py has the following lines in it:

from .foo import Foo
from .bar import Bar

then executing the following puts a name binding to foo and bar in the spam module:

>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>

Given Python’s familiar name binding rules this might seem surprising, but it’s actually a fundamental feature of the import system. The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former.

If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.