Python packaging - that __init__.py

Question

Say I have a directory structure like this:

mydir
    __init__.py
    innerdir
        __init__.py
        mymodule.py

And the content of mymodule.py is simply:

def hi():
    print 'hello, welcome'

while both of the __init__.pys are empty.

Now if I am outside the mydir directory, and I open a Python prompt and try to import mydir and then use mydir.innerdir, I get an error (an NameError: 'innerdir' is not defined).

Same if I move into mydir directory, prompt there and try to import innerdir and innerdir.mymodule.

I can successfully import mydir.innerdir, but then I can use it only as mydir.innerdir, not as just innerdir.

Answer 1

Some of the stuff you describe, I'm not even sure what you expected it to do. But from your last paragraph, it sounds like you're pretty close, so let's start there.

I can successfully import mydir.innerdir, but then I can use it only as mydir.innerdir, not as just innerdir. Why do I have to specify the whole path?

That's how import works in Python, even if you ignore packages and just have a bunch of flat stuff, or even just the standard library. For example, if you import os, you can do os.listdir('.'), but you can't just do listdir('.').

And the solution here is the same as it is there:

from mydir import innerdir

This isn't actually very useful, because innerdir is a package with nothing directly defined in it. What you probably really want to do is from mydir.innerdir import mymodule. But this is how to do what you asked.

---

If you want to know why import mydir followed by mydir.innerdir doesn't work, I can explain that.

From the Packages section of the tutorial:

… when using syntax like import item.subitem.subsubitem, each item except for the last must be a package; the last item can be a module or a package but can’t be a class or function or variable defined in the previous item.

So, what you're doing shouldn't do anything useful at all. But what does it actually do?

It imports the package as if it were a module. This means anything you define in the package itself—functions, classes, even modules—will be available as a member of mydir. But that doesn't automatically import any subpackages or submodules inside the package directory—remember, you're importing it as a module, not as a package.

---

If you're asking about what happens when you run from inside mydir (or add it to your PYTHONPATH)… well, in that case, mydir isn't a package at all, it's just a directory full of modules and packages (in this case, just the one package, innerdir). It's pretty much like doing from mydir import innerdir.

Answer 2

A package directly-imported doesn't automatically place subpackages or submodules into its namespace:

>>> import mydir
>>> dir(mydir)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']

(Notice that there is no innerdir here.)

But if you import through a package, Python will connect the namespaces together:

>>> import mydir.innerdir
>>> dir(mydir)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'innerdir']
>>> dir(mydir.innerdir)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']

(Notice that there is no mymodule in mydir.innerdir.)

Python allows you to say explicitly what some submodules to include automatically in a package using the __all__ magic variable in the __init__.py, but by default it will just run the code in __init__.py code and not do anything else.

You can do any of the following:

import mydir.innerdir.mymodule; mydir.innerdir.mymodule.hi()
from mydir.innerdir import mymodule; mymodule.hi()
from mydir.innerdir.mymodule import hi; hi()

Alternatively, you can edit your __init__.py files to include __all__ vars for wildcard importing.

# file mydir/innerdir/__init__.py
print 'Running mydir/innerdir/__init__.py'
__all__ = ['mymodule']

Then you can do stuff like this:

from mydir.innerdir import *; mymodule.hi()

Even crazier, you can eagerly import subpackages and modules (I don't recommend this!):

# file mydir/__init__.py
print 'Running mydir/__init__.py'
import innerdir

# file mydir/innerdir/__init__.py
print 'Running mydir/innerdir/__init__.py'
import mymodule

Now these will work:

import mydir; mydir.innerdir.mymodule.hi()
from mydir import innerdir; innerdir.mymodule.hi()

You might want to brush up on the documentation for packages, which explains all this and has a directory structure exactly like the one you show here.