Resolving intra-project imports in Python — A simple guide

When importing between Python files in a project (i.e. importing from one project file to another), exceptions such as ModuleNotFoundError or ImportError may occur if one file is outside the directory of another. The following article explains why this occurs and how to resolve these import errors, which involves configuring your editor and/or installing your project as a package.

An Example Broken Project

Consider a very basic Python project. It has two code folders, each with a Python code file in; plus a test folder with one test.

project 
│
├─── folder1
│    │    __init__.py
│    │    file1.py
│
├─── folder2
│    │    __init__.py
│    │    file2.py
│
└─── tests
     │    test_file.py

project/folder1/file1.py contains a simple add() function.

# project/folder1/file1.py
def add(num1, num2):
    return num1 + num2

project/folder2/file2.py is a Python file that we want to import our add() function into and use. We use an absolute import starting from under our project folder: from folder1 import file1.

# project/folder2/file2.py
from folder1 import file1
print('1 + 2 =', file1.add(1, 2))

Lastly, tests/test_file.py contains a simple pytest-style test method for our add() function.

# project/tests/test_file.py
from folder1 import file1
def test_add():
    assert file1.add(1, 2) == 3

This is one of the simplest Python projects there can be, so everything should work perfectly, right? Not so much; as we will see below.

Firstly, let’s try to run our file2.py script:

C:\...\project> py folder2/file2.py
Traceback (most recent call last):
  File "C:\...\project\folder2\file2.py", line 1, in <module>
    from folder1 import file1
ModuleNotFoundError: No module named 'folder1'

Oh dear! Our script tripped up on what should be a very simple import by raising the error message ModuleNotFoundError: No module named ‘folder1’, with Python complaining that it cannot find the module in question.

Let’s check how our test runner fares:

# output trimmed for brevity
C:\...\project> pytest
ImportError while importing test module 'C:\...\project\tests\test_file.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files\Python39\lib\importlib\__init__.py:127: in import_module
    from folder1 import file1
E   ModuleNotFoundError: No module named 'folder1'
1 error in 0.07s

The exact same error again. Not good. But, perhaps we can fix it, right? Let’s try to use a relative import in file2.py instead:

# project/folder2/file2.py
from ..folder1 import file1
print('1 + 2 =', file1.add(1, 2))

With this import, we use .. to go up one folder from where we currently are (i.e. going up from folder2 to project); before going back down into folder1 and into file1.py.

Let’s try to run the file:

(venv) C:\...\project> py folder2/file2.py
Traceback (most recent call last):
  File "C:\...\project\folder2\file2.py", line 1, in <module>
    from ..folder1 import file1
ImportError: attempted relative import with no known parent package

Unfortunately, now we get a completely different error. Python raises the error message ImportError: attempted relative import with no known parent package and refuses to perform the relative import.

So, why do these errors occur? And how can they be fixed?

An IMPORTant PATH

To understand why these errors occur, first we must understand how Python deals with imports.

Python resolves import statements using a list variable called sys.path. This variable contains a list of directories, and Python searches each directory in turn looking for a Python package to import.

>>> import sys
>>> for path in sys.path:
...     print(repr(path))
''
'C:\\Program Files\\Python39\\python39.zip'
'C:\\Program Files\\Python39\\DLLs'
'C:\\Program Files\\Python39\\lib'
'C:\\Program Files\\Python39'
'C:\\Program Files\\Python39\\lib\\site-packages'

As you can see, the only directories listed are those where Python is installed. So how do intra-project imports work? The key is in the very first entry; that empty '' string. This empty string represents the directory that contains the Python file that was run or where a Python REPL shell was invoked. In our above example, if we run project/folder1/file1.py , then project/folder1/ is what the empty string would represent.

The rules of how Python decides what it can import from these directories are complex to describe accurately; a condensed version is as follows: you can import from within any directory in this list (including subdirectories), but you can’t import from outside them.

If we look at the example project structure again:

project 
│
├─── folder1
│    │    __init__.py
│    │    file1.py
│
├─── folder2
│    │    __init__.py
│    │    file2.py
│
└─── tests
     │    test_file.py

We tried to run project/folder1/file1.py , which means project/folder1/ is in the Python path. However, project/folder2/file2.py , which we want to import from, is outside of this directory. We have to go up one directory to project — Python will not allow us to do this at the moment.

What we really want is for the project folder to be in sys.path. If the project folder was in sys.path, then we would be able to import from any folder or file within it.

Fear is the PATH to the Dark Side

Assys.path is a list, we can directly modify it to suit our needs if we desire. We can append to it just you can with any other list. Adding the project root is as simple as appending the absolute path of the folder to sys.path.

sys.path.append(r'C:\...\project')

If we place this line in our file2.py script,

import sys

sys.path.append(r'C:\...\project')

from folder1 import file1

print(
    '1 + 2 =',
    file1.add(1, 2),
)

then the imports will resolve correctly when we run the file.

C:\...\project> py folder2/file2.py
1 + 2 = 3

As you can see though, this is a pretty ugly method of setting the import correctly. Firstly, we have to perform the sys.path append before we try to import file1; we can’t group all our imports together like we would in any normal file. Perhaps more significantly, this operation will only work for this file as it stands. If we wanted to import file1.py into other files in our project structure, we might have place this line somewhere different or append to sys.path in multiple places. Regardless, it’s an ugly way to resolve the issue.

There Must Be a Better Way

Fixing the issue properly, without resorting to hacking the sys.path, involves two main approaches.

If you run your code through PyCharm or Visual Studio Code run configurations only, then you can set up those editors to add your project root folder to sys.path. In the case of PyCharm, this is done automatically for you; in the case of Visual Studio Code, you must do this manually for each new project.

If, however, you invoke your code manually through the terminal, whether through PyCharm or Visual Studio Code or any other terminal instance; then you can add your project root folder to sys.path by installing your project as a package.

Below are step-by-step guides on how to carry out these approaches. If you use PyCharm or Visual Studio Code, please choose those articles; if you use any other editor or IDE, please choose the “Text Editor and Terminal” article instead.

• PyCharm
• VS Code
• Text Editor and Terminal

Afterword

Issues dealing with intra-project imports are extremely common, and encountered by Python programmers of all skill levels. They can be extremely frustrating to deal with. Error information gives little to no direction on how to fix the issue, and online resources like StackOverflow often confuse by providing a multitude of answers, many of which are incorrect and/or incomplete. I hope the above helps in providing a simple guide for resolving intra-project imports.

The information in these resources have been collated from multiple sources; the most important source was pytest’s article on Good Integration Practices. The guides above provide a detailed procedure of the first section in that article.