TDD example: Test an existing Flask app
It's difficult to make sure that the change you're making to your API doesn't break existing behaviors.
One way to do that would be to use TDD when developing the new functionality.
To use TDD on an existing API, you'll first need to load the app into the test suite.
Assuming your app is in a file called app.py, this is how to do it:
from app import app
def test_import_app():
pass
Run the test:
$ pytest test_app.py
The test fails. Error is:
ImportError while importing test module '/home/patryk/Workshop/things/ghost-pwa/server/test_app.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.11/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test_app.py:3: in <module>
from app import app
E ModuleNotFoundError: No module named 'app'
This is because I don't yet have an app like that in that file. Creating the file app.py!
from flask import Flask
app = Flask(__name__)
Run the test:
$ pytest test_app.py
__________________ ERROR collecting test_app.py _________________
ImportError while importing test module '/home/patryk/Workshop/things/ghost-pwa/server/test_app/test_app.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.11/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test_app.py:3: in <module>
from app import app
app.py:1: in <module>
from flask import Flask
E ModuleNotFoundError: No module named 'flask'
==================== short test summary info ====================
ERROR test_app.py
!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!
==================== 1 error in 0.11s ====================
Error again!
What is the error this time? It's ModuleNotFoundError (the line above the "test summary").
Let's install the missing module.
To avoid adding packages on the system level, we'll use Poetry.
Poetry is a dependency management system designed for Python package creators.
It uses a configuration file named pyproject.toml to track dependencies of your package, and offers a quick way to install all of them into an automatically created virtual environment.
This way, all system packages remain unaffected, so you don't risk breaking all your other flask apps by installing the newest version of Flask for use with this tutorial:)
To use Poetry with the app, we first initialize the project:
$ poetry init --no-interaction
We use the flag --no-interaction this time to be able to quickly proceed to specifying dependencies. Without that flag, we would need to provide details about our project that are not relevant for now.
The result of poetry init --no-interaction is a new file called pyproject.toml.
$ cat
cat pyproject.toml
[tool.poetry]
name = "test-app"
version = "0.1.0"
description = ""
authors = ["Patryk Kocielnik <kocielnik.dev@gmail.com>"]
readme = "README.md"
packages = [{include = "test_app"}]
[tool.poetry.dependencies]
python = "^3.11"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
We can now specify that our project requires the Flask package:
$ poetry add flask
Using version ^2.3.2 for flask
Updating dependencies
Resolving dependencies... (0.4s)
Writing lock file
Package operations: 7 installs, 0 updates, 0 removals
• Installing markupsafe (2.1.3)
• Installing blinker (1.6.2)
• Installing click (8.1.6)
• Installing itsdangerous (2.1.2)
• Installing jinja2 (3.1.2)
• Installing werkzeug (2.3.6)
• Installing flask (2.3.2)
Let's add PyTest too, even though it was already in the system, for the sake of completeness. We'll install it into the dev group of packages.
$ poetry add --group dev pytest
poetry add --group dev pytest
Using version ^7.4.0 for pytest
Updating dependencies
Resolving dependencies... (1.1s)
Writing lock file
Package operations: 4 installs, 0 updates, 0 removals
• Installing iniconfig (2.0.0)
• Installing packaging (23.1)
• Installing pluggy (1.2.0)
• Installing pytest (7.4.0)
The "dev" group contains packages that can be omitted when installing the package for users.
These packages are still required by us, the developers, so we want to keep their list around.
Let's run the test again:
$ pytest test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/patryk/Workshop/things/test_app
plugins: dvc-3.2.1, hydra-core-1.3.2, anyio-3.5.0
collected 0 items / 1 error
==================================== ERRORS ====================================
_________________________ ERROR collecting test_app.py _________________________
ImportError while importing test module '/home/patryk/Workshop/things/test_app/test_app.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.11/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
/home/patryk/Workshop/things/ghost-pwa/server/test_app/test_app.py:3: in <module>
???
app.py:1: in <module>
from flask import Flask
E ModuleNotFoundError: No module named 'flask'
=========================== short test summary info ============================
ERROR test_app.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.11s ===============================
Right, same error. Before we can access Flask in our app, we need to enter Poetry shell.
$ poetry shell
poetry shell
Spawning shell within /home/patryk/.cache/pypoetry/virtualenvs/test-app-FjGf7Qen-py3.11
$ emulate bash -c '. /home/patryk/.cache/pypoetry/virtual
envs/test-app-FjGf7Qen-py3.11/bin/activate'
(test-app-py3.11) $
The last line indicates we are now in a sub-shell with Python 3.11 as the default and a configuration named test-app-py3.11.
Let's try running our tests again.
$ pytest test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/test_app
collected 1 item
test_app.py . [100%]
============================== 1 passed in 0.19s ===============================
Passed this time!
To make sure our API works, we need to query it.
We'll do it by using a test interface within the app. Using that interface resembles using the Python Requests module that may already be familiar to you.
Let's create a test client to the API:
#!/usr/bin/env pytest
from app import app
def test_import_app():
test_client = app.test_client()
Note I've added a shebang (#!) line at the top.
Let's make sure this file can be called as a script to simplify our testing:
$ chmod +x test_app.py
Run the test now... it passes!
$ ./test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/test_app
collected 1 item
test_app.py . [100%]
============================== 1 passed in 0.19s ===============================
No surprises here!
Let's see what our API serves under the default route.
def test_import_app():
test_client = app.test_client()
res = test_client.get('/')
Test it (pytest test_app.py):
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/test_app
collected 1 item
test_app.py . [100%]
============================== 1 passed in 0.19s ===============================
Still passes! I expected it to fail at this point, but... not yet!
What do you expect the API you're working with to return from the root endpoint (/)?
I expect my endpoint to return... The value 42, as a JSON. Let's use that and afterwards, you'll substitute this value with the one you need to see.
def test_import_app():
test_client = app.test_client()
res = test_client.get('/')
+ data = res.json()
Re-run the test:
./test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/etudes/python/test_app
collected 1 item
test_app.py F [100%]
=================================== FAILURES ===================================
_______________________________ test_import_app ________________________________
def test_import_app():
test_client = app.test_client()
res = test_client.get('/')
> data = res.json()
E TypeError: 'NoneType' object is not callable
test_app.py:9: TypeError
=========================== short test summary info ============================
FAILED test_app.py::test_import_app - TypeError: 'NoneType' object is not callable
============================== 1 failed in 0.23s ===============================
Whoops! Got an error already at the stage of obtaining the JSON from the response object.
Let's add the necessary function to the app:
--- a/python/test_app/app.py
+++ b/python/test_app/app.py
@@ -1,3 +1,7 @@
from flask import Flask
app = Flask(__name__)
+
+@app.route('/')
+def get_root():
+ return 42
Test again:
$ ./test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/etudes/python/test_app
collected 1 item
test_app.py F [100%]
=================================== FAILURES ===================================
_______________________________ test_import_app ________________________________
def test_import_app():
test_client = app.test_client()
res = test_client.get('/')
> data = res.json()
E TypeError: 'NoneType' object is not callable
test_app.py:9: TypeError
------------------------------ Captured log call -------------------------------
ERROR app:app.py:1414 Exception on / [GET]
Traceback (most recent call last):
File "/home/patryk/.cache/pypoetry/virtualenvs/test-app-FjGf7Qen-py3.11/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/patryk/.cache/pypoetry/virtualenvs/test-app-FjGf7Qen-py3.11/lib/python3.11/site-packages/flask/app.py", line 1487, in full_dispatch_request
return self.finalize_request(rv)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/patryk/.cache/pypoetry/virtualenvs/test-app-FjGf7Qen-py3.11/lib/python3.11/site-packages/flask/app.py", line 1506, in finalize_request
response = self.make_response(rv)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/patryk/.cache/pypoetry/virtualenvs/test-app-FjGf7Qen-py3.11/lib/python3.11/site-packages/flask/app.py", line 1837, in make_response
raise TypeError(
TypeError: The view function did not return a valid response. The return type must be a string, dict, list, tuple with headers or status, Response instance, or WSGI callable, but it was a int.
=========================== short test summary info ============================
FAILED test_app.py::test_import_app - TypeError: 'NoneType' object is not callable
============================== 1 failed in 0.23s ===============================
So we can't just return "42". We need to make it a str or dict, for example.
@app.route('/')
def get_root():
- return 42
+ return {"value": 42}
Test it:
$ ./test_app.py
./test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/etudes/python/test_app
collected 1 item
test_app.py F [100%]
=================================== FAILURES ===================================
_______________________________ test_import_app ________________________________
def test_import_app():
test_client = app.test_client()
res = test_client.get('/')
> data = res.json()
E TypeError: 'dict' object is not callable
test_app.py:9: TypeError
=========================== short test summary info ============================
FAILED test_app.py::test_import_app - TypeError: 'dict' object is not callable
============================== 1 failed in 0.23s ===============================
Right, the .json part should go without brackets. It's how the interface is defined here.
In Requests, this logic is written as .json() -- that's a difference.
Fixing that:
def test_import_app():
test_client = app.test_client()
res = test_client.get('/')
- data = res.json()
+ data = res.json
$ ./test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/etudes/python/test_app
collected 1 item
test_app.py . [100%]
============================== 1 passed in 0.19s ===============================
It passed, finally!
Let's check for the exact data we expect:
res = test_client.get('/')
data = res.json
+ assert data == {"value": 42}
Test it... Passes too!
You can check for HTTP error codes as well.
In this case, the code should be 200. Let's check it!
data = res.json
assert data == {"value": 42}
+ assert res.status_code == 200
$ ./test_app.py
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/patryk/Workshop/things/etudes/python/test_app
collected 1 item
test_app.py . [100%]
============================== 1 passed in 0.20s ===============================
Yes! Still passes!
Now you can adjust the expected result to what you need to see in your API.
Afterwards, you will be able to make the necessary updates and still check if the API functions properly.
All that without having to start a separate server process and using complicated hacks to detect when exactly that server will be ready to handle your requests.
A final remark!
I once understood TDD as a way to develop an entire application without talking to your users.
However, the Agile movement, of which TDD is the core part, emphasizes talking to your users on a regular basis to see if the new functionality is heading in the direction that will be the most useful to them.
Once you've added tens of new functions, your users won't be able to test them as well as if you had implemented a few and sought feedback on that few.
It will also be more difficult for you to hear at that stage: "I don't need this." about your shiny new features.
For this reason, it's better to seek feedback early and frequently.
The fact that you already have (or will already have) tests for your key logic makes you much better prepared for changing it to your users' needs and taste.
This post first appeared at https://sbctricks.com/tdd-example-testing-a-flask-app.