Testing#

Tests are part of the deliverable, not a follow-up. Every module ships pytest coverage of its own behaviour and tests/test_smoke.py must keep passing.

Running the suites#

Everything runs in Docker. There is no host Python environment and running pytest on the host is the fastest way to produce a bug nobody else can reproduce.

# the whole Python suite
docker compose run --rm -e RT_DATABASE_URL=sqlite:////tmp/t.db web pytest

# one file, stop at the first failure
docker compose run --rm -e RT_DATABASE_URL=sqlite:////tmp/t.db web pytest tests/test_docs.py -x

# one test, verbose
docker compose run --rm -e RT_DATABASE_URL=sqlite:////tmp/t.db web \
  pytest tests/test_cube_moves.py::test_sexy_move_order -vv

# the browser-side JS suite (node's built-in runner, no package manager)
docker compose run --rm jstest

The RT_DATABASE_URL override points the app at a throw-away database. Without it the suite would touch the real one on the rt-data volume. The app/, tests/ and scripts/ directories are bind-mounted, so no rebuild is needed after editing code — only after changing requirements.txt or the Dockerfile:

docker compose build web

Coverage#

pytest-cov is installed:

docker compose run --rm -e RT_DATABASE_URL=sqlite:////tmp/t.db web \
  pytest --cov=app --cov-report=term-missing

Coverage is a tool for finding untested branches, not a number to hit. A high percentage with no test of the empty state is worse than a lower one with it.

Fixtures#

tests/conftest.py gives you three, and they compose:

app

A real application from create_app(), in TESTING mode, pointed at a SQLite file under pytest's tmp_path. Because create_app() seeds on first boot, the real library is loaded — tests exercise the same 41 / 57 / 21 cases students see, not hand-made doubles.

client

app.test_client(). Sessions work, so client.post('/profiles/2/select') followed by another request behaves like a browser.

ctx

The app inside an application context, for tests that call services or query models directly.

def test_case_page_renders(client):
    response = client.get("/library/oll/oll-21")
    assert response.status_code == 200


def test_score_is_none_without_evidence(ctx):
    from app.services import scoring
    from app.models import Profile

    profile = Profile.query.filter_by(slug="me").one()
    scores = scoring.case_scores(profile)
    assert all(s.score is None for s in scores.values())

Each test gets a fresh database, so tests do not order-depend. That also means seeding runs per test — keep the suite reasonably shaped by not asking for the app fixture in a test that only needs a pure function.

Test layout#

The convention from the build contract is tests/test_<yourmodule>_*.py, and you own the tests for the module you own.

What a good test looks like here#

Parametrise over real data, not over a list you have to maintain. The docs suite discovers every markdown file on disk and asserts each one renders, so a new page is covered the moment it is written and there is no list to forget:

DOC_PAGES = sorted(p.relative_to(DOCS).with_suffix("").as_posix()
                   for p in DOCS.rglob("*.md"))

@pytest.mark.parametrize("slug", DOC_PAGES)
def test_every_doc_page_renders(client, slug):
    assert client.get(f"/docs/{slug}").status_code == 200

Test the property, not the implementation. The cube engine's real guarantees are "a move and its inverse cancel", "R four times is identity", "a track round-trips" — those survive a rewrite of the move tables; asserting a specific permutation tuple does not.

Test the boundary, not the middle. The scoring model's interesting values are: no evidence at all, exactly min_attempts_for_measured - 1 attempts, exactly that many, a self-rating of 5 (must cap at 80), and a time at exactly the target (must be 80).

Test the empty state. A page with no data is a state you shipped and it should have a test.

Seed the randomness. scramble(seed=42) is reproducible; scramble() in an assertion is a flaky test waiting to happen.

Cross-engine parity#

The Python and JavaScript cube engines must agree. That is enforced by tests/fixtures/cube_vectors.json, generated by the Python engine and checked by both suites — every move applied to a solved cube, every case setup, seeded scrambles.

If you change either engine, run both:

docker compose run --rm -e RT_DATABASE_URL=sqlite:////tmp/t.db web pytest
docker compose run --rm jstest

A green Python suite with a red JS suite is exactly the failure the fixtures exist to catch. Regenerating the fixtures to make a JS failure go away is almost always the wrong fix.

The docs suite specifically#

tests/test_docs.py is a little unusual and worth copying the idea from. It asserts:

• /docs/ renders, and every markdown file in the tree renders with 200 — parametrised over the real files;
• path traversal is blocked — .., encoded .., absolute paths and symlinks pointing outside the tree all return 404, not file contents;
• no internal link 404s — it crawls every rendered page, collects every href that points inside the app, and requests it. This is the check that keeps the documentation honest: a link to a page you renamed fails the build;
• /docs/search.json covers every page;
• /docs/api renders and lists the routes that exist.

The link crawl is the valuable one. Documentation rots by accumulating dead links faster than by going out of date, and a test is the only thing that notices.

Before you open a pull request#

docker compose run --rm -e RT_DATABASE_URL=sqlite:////tmp/t.db web pytest
docker compose run --rm jstest

Both green, plus the manual checks that no test can do for you: the page at 375px wide, the browser console with no errors, and keyboard-only navigation through whatever you added. See Contributing.