pytest
Full-featured Python testing framework — test discovery, fixtures, parametrize, plugins, and detailed failure reporting. pytest features: automatic test discovery (test_*.py files, test_* functions), assert introspection for detailed failure messages, @pytest.fixture for reusable test setup/teardown, @pytest.mark.parametrize for data-driven tests, conftest.py for shared fixtures, @pytest.mark for test categorization, pytest.raises() for exception testing, tmp_path/capsys/monkeypatch built-in fixtures, pytest-asyncio for async tests, pytest-cov for coverage, pytest-mock for unittest.mock integration, xfail/skip markers, and 1000+ plugins.
Score Breakdown
⚙ Agent Friendliness
🔒 Security
Testing framework with no network calls in core. Test code security: do not hardcode real credentials in tests — use monkeypatch.setenv() or pytest-dotenv; tmp_path cleaned after test — safe for temp secrets; avoid real external API calls in tests — use mock/VCR.
⚡ Reliability
Best When
All Python unit, integration, and functional testing — pytest is the de facto standard Python testing framework with the best fixture system and plugin ecosystem.
Avoid When
Load testing (use locust), browser E2E testing (use playwright with pytest-playwright), or when unittest is mandated by existing codebase.
Use Cases
- • Agent basic test — def test_addition(): assert 1 + 1 == 2; def test_string(): result = 'hello'.upper(); assert result == 'HELLO' — no class needed; agent writes plain functions starting with test_; pytest discovers and runs automatically; assert messages show actual vs expected values on failure
- • Agent fixtures — import pytest; @pytest.fixture; def db_connection(): conn = create_connection(); yield conn; conn.close(); def test_query(db_connection): result = db_connection.query('SELECT 1'); assert result == [(1,)] — fixture; agent defines reusable setup/teardown; yield before cleanup; fixture injected by parameter name
- • Agent parametrize — @pytest.mark.parametrize('input,expected', [('hello', 'HELLO'), ('world', 'WORLD'), ('', '')]); def test_upper(input, expected): assert input.upper() == expected — data-driven; agent tests multiple cases in one function; 3 separate test cases generated; each shown individually in output
- • Agent exception testing — import pytest; def test_divide_by_zero(): with pytest.raises(ZeroDivisionError): 1 / 0; def test_value_error(): with pytest.raises(ValueError, match='invalid'): int('abc') — exception test; agent verifies exceptions are raised; match= checks exception message with regex
- • Agent async test — import pytest; @pytest.mark.asyncio; async def test_async_fetch(httpx_mock): httpx_mock.add_response(json={'status': 'ok'}); result = await fetch_data(); assert result == {'status': 'ok'} — async; agent tests async functions with pytest-asyncio; @pytest.mark.asyncio required; pytest.ini: asyncio_mode = auto to apply globally
Not For
- • unittest-only environments — pytest runs unittest.TestCase but for pure unittest style use unittest directly
- • Non-Python testing — pytest is Python-only; for polyglot test infrastructure use separate tools
- • Load/performance testing — pytest is for unit/integration tests; for load testing use locust or k6
Interface
Authentication
No auth — testing framework.
Pricing
pytest is MIT licensed. Free for all use.
Agent Metadata
Known Gotchas
- ⚠ Fixture scope affects when setup runs — @pytest.fixture(scope='function') (default): runs per test; scope='module': once per module; scope='session': once per test run; agent code: expensive fixtures (DB setup, server start) should use scope='module' or 'session'; test isolation requires function scope
- ⚠ conftest.py fixtures are auto-discovered — fixtures defined in conftest.py are available to all tests in same directory and subdirectories; no import needed; agent code: put shared fixtures in conftest.py at project root; per-directory fixtures in local conftest.py; fixture name must match parameter name
- ⚠ pytest.mark.parametrize IDs are auto-generated — @pytest.mark.parametrize('x', [1, 2, 3]) generates IDs: test[1-0], test[2-1], test[3-2]; agent code: use explicit IDs for readable output: parametrize('x', [pytest.param(1, id='one'), pytest.param(2, id='two')]); or parametrize('x,id', [(1, 'one'), (2, 'two')])
- ⚠ async tests need asyncio_mode config — pytest-asyncio: without config, @pytest.mark.asyncio required on each test; with asyncio_mode = 'auto' in pytest.ini: all async test functions auto-detected; agent code for async projects: add to pytest.ini: [pytest] asyncio_mode = auto; eliminates per-test marker
- ⚠ monkeypatch is function-scoped — monkeypatch fixture undoes all patches after each test function; agent code: cannot use monkeypatch in session-scoped fixtures; for module-level patching: use unittest.mock.patch as context manager in module-scoped fixture; monkeypatch.setenv() for env vars
- ⚠ tmpdir vs tmp_path — legacy: tmpdir returns py.path.local object; modern: tmp_path returns pathlib.Path; agent code: use tmp_path (pathlib-based); tmp_path is unique per test and cleaned up after; tmp_path_factory for session-scoped temp directories: @pytest.fixture(scope='session'); def session_dir(tmp_path_factory): return tmp_path_factory.mktemp('data')
Full Evaluation Report
Detailed scoring breakdown, competitive positioning, security analysis, and improvement recommendations for pytest.
Scores are editorial opinions as of 2026-03-06.