gh-108828: Support selecting tests by labels by serhiy-storchaka · Pull Request #108829 · python/cpython

serhiy-storchaka · 2023-09-02T19:39:27Z

Issue: Support selecting tests by labels #108828

vstinner · 2023-09-05T20:02:44Z

If possible, I would prefer to land my PR #108858 before.

sobolevn

Thanks a lot for working on this, it is definetely a step in the right direction for our test suite.

I have a question about naming, though.
Some tests in Lib/test/test_regrtest.py would also be nice, or I can add them later :)

I already have several ideas on top of this PR!

sobolevn · 2023-10-09T18:03:38Z

   rather than looking directly in the path directories.


+.. function:: mark(label, *, globals=None)


I would prefer to always call them "marks". The main reason is that pytest uses the same idea and pytest's marks are well-known.

Suggested change

.. function:: mark(label, *, globals=None)

.. function:: mark(name, *, globals=None)

Right now mark function adds a "label". It does not sound right.

Or we can always call it "label".

Yes, it is similar to pytest's markers, but there are enough differences in applying them and filtering by them. I do not have preference, "label", "marker" and "tag" are all look like synonyms in this context to me.

sobolevn · 2023-10-09T18:03:46Z


+.. function:: mark(label, *, globals=None)
+
+   Add a label to tests.


Suggested change

Add a label to tests.

Add a mark to tests.

sobolevn · 2023-10-09T18:09:08Z

        else:
            self.ignore_tests = None
+        if ns.accept_labels:
+            self.accept_labels: tuple[str, ...] = tuple(ns.accept_labels)


Maybe set[str]?

For some reasons they should be immutable, as well as match_tests and ignore_tests.

In any case, I am planning to rewrite filtering by names. Instead of two tuples there will be more complex structure where order matters. And the same can be used for labels.

sobolevn · 2023-10-09T18:11:48Z


-def _id(obj):
-    return obj
+def mark(label, *, globals=None):


I had a similar API in one of my OS libs, users didn't like it at all :(

I think it would be better to use mark(..., *, module=False)

This way we can:

Simplify the API for readers (even if our users are people working on CPython)

Use frame hack to get the needed globals()

I do not like to depend on such hack it tests. If it does not work, many unrelated tests will not even be able to load.

Since it is an internal API used in limited number of places, simplicity for users is less important. support.mark('gui', globals=globals()) is not much worse than support.mark('gui', module=True)

It is now derived by default, but there are cases when we need to specify it explicitly.

github-actions · 2026-04-13T18:24:44Z

This PR is stale because it has been open for 30 days with no activity.

…all" This change was accidentally reverted by f0daba1 (pythongh-106693, pythonGH-149514), which only intended to revert the ob_sval change. The tests were already restored by pythonGH-149569. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

# Conflicts: # Lib/test/libregrtest/cmdline.py # Lib/test/support/__init__.py # Lib/test/support/threading_helper.py

# Conflicts: # Lib/test/support/__init__.py

mark() had two different meanings depending on whether the globals argument was passed. It is now only a decorator of a test method or class, and mark_module() marks a whole module, by default the caller's. The label and its value are positional-only: a keyword would be mistaken for the name of the label. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

read-the-docs-community · 2026-08-08T19:51:54Z

Documentation build overview

📚 cpython-previews | 🛠️ Build #33975420 | 📁 Comparing 1b3eb74 against main (998b890)

🔍 Preview build

2 files changed

± library/test.html
± whatsnew/changelog.html

serhiy-storchaka · 2026-08-08T19:56:30Z

It now supports labels with values and separate function is now used to mark the module.

Just yesterday Claude wrote a script to extract all bigmem test names from sources. With this feature it would be a one line.

support_remote_exec_only(), added in main after this branch replaced the other uses of _id() with combine(), still returns it. The merge was clean, but importing test.support failed with a NameError. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

serhiy-storchaka added type-feature A feature request or enhancement tests Tests in the Lib/test dir needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Sep 2, 2023

serhiy-storchaka requested a review from vstinner September 2, 2023 19:39

bedevere-bot mentioned this pull request Sep 2, 2023

Support selecting tests by labels #108828

Open

serhiy-storchaka marked this pull request as ready for review September 3, 2023 09:56

serhiy-storchaka requested review from brettcannon, ericsnowcurrently, ncoghlan and warsaw as code owners September 3, 2023 09:56

bedevere-bot added the awaiting core review label Sep 3, 2023

brettcannon removed their request for review September 5, 2023 22:05

serhiy-storchaka added 4 commits September 11, 2023 19:41

pythongh-108828: Support selecting tests by labels

6ea2f47

Support marking modules.

16ee29f

Add documentation.

fe7fcd0

Fix Sphinx warnings.

5ae7161

serhiy-storchaka force-pushed the libregrtest-labels branch from 2b24913 to 5ae7161 Compare September 11, 2023 16:42

sobolevn reviewed Oct 9, 2023

View reviewed changes

serhiy-storchaka added 3 commits October 29, 2023 19:03

Merge branch 'main' into libregrtest-labels

4f41c75

Make the order of options matter.

c884b55

Merge branch 'main' into libregrtest-labels

22d68c6

serhiy-storchaka added needs backport to 3.13 bugs and security fixes and removed needs backport to 3.11 only security fixes labels May 9, 2024

Yhg1s removed the needs backport to 3.12 only security fixes label Apr 8, 2025

serhiy-storchaka added the needs backport to 3.14 bugs and security fixes label May 8, 2025

serhiy-storchaka added 2 commits June 28, 2025 17:32

Merge branch 'main' into libregrtest-labels

93ecca0

Fix set_match_tests().

a2c44a5

github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 13, 2026

serhiy-storchaka added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 30, 2026

github-actions Bot removed the stale Stale PR or inactive for long period of time. label Jun 5, 2026

serhiy-storchaka and others added 6 commits June 28, 2026 22:24

Merge branch 'main' into libregrtest-labels

870d104

# Conflicts: # Lib/test/libregrtest/cmdline.py # Lib/test/support/__init__.py # Lib/test/support/threading_helper.py

Add requires_zstd label

d1d88e2

Support labels with values

a5ffcf5

Merge remote-tracking branch 'upstream/main' into libregrtest-labels

feb3f8e

# Conflicts: # Lib/test/support/__init__.py

serhiy-storchaka requested a review from FFY00 as a code owner August 8, 2026 19:47

		rather than looking directly in the path directories.


		.. function:: mark(label, *, globals=None)

	.. function:: mark(label, *, globals=None)
	.. function:: mark(name, *, globals=None)


		.. function:: mark(label, *, globals=None)

		Add a label to tests.

Uh oh!

Uh oh!

Conversation

serhiy-storchaka commented Sep 2, 2023 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Sep 5, 2023

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

read-the-docs-community Bot commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

serhiy-storchaka commented Aug 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

serhiy-storchaka commented Sep 2, 2023 •

edited by bedevere-bot

Loading

read-the-docs-community Bot commented Aug 8, 2026 •

edited

Loading