way to control how pytest-xdist runs tests in parallel?

[pytestbot]

Originally reported by: Anonymous

---

Apologies if this has been raised and triaged already. I could not find it in the archives.

The enhancement I request in py.test is the ability to control how tests are executed in parallel. It would be really nice to have a way to run multiple classes/modules in parallel, while having serial execution within a given set of classes/modules.

TestNG (Java) provides similar functionality. Please see:
http://testng.org/doc/documentation-main.html#parallel-running

This was also discussed a while ago on StackOverflow. Please see discussion:
http://stackoverflow.com/questions/4637036/is-there-a-way-to-control-how-pytest-xdist-runs-tests-in-parallel

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/175

[pytestbot]

Original comment by Ronny Pfannschmidt (BitBucket: RonnyPfannschmidt, GitHub: RonnyPfannschmidt):

---

i started work to prepare xdist for such a refactoring

(the particulars are removing features that are not related to parallel testruns from xdist)

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

To the original poster: you are talking about load-scheduling, to speed up tests, right?

[pytestbot]

Original comment by Denis K. (BitBucket: krya, GitHub: krya):

---

would be very usefull in django plugin aswell.
for instance - I need a way to create database only in one process before rest is started and a way to run some tests in a single thread ( transaction tests ) so they wont fight each other trying to flush database

[pytestbot]

Original comment by Robin Dunn (BitBucket: robind42, GitHub: robind42):

---

Another nice-to-have would be options to set the tests-per-process granularity. In other words, to control when an existing testing process is recycled and when it will be replaced by a new process. I have some tests that should have a new process for each test, or at least a new process per module, just to ensure that the tests are running in a pristine process environment.

[pytestbot]

Original comment by Ronny Pfannschmidt (BitBucket: RonnyPfannschmidt, GitHub: RonnyPfannschmidt):

---

thats also part of the plans (the idea is to have --boxed use dist facilities as well, --boxed-mode=module/class is thinkable)

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

@RobinD42 @denis_kasakov if you could try to write documentation on the API and/or command line options, you'd like to see, that would be much appreciated. Once we have a clear documented concept/user stories, the implementation might not be that hard.

[pytestbot]

Original comment by Dmytro Makhno (BitBucket: dmytro_makhno, GitHub: dmytro_makhno):

---

https://gist.github.com/dmakhno/9970080#file-test_parallel-py

Here is comparision with nose.

At first it is nice to have similar to nose abilities.

@pytest.distribute("split") # "shared" or "per_class", see examples below
class MyTest(unittest.TestCase)

Next step may be apply decorators to fixture taking into account their ability to be distributed.

@pytest.fixture(scope="class", distribute="shared")
def db_class(request): pass

@pytest.fixture(scope="class", distribute="per_class")
def webdriver_class(request): pass

@pytest.fixture(scope="class") # default distribute="split", how it works now
def mockapi_class(request): pass

What do you think?

[pytestbot]

Original comment by Prasanna Santhanam (BitBucket: psanthanam, GitHub: psanthanam):

---

I've moved away from nose because of its lack of support for this. Was hoping this was not a problem with py.test.

[pytestbot]

Original comment by Dmytro Makhno (BitBucket: dmytro_makhno, GitHub: dmytro_makhno):

---

@psanthanam,
I'm new to both, but py.test seems more convenient for me due to other xUnit experience. Comparing them, nose seems has more options here to deal with this.

2 all,

I do need run setupClass once per class, in my integration tests it is very heavy and expensive operation, but the rest is designed to be well-paralled.
If just anyone can suggest how to achieve this with pytest... I'll be uncountably happy.

Maybe there are some cross-mulitprocess-singleton-fixtures? :)

[pytestbot]

Original comment by Ivan Kalinin (BitBucket: PuPSSMaN, GitHub: PuPSSMaN):

---

So, it there any progress / consideration for this?

At least, where is the scheduling / distribution logic located in the xdist code so one can have a look at that?

[pytestbot]

Original comment by Ivan Kalinin (BitBucket: PuPSSMaN, GitHub: PuPSSMaN):

---

Looks like https://bitbucket.org/hpk42/pytest-xdist/src/5ba5bdb77d302b621508bde6fdca37349f7d17d7/xdist/dsession.py?at=default#cl-153 controls actual distribution. One should be certainly capable of subclassing LoadScheduling to implement per-module distribution, IMO

[pytestbot]

Original comment by Bruno Oliveira (BitBucket: nicoddemus, GitHub: nicoddemus):

---

I've created a PR that's related to this issue, allowing to use pytest.mark to mark tests that should be executed serially. Perhaps this covers OP's needs?

[pytestbot]

Original comment by Dmytro Makhno (BitBucket: dmytro_makhno, GitHub: dmytro_makhno):

---

This is preliminary result that might be:
https://gist.github.com/polusok/4e71f7e3d3dbf437cc25
But still very "hacky" and noisy.

When shared fixture is singleton.
But maybe another way, having singleton fixture, run depending tests on the same slave.

@pupssman, thanks I didn't get chance to dig. What is general intention for *Scheduling classes, how they distribute tests among nodes, key activities? (Please treat me as python newbie)

[pytestbot]

Original comment by Pierre-Yves Rofes (BitBucket: piwai, GitHub: piwai):

---

Hey all,

I think my use case is roughly the same as the original poster, as I have test classes with setUp() methods that are time-consuming, so to avoid running them on each node i'd like to ensure that every test of the same test class run on the same node.
Followin Ivan lead to subclass LoadScheduling, I started something here to implement per-testclass distribution : https://bitbucket.org/piwai/pytest-xdist/commits/all

It works, but it's still a Proof of Concept and it needs more work to improve/document/cleanup... But, before investing more time I'd like to have a feedback from the maintainers to know if this has a reasonable chance of getting merged when finished.

Thanks,

[pytestbot]

Original comment by Ivan Kalinin (BitBucket: PuPSSMaN, GitHub: PuPSSMaN):

---

That's brilliant! Looks like it does the very thing it needs to do, and
does so simply.

BTW, it looks like adding a Scheduling implementation requires a bit of
inner manipulation -- maybe we could, you know, device a plug-in mechanism
for that?