Skip to content

Share with workers a private copy of data structures set by main thread, stored on the namespace (ns) object #252

@eldipa

Description

@eldipa

Describe the feature you'd like
The namespace or ns object is used to communicate or share data structures between the main thread and the workers and between the workers.
These data structures need to be created by the sharer in order to be, well, really shared and if they are mutable, they must also be protected with a kind of a lock (RLock), also created by sharer.

This allows a communication back and forward between the main thread and the workers and between the workers.

But in some cases it is needed something simpler: a main thread share information with its worker in one way fashion and the workers don't need to communicate between themselves. This scenario makes a fully N-to-N shared data structure and the requirement of a lock totally unnecessary.

In this case byexample could allow to create non-shared data-structures like plain list, dict or int in the main thread and make them visible in the workers as private copies.

To ensure that this will work in threading and in multiprocessing environments, the private objects must be pickle-able and the copy process should be (take main's object)->pickle->(get the bytes)->unpickle->(get the copy object)

Additional context (optional)
The documentation of byexample does not explain what happen if ns is used to store objects not created by sharer. In principle we could forbid them or we could make it private. In any case it may break some 3rd party plugin but the impact should be quite low.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementsomething nice to have but it is not neither critical nor urgent

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions