ADR Suggestion Using __slots__ for performance improvement.

[damskii9992]

General

The python __slots__ dunder method, when defined, creates "slots" for each attributes of the class defined in its __slots__ method.
This is an alternative to the normal, implicit __dict__ method which stores all instance attributes as a dictionary on the instance.
Using __slots__ speeds up attribute access by app. 30% and reduces memory consumption of the class instance.

Caveats

While a 30% speed up and a lower memory footprint sounds very good, it comes with certain caveats:

No dynamic attributes

If __slots__ is defined, but __dict__ isn't, the class will not create the internal dictionary, thereby saving even more memory.
This comes at the cost of not being able to dynamically add attributes to a class instance, if the attribute is not defined in __slots__, it cannot be set. I.e. ALL attributes that can ever be set on the class instance, needs to be in __slots__.
This downside can be circumvented by adding __dict__ to ___slots__:

class Foo(object):
    __slots__ = 'bar', 'baz', '__dict__'

Which then enables:

>>> foo = Foo()
>>> foo.boink = 'boink'

Which would otherwise crash. Adding __dict__ to ___slots__ will however mean losing out on some of the potential memory gain by using __slots__.

No weak references

If __slots__ is defined, you cannot create a weak reference to instances of the class, UNLESS we add __weakref__ to __slots__.
Considering that we rely on weak references in our global_object to store unique_name references, this is a must.

No multiple class inheritances

If we use __slots__, then we can't have multiple inheritances in our classes.
The exception is if one of the classes don't have __slots__ defined, or it is defined to be empty, then it can be used for multiple inheritance.
But since we're here considering adding __slots__ to our base class, this would mean we can't have multiple inheritances from any class inheriting from our base classes.

More information can be found here:
https://stackoverflow.com/questions/472000/usage-of-slots

Current Implementation

The current new base classes does not have __slots__ defined and thus use __dict__ by default.

Proposed Implementation

Using __slots__ but also adding __weakref__ and __dict__ to it, to allow for dynamic attribute creation and weak references (weakrefs are non-negotiable with our current unique_name implementation).
Attributes which are properties of the classes should then be in __slot__, for faster access times, whereas transient attributes (like __serialization_id) can be dynamically added to the class instance.

[seventil]

What's the point of adding it to the baseclass if we don't know which attributes to mention to have any optimization?
(just read in the link that you don't have to rewrite parent's attributes which is a big win)
How do we know when it's appropriate to introduce __slots__ to a class? benchmarks? have it by default in any class we implement?

On the other hand this is really cool and seems easy to implement for any class in which we will see an optimization benefit.

[rozyczko]

If we have many, many base objects (parameters, descriptors), and query them in quick succession (loop? list compr), using __slots__ would be beneficial.

I think __slots__ make the codebase slightly safer, since they prevent typos for mis-named attributes. This is something that we wanted to implement in the Parameter class anyway but might be better to put these restrictions in the base class.

Additionally, they give you an "interface-like" behaviour where you see immediately what is being exposed by the class.

I might be wrong, but linters and type checkers might use __slots__ to assure code correctness as well.

The BAD side is that the transient attributes like __serialization_id are no longer transient :)
Also, patching things in tests might get a bit more difficult.

__weakref__ is another issue - we'd have to be consistent adding it in all subclasses...

[damskii9992]

Any slots added to a parent class are NOT added to children classes. So if we add __weakref__ to our base class we should be all good :)
It sounds to me like you are not in favor of adding __dict__ to the __slots__ then? You actually like the limitation of not being able to make attributes dynamically?

[damskii9992]

At our daily stand-up we decided that we would wait with implementing __slots__ until we actually need the performance gain.