Improving Python class efficiency with slots

•Python
Back

If you're a Python developer, optimizing your code's memory usage and performance is crucial. One effective way to achieve this is by leveraging the __slots__ feature in your Python classes. This guide will explore the advantages of using __slots__ and provide practical insights.

What is __slots__?

In Python, every object created from a class comes with a dictionary that stores its attributes and their values. While this dynamic nature of Python is convenient, it can lead to inefficient memory usage and slower performance when dealing with numerous objects. Here's where __slots__ comes in.

A memory and performance game changer

By explicitly declaring which attributes a class can have, __slots__ allows you to allocate a fixed amount of memory for those attributes. This bypasses the need for a dictionary for each instance, reducing memory overhead and improving code execution speed. The result? A more efficient and responsive program.

Real-world application: The Point class

Let's explore a practical example by considering a Point class that represents a point in 3D space with x, y, and z coordinates. Without __slots__, the class might look like this:

class Point:
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

While this code is easy to understand, it can be memory-intensive and less efficient when dealing with many Point instances. For instance, the memory-profiler (opens in a new tab) library will show the required memory to create one million points. We create a file called points.py and write the following:

class Point:
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z
 
from memory_profiler import profile
 
@profile()
def create_points():
    return [Point(1, 2, 3) for _ in range(1_000_000)]
 
if __name__ == "__main__":
     create_points()

Profiling the creation of points with python3 -m memory_profiler points.py gives this result:

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     8     21.1 MiB     21.1 MiB           1   @profile()
     9                                         def create_points():
    10    136.0 MiB    114.9 MiB     1000003       return [Point(1, 2, 3) for _ in range(1_000_000)]

This means that creating one million points requires 93.8 Mebibytes (MiB), because that's the increment from 21.1 MiB to 114.9 MiB.

Optimizing the Point class with _slots_

Now, let's optimize the Point class with _slots_:

class SlottedPoint:
    __slots__ = ('x', 'y', 'z')
    
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

By specifying __slots__, we explicitly declare that instances of the Point class should only have attributes named 'x', 'y', and 'z'. This informs Python to allocate memory accordingly, reducing overhead and improving performance. We benchmark the optimization by creating one million points and profiling with the same method as before. Let's create a file called slotted_points.py with this content:

class SlottedPoint:
    __slots__ = ('x', 'y', 'z')
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z
 
from memory_profiler import profile
 
@profile()
def create_points():
    return [Point(1, 2, 3) for _ in range(1_000_000)]
 
if __name__ == "__main__":
     create_points()

The profiling results after running python3 -m memory_profiler slotted_points.py are:

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     9     21.6 MiB     21.6 MiB           1   @profile()
    10                                         def create_points():
    11     90.5 MiB     68.9 MiB     1000003       return [Point(1, 2, 3) for _ in range(1_000_000)]

Using __slots__ for our point class lowered the required memory to 47.3 MiB, because that's the increment from 21.6 MiB to 68.9 MiB. This simple modification reduced the memory requirements in roughly ~49% for our example. However we must note that using __slots__ limits the possible member variables in the point instances. It is not possible to assign member variables beyond those declared in the slots. In our example, it looks as follows:

point = Point(1, 2, 3)
point.a = 4 # works fine
 
slotted_point = SlottedPoint(1, 2, 3)
slotted_point.a = 4 # triggers an AttributeError

The benefits of using __slots__

Utilizing __slots__ in your Python classes provides several advantages:

  1. Improved memory usage: Explicitly declaring attributes reduces memory overhead, especially helpful with many instances.

  2. Faster attribute access: Accessing attributes becomes faster, as Python no longer needs to search through dictionaries.

  3. Enforced attribute names: __slots__ enforces attribute names, reducing the risk of accidental attribute name clashes or typos.

  4. Code Documentation: It enhances code readability and documentation, explicitly listing allowed attributes for each class.

Conclusion

In conclusion, __slots__ is a powerful tool to optimize memory usage and enhance the performance of your Python classes. By balancing readability and efficiency, you can ensure that your code runs smoothly and efficiently, regardless of the class complexity. Consider implementing __slots__ in your Python projects and experience the benefits firsthand.