Garbage Collection and greenlets¶
If all the references to a greenlet object go away (including the
references from the parent attribute of other greenlets), then there
is no way to ever switch back to this greenlet. In this case, a
GreenletExit exception is generated into the greenlet. This is
the only case where a greenlet receives the execution asynchronously
(without an explicit call to
greenlet.switch()). This gives
try/finally blocks a chance to clean up resources held by the
greenlet. This feature also enables a programming style in which
greenlets are infinite loops waiting for data and processing it. Such
loops are automatically interrupted when the last reference to the
greenlet goes away.
>>> from greenlet import getcurrent, greenlet, GreenletExit >>> def run(): ... print("Beginning greenlet") ... try: ... while 1: ... print("Switching to parent") ... getcurrent().parent.switch() ... except GreenletExit: ... print("Got GreenletExit; quitting") >>> glet = greenlet(run) >>> _ = glet.switch() Beginning greenlet Switching to parent >>> glet = None Got GreenletExit; quitting
The greenlet is expected to either die or be resurrected by having a
new reference to it stored somewhere; just catching and ignoring the
GreenletExit is likely to lead to an infinite loop.
Cycles In Frames¶
Greenlets participate in garbage collection in a limited fashion; cycles involving data that is present in a greenlet’s frames may not be detected.
In particular, storing references to other greenlets cyclically may lead to leaks.
We use an object with
__del__ methods to demonstrate when they
are collected. These examples require Python 3 to run; Python 2
won’t collect cycles if the
__del__ method is defined.
Manually Clearing Cycles Works¶
Here, we define a function that creates a cycle; when we run it and then collect garbage, the cycle is found and cleared, even while the function is running.
The examples that find and collect the cycle do so because we’re manually removing the top-level references to the cycle by deleting the variables in the frame.
>>> import gc >>> class Cycle(object): ... def __del__(self): ... print("(Running finalizer)") >>> def collect_it(): ... print("Collecting garbage") ... gc.collect() >>> def run(collect=collect_it): ... cycle1 = Cycle() ... cycle2 = Cycle() ... cycle1.cycle = cycle2 ... cycle2.cycle = cycle1 ... print("Deleting cycle vars") ... del cycle1 ... del cycle2 ... collect() ... print("Returning") >>> run() Deleting cycle vars Collecting garbage (Running finalizer) (Running finalizer) Returning
If we use the same function in a greenlet, the cycle is also found while the greenlet is active:
>>> glet = greenlet(run) >>> _ = glet.switch() Deleting cycle vars Collecting garbage (Running finalizer) (Running finalizer) Returning
If we tweak the function to return control to a different greenlet (the main greenlet) and then run garbage collection, the cycle is also found:
>>> glet = greenlet(run) >>> _ = glet.switch(getcurrent().switch) Deleting cycle vars >>> collect_it() Collecting garbage (Running finalizer) (Running finalizer) >>> del glet
Cycles In Suspended Frames Are Not Collected¶
Where this can fall apart is if a greenlet is left suspended and not
switched to. Cycles within the suspended frames will not be detected;
note how we don’t run finalizers here when the
outer greenlet runs
>>> def inner(): ... cycle1 = Cycle() ... cycle2 = Cycle() ... cycle1.cycle = cycle2 ... cycle2.cycle = cycle1 ... getcurrent().parent.switch() >>> def outer(): ... glet = greenlet(inner) ... glet.switch() ... collect_it() >>> outer_glet = greenlet(outer) >>> outer_glet.switch() Collecting garbage
It’s only when the
inner greenlet becomes garbage itself that its
frames and cycles can be freed:
>>> outer_glet.dead True >>> collect_it() Collecting garbage (Running finalizer) (Running finalizer)
A Cycle Of Greenlets Is A Leak¶
What if we introduce a cycle among the greenlets themselves while also
leaving a greenlet suspended? Here, the frames of the
greenlet refer to the
outer (as the
inner greenlet itself
does), and both the frames of the
outer, as well as the
greenlet itself, refer to the
>>> def inner(): ... cycle1 = Cycle() ... cycle2 = Cycle() ... cycle1.cycle = cycle2 ... cycle2.cycle = cycle1 ... parent = getcurrent().parent ... parent.switch() >>> def outer(): ... glet = greenlet(inner) ... getcurrent().child_greenlet = glet ... glet.switch() ... collect_it()
This time, even letting the outer and inner greenlets die doesn’t find the cycle hidden in the inner greenlet’s frame:
>>> outer_glet = greenlet(outer) >>> outer_glet.switch() Collecting garbage >>> outer_glet.dead True >>> collect_it() Collecting garbage
Even explicitly deleting the outer greenlet doesn’t find and clear the cycle; we have created a legitimate memory leak, not just of the greenlet objects, but also the objects in any suspended frames:
>>> del outer_glet >>> collect_it() Collecting garbage