I was surprised with the
with open("somefile"): syntax in Python. This is useful for automatically freeing resources, such as opening files:
with open("some_file") as f: print(f.read()) # f is automatically closed at the end of the with statement and cannot be used: print(f.tell())
If we run this code, you’ll have an error because the file is closed and that’s what is expected:
Traceback (most recent call last): File "<stdin>", line xy, in <module> ValueError: I/O operation on closed file.
There are some more examples in the
contextlib package, for instance in order to acquire a database connection for a query.
It turns out this pattern is called a context manager and you can write your own. All you have to do is implement
Here is an example where we temporarilly increase the allowed memory limit and recursion depth, then we restore it to its initial value when it’s over.
import resource, sys # Custom context manager for max recursion depth and memory usage # https://docs.python.org/3/reference/datamodel.html#context-managers class ResourceLimit: def __init__(self, recursion_limit, stack_limit): self.recursion_limit = recursion_limit self.stack_limit = stack_limit def __enter__(self): self.old_recursion_limit = sys.getrecursionlimit() self.old_memory_limit = resource.getrlimit(resource.RLIMIT_STACK) # https://docs.python.org/3/library/sys.html#sys.setrecursionlimit # https://docs.python.org/3/library/resource.html#resource.setrlimit sys.setrecursionlimit(self.recursion_limit) resource.setrlimit(resource.RLIMIT_STACK, (self.stack_limit, -1)) def __exit__(self, type, value, tb): sys.setrecursionlimit(self.old_recursion_limit) resource.setrlimit(resource.RLIMIT_STACK, self.old_memory_limit) print("before") print(sys.getrecursionlimit()) print(resource.getrlimit(resource.RLIMIT_STACK)) # We may reach 1_000_000 levels of recursion # and we have up to 512 MB of memory with ResourceLimit(1_000_000, 512 * 1024 * 1024): print("during") print(sys.getrecursionlimit()) print(resource.getrlimit(resource.RLIMIT_STACK)) complex_function_that_needs_a_lot_of_ram_and_uses_recursion_a_lot() print("after") print(sys.getrecursionlimit()) print(resource.getrlimit(resource.RLIMIT_STACK))
The resources are increased then restored, as you can see in the output:
before 1000 (8388608, -1) during 1000000 (536870912, -1) after 1000 (8388608, -1)
This is a contrieved example (everything goes back to normal at the end of the program anyway), but this can be useful for chaining some problems that may require extra resources.