Skip to content

Handling Parallelism

Caio Lima edited this page Jul 17, 2024 · 6 revisions

I make a point to reinforce that the HourGlass and Alarm classes utilize parallel processes from the multiprocessing package. If you want to deeply understand how parallelism works, I strongly recommend taking a look at the official documentation. Furthermore, there are some precautions that need to be highlighted here to deal with potential issues arising from parallelism:


Avoid Global State

Global variables may not be shared between processes, leading to inconsistent states, because each process has its own separate memory space. You should use shared memory objects from the multiprocessing module, such as Value or Array, to share data between processes:

❌ Instead of this:

from ptymer import Alarm

# Global var
days = 0

def add_count():
    global days
    days+=1

if __name__ == '__main__':
    al = Alarm(target=add_count, schedules=['10:00:00'], persist=True)

✅ Use this:

from multiprocessing import Value
from ptymer import Alarm

# Initialize shared value
days = Value('i', 0)

def add_count(days):
    with days.get_lock():
        days.value += 1

if __name__ == '__main__':
    al = Alarm(target=add_count, args=(days,), schedules=['10:00:00'], persist=True)

Additionally, using days.get_lock() ensures that only one process at a time can execute the block of code that increments days.value, preventing race conditions and ensuring the integrity of the shared value in case the function is accessed from multiple Alarm instances or processes. Here are some useful links:

Moreover, multiprocessing's Value and Array use ctypes, which means you may encounter some challenges fitting your variables. More information can be found here.


Use if __name__ == "__main__": Guard

Failing to use this guard can lead to the script being executed multiple times, causing issues such as spawning excessive processes. This is because the default process starting method in Windows is spawn, which initiates a completely new Python interpreter process and re-imports the module containing the multiprocessing code. As a result, it executes the entire module from the top. Here's how to implement it, using foo() as an example:

from ptymer import *

def foo():
    print('foo!')
# functions code here

if __name__ = '__main__':
    foo()
    # rest of your code


Errors and Return Handling

Errors or returns from child processes may not be propagated to the parent process. Therefore, it's highly recommended to implement shared value validations to handle these callbacks.

from ptymer import HourGlass
from multiprocessing import Value

def child(value, return_var):
    # Simulate some process
    result = int(100 / value)

    return_var.value = result

if __name__ == '__main__':
    # Shared value for return handling
    return_var = Value('i', 0)  # 'i' for integer, initialized with 0
    
    # Create a process
    hg = HourGlass(seconds=8, target=child, args=(5, return_var,), visibility=True).start()  # Example value 5
    while hg.is_active():
        pass

    # Handle the result
    print("Result from child process:", return_var.value)


Resource Management and Graceful Shutdown

Processes consume system resources such as memory and file handles. Be mindful of the number of processes your system can handle and ensure proper process termination. We definitely don't want this to happen:

100% cpu usage task manager print

Improper shutdown of processes can leave zombie processes or cause resource leaks. Therefore, ensure proper cleanup by handling signals and using the stop() method when necessary. But when is it necessary? Simple! When the Alarm or HourGlass is still active but no longer needed. Naturally, the HourGlass countdown reaches 0 or all alarms from Alarm are triggered (and persist=False), their processes terminate automatically. Otherwise, you can simply:

from ptymer import HourGlass

if __name__ == '__main__':
    hg1 = HourGlass(60, visibility=True).start()
    hg2 = HourGlass(60, visibility=True).start()
    hg3 = HourGlass(60, visibility=True).start()
    hg4 = HourGlass(60, visibility=True).start()

    hg1.stop()
    hg2.stop()
    hg3.stop()
    hg4.stop()

And you can always check activity:

hg.is_active()
# Returns True or False, according to its status