Python share values

Question

In my project I have multiple flags like this:

file_a = False
file_b = False
file_c = False

I'm trying to run two processes: one (call it A from now) handles incoming messages on message queue, second (call it B from now) handles some data processing. B operates on boolean flags, and A sets its values:

def a():
    while True:
        ...
        ...
        file_a = True
        ...

def b():
    while True:
        ...
        if file_a:
            process(file_a)
            ...

a_proc = Process(target=a)
b_proc = Process(target=b)
a_proc.start()
b.proc.start()

But, the values don't seem to change. I read that I should use threading, and it seems to work, but my guideline is to use multiprocessing instead of threads.

Answer 1

If you need to exchange data between processes with multiprocessing module, you can directly share memory:

multiprocessing.Value

Value is a wrapper around a ctypes object, which has an underlying value attribute representing the actual object in memory. All Value does is ensure that only a single process or thread may read or write this value attribute simultaneously.

from multiprocessing import Value
file_a = Value('i', 0)
file_b = Value('i', 0)
file_c = Value('i', 1)

This will create shared integer values for your file flags. Since it's python, the conversion between integer values and boolean values is simple:

>>> Value('i', True)
<Synchronized wrapper for c_int(1)>
>>> Value('i', False)
<Synchronized wrapper for c_int(0)>
>>> bool(Value('i', False).value)
False
>>> bool(Value('i', 50).value)
True

Preference of a taste, but maybe a better option, you can use c_bool from ctypes:

from multiprocessing import Value
from ctypes import c_bool
file_a = Value(c_bool, False)
file_n = Value(c_bool, False)
file_c = Value(c_bool, True)

>>> Value(c_bool, False)
<Synchronized wrapper for c_bool(False)>
>>> Value(c_bool, 5)
<Synchronized wrapper for c_bool(True)>

multiprocessing.Manager dictionary:

To collect multiple boolean flags, you could use dictionary, but it needs to be shared between processes, so Manager() comes in handy.

from multiprocessing import Manager
manager = Manager()
flags = manager.dict({'file_a' : False, 'file_b' : False, 'file_c' : True})

>>> flags
<DictProxy object, typeid 'dict' at 0x7f70822f06d0>
>>> flags['file_a']
False
>>> dict(flags)
{'file_a': False, 'file_c': True, 'file_b': False}

And finally collecting it all together:

I will go with Manager approach, simply because it will make the code cleaner:

from multiprocessing import Process, Manager
manager = Manager()

    def a():
        while True:
            ...
            ...
            flags['file_a'] = True
            ...

    def b():
        while True:
            ...
            if flags['file_a']:
                process(file_a)
                ...

    if __name__ == '__main__':
        flags = manager.dict({'file_a' : False, 'file_b' : False, 'file_c' : True})
        a_proc = Process(target=a)
        b_proc = Process(target=b)
        a_proc.start()
        b.proc.start()

Answer 2

Your file_a, file_b, file_c are being loaded into each process separately. You need to use Value from multiprocessing