i have code given 2 parameters, (k, m) return 4d numpy array, requirement need calculate array possible values of (k,m) k,m < n , add them up. slow in serial trying learn multiprocessing module in python it. https://docs.python.org/2/library/multiprocessing.html
essentially want use 8 cores parallely compute these 4d arrays , add them up. question how design this. each array can around 100 mb , n around 20. storing 20**2 * 100 mb in queue not possible. solution have shared memory object, result array each process keep adding results into.
multiprocessing has 2 means doing this, using shared memory or server process. neither of them seem support mutlidim arrays. can suggest way implement program? thx in advance.
one approach create memory mapped arrays in parent process, , pass them children fill. additionally should have multiprocessing.event every mapped array, chld process can signal parent array done.
