multiprocessing - Python multicore without communication -


i want call methode getrecommendations pickle recommendations specific user file. used code book works. saw 1 core works , want cores work, because faster.

here method.

def getrecommendations(prefs,person,similarity=sim_pearson):     print "working on recommendation"     totals={}     simsums={}     other in prefs:     # don't compare me myself         if other==person: continue         sim=similarity(prefs,person,other)         # ignore scores of 0 or lower         if sim<=0: continue         item in prefs[other]:             # score movies haven't seen yet             if item not in prefs[person] or prefs[person][item]==0:                 # similarity * score                 totals.setdefault(item,0)                 totals[item]+=prefs[other][item]*sim                 # sum of similarities                 simsums.setdefault(item,0)                 simsums[item]+=sim     # create normalized list     rankings=[(total/simsums[item],item) item,total in totals.items( )]     # return sorted list     rankings.sort( )     rankings.reverse( )     ranking_output = open("data/rankings/"+str(int(person))+".ranking.recommendations","wb")     pickle.dump(rankings,ranking_output)     return rankings 

it called via

for in customerid:          print "working on ", int(i)         #make working multiple cpu's         getrecommendations(pickle.load(open("data/critics.recommendations", "r")), int(i)) 

as can see try make recommendation every customer. used later.

so how can multiprocess method? don't reading few examples or documentation

you want (roughly, untested) like:

from multiprocessing import pool number_of_procs = 5 # number... not number of cores due i/o  pool = pool(number_of_procs)  in customerid:     pool.apply_async(getrecommendations, [i])  pool.close() pool.join() 

(this assuming pass 'i' getrecommendations, since pickle.load should done once)


Comments

Popular posts from this blog

design - Custom Styling Qt Quick Controls -

Unable to remove the www from url on https using .htaccess -