multiprocessing - Python multicore without communication -
i want call methode getrecommendations pickle recommendations specific user file. used code book works. saw 1 core works , want cores work, because faster.
here method.
def getrecommendations(prefs,person,similarity=sim_pearson): print "working on recommendation" totals={} simsums={} other in prefs: # don't compare me myself if other==person: continue sim=similarity(prefs,person,other) # ignore scores of 0 or lower if sim<=0: continue item in prefs[other]: # score movies haven't seen yet if item not in prefs[person] or prefs[person][item]==0: # similarity * score totals.setdefault(item,0) totals[item]+=prefs[other][item]*sim # sum of similarities simsums.setdefault(item,0) simsums[item]+=sim # create normalized list rankings=[(total/simsums[item],item) item,total in totals.items( )] # return sorted list rankings.sort( ) rankings.reverse( ) ranking_output = open("data/rankings/"+str(int(person))+".ranking.recommendations","wb") pickle.dump(rankings,ranking_output) return rankings
it called via
for in customerid: print "working on ", int(i) #make working multiple cpu's getrecommendations(pickle.load(open("data/critics.recommendations", "r")), int(i))
as can see try make recommendation every customer. used later.
so how can multiprocess method? don't reading few examples or documentation
you want (roughly, untested) like:
from multiprocessing import pool number_of_procs = 5 # number... not number of cores due i/o pool = pool(number_of_procs) in customerid: pool.apply_async(getrecommendations, [i]) pool.close() pool.join()
(this assuming pass 'i' getrecommendations, since pickle.load should done once)
Comments
Post a Comment