php - mongodb queries per second -

i have server 64g ram , running script compares 1 million data in csv file against database. if matches found, script prints number of matches @ end of execution.

the script when ran, taking 3 minutes finish. tested 50,000, 1 lakh, 3 lakh, 5 lakh data files , performance rate or rate @ script executed proportional. there enough memory free in server. mongostat output when script running pasted below. questions is, believe script executing close 5000 queries per second. have read in many posts, getting average of 50k queries per second. how can achieved? server running ubuntu, 64 bit, , 24 cores.

insert  query update delete getmore command flushes mapped  vsize res faults            locked db idx miss %     qr|qw   ar|aw  netin netout  conn       time     *0   3885     *0     *0       0     1|0       0    12g  24.2g 64m      0 db_list_restore:0.0%          0       0|0     1|0 380k   142k     2   03:09:26     *0   4188     *0     *0       0     1|0       0    12g  24.2g 68m      0 db_list_restore:0.0%          0       0|0     0|0 410k   153k     2   03:09:27     *0   4462     *0     *0       0     1|0       0    12g  24.2g 72m      0 db_list_restore:0.0%          0       0|0     0|0 440k   163k     2   03:09:28     *0   4401     *0     *0       0     1|0       0    12g  24.2g 76m      0 db_list_restore:0.0%          0       0|0     0|0 435k   161k     2   03:09:29     *0   4368     *0     *0       0     2|0       0    12g  24.2g 81m      0 db_list_restore:0.0%          0       0|0     1|0 432k   160k     2   03:09:30     *0   4416     *0     *0       0     1|0       0    12g  24.2g 84m      0 db_list_restore:0.0%          0       0|0     1|0 437k   161k     2   03:09:31     *0   4245     *0     *0       0     1|0       0    12g  24.2g 89m      0 db_list_restore:0.0%          0       0|0     0|0 420k   155k     2   03:09:32     *0   4561     *0     *0       0     1|0       0    12g  24.2g 93m      0 db_list_restore:0.0%          0       0|0     1|0 451k   167k     2   03:09:33     *0   3920     *0     *0       0     1|0       0    12g  24.2g 97m      0 db_list_restore:0.0%          0       0|0     0|0 388k   144k     2   03:09:34     *0   4307     *0     *0       0     2|0       0    12g  24.2g 105m      0 db_list_restore:0.0%          0       0|0     0|0 426k   157k     2   03:09:35  

these number relatively reasonable single process doing work - depending on sort of query doing of course.

in general you'll have thousands of webserver processes (depending on application workload , cpu count, maybe few hundreds), generating traffic - you'll have thousands of sockets open server allowing scale whichever component in infrastructure independently.

for single process application you'll stuck in "wait loops" there individual resource asked @ each time never able maximize potential 1 process, 1 socket, 1 task. want occupy resources time.

to achieve want parallelize workflow as can. there no predefined number of workers correct answer, how work each process should doing you'll have figureout trying out.

if have million entries in csv file, may want start dividing work in 10 pieaces, create 10 php processes seek file , work read through 100.000 entries. benchmark that, , maybe try 100 processes, each working on 10.000 entries @ time, , compare previous results.