2021年5月2日星期日

Errors when parallelizing for loop

I'm new to parallel-processing and attempting to parallelize a for loop in which I create new columns in a data frame by matching a column in said data frame with two other data frames. j, the data frame I'm attempting to create columns in is 400000 x 54. a and c, the two data frames I'm matching j with are 5000 x 12 and 45000 x 8 (respectively).

Below is my initial loop prior to the attempt at parallelizing:

for(i in 1:nrow(j)) {     if(j$Inspection_Completed[i] == TRUE) {        next    }      j$Assigned_ID <- a$Driver[match(j$car_name, a$CarName)]      j$Title <- c$Title[match(j$Site_ID, c$LocationID)]      j$Status <- c$Status[match(j$Site_ID, c$LocationID)]           }  

So far I have attempted the following:

cl <- snow::makeCluster(4)  doSNOW::registerDoSNOW(cl)    foreach::foreach(i = 1:nrow(j)) foreach::`%dopar%` {     if(j$Inspection_Completed[i] == TRUE) {        next     }      j$Assigned_ID <- a$Driver[match(j$car_name, a$CarName)]      j$Title <- c$Title[match(j$Site_ID, c$LocationID)]      j$Status <- c$Status[match(j$Site_ID, c$LocationID)]   }   stopCluster(cl)  

However, when I run the code above I receive several errors.

Error: unexpected symbol in "foreach::foreach(i = 1:nrow(j)) foreach"  

And:

Error: object 'i' not found  

Lastly:

Error: unexpected '}' in "}"  

I'm not sure why I'm getting these errors. None of the columns in any of the data frames are factors and I haven't been able to spot any mismatched parentheses or brackets. I've also done this without the snow and doSNOW packages and the result is the same. I've ran it without the tick marks around dopar as well with the same result.

https://stackoverflow.com/questions/67362036/errors-when-parallelizing-for-loop May 03, 2021 at 07:43AM

没有评论:

发表评论