问题
I am using R lpsolve package to optimize my transportation model. My code runs fine but it takes a lot of time to run as I have huge number of nodes and paths. I am planning to run my code over hadoop cluster.
Please guide me regarding changes that i need to make to my code. I think that running optimization over hadoop cluster might be impossible as we might end up with local minimums instead of the global minimum.
I search internet for terms like "lpsolve hadoop" but didn't get anything useful.
Please direct me to material or examples that i should look at. =====================================update 1====================================== The original problem that I had is here.
I simplified the problem further and the current problem that I am solving is as below.
The R code and the input data file that I have built using excel is attached. In real scenario, input data file will be generated using SQL and will be >30,000 rows in length.
My input excel file:
startlink endlink link_dsc lnk_type cons_type cost equality_const fpc_const max_const
"source","-","-","0" "vmi1","MM1","VMI","1" source_to_VMI supply equality supply 0 100 null null
"vmi1","MM1","VMI","0" "cust1","MM1","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","1" "cust1","MM1","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","2" "cust1","MM1","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","3" "cust1","MM1","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","4" "cust1","MM1","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","5" "cust1","MM1","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","6" "cust1","MM1","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","7" "cust1","MM1","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","8" "cust1","MM1","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","9" "cust1","MM1","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","0" "vmi1","MM1","VMI","1" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","1" "vmi1","MM1","VMI","2" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","2" "vmi1","MM1","VMI","3" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","3" "vmi1","MM1","VMI","4" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","4" "vmi1","MM1","VMI","5" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","5" "vmi1","MM1","VMI","6" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","6" "vmi1","MM1","VMI","7" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","7" "vmi1","MM1","VMI","8" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","8" "vmi1","MM1","VMI","9" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","9" "vmi1","MM1","VMI","10" vmi_to_vmi_inv flow null 0 null null null
"vmi1","MM1","VMI","10" "SINK","-","-","100" vmi_to_sink esacpe null 100 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","1" "cust1","MM1","CUST","1" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","2" "cust1","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","3" "cust1","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","4" "cust1","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","5" "cust1","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","6" "cust1","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","7" "cust1","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","8" "cust1","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","9" "cust1","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_1d_AIR","10" "cust1","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"cust1","MM1","CUST","4" "SINK","-","-","100" cust_to_sink flow demand 0 null null 50
"vmi1","MM1","VMI","0" "cust1","MM1","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","1" "cust1","MM1","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","2" "cust1","MM1","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","3" "cust1","MM1","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","4" "cust1","MM1","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","5" "cust1","MM1","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","6" "cust1","MM1","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","7" "cust1","MM1","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 5 null null null
"vmi1","MM1","VMI","8" "cust1","MM1","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 5 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","2" "cust1","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","3" "cust1","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","4" "cust1","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","5" "cust1","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","6" "cust1","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","7" "cust1","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","8" "cust1","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","9" "cust1","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM1","SHIP_CUST_2d_AIR","10" "cust1","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"source","-","-","0" "vmi2","MM2","VMI","2" source_to_VMI supply equality supply 0 50 null null
"vmi2","MM2","VMI","0" "cust1","MM2","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","1" "cust1","MM2","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","2" "cust1","MM2","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","3" "cust1","MM2","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","4" "cust1","MM2","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","5" "cust1","MM2","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","6" "cust1","MM2","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","7" "cust1","MM2","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","8" "cust1","MM2","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","9" "cust1","MM2","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","0" "vmi2","MM2","VMI","1" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","1" "vmi2","MM2","VMI","2" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","2" "vmi2","MM2","VMI","3" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","3" "vmi2","MM2","VMI","4" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","4" "vmi2","MM2","VMI","5" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","5" "vmi2","MM2","VMI","6" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","6" "vmi2","MM2","VMI","7" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","7" "vmi2","MM2","VMI","8" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","8" "vmi2","MM2","VMI","9" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","9" "vmi2","MM2","VMI","10" vmi_to_vmi_inv flow null 0 null null null
"vmi2","MM2","VMI","10" "SINK","-","-","100" vmi_to_sink esacpe null 100 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","1" "cust1","MM2","CUST","1" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","2" "cust1","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","3" "cust1","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","4" "cust1","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","5" "cust1","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","6" "cust1","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","7" "cust1","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","8" "cust1","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","9" "cust1","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_1d_AIR","10" "cust1","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"cust1","MM2","CUST","9" "SINK","-","-","100" cust_to_sink flow demand 0 null null 10
"vmi2","MM2","VMI","0" "cust1","MM2","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","1" "cust1","MM2","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","2" "cust1","MM2","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","3" "cust1","MM2","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","4" "cust1","MM2","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","5" "cust1","MM2","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","6" "cust1","MM2","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","7" "cust1","MM2","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 1.4 null null null
"vmi2","MM2","VMI","8" "cust1","MM2","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 1.4 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","2" "cust1","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","3" "cust1","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","4" "cust1","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","5" "cust1","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","6" "cust1","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","7" "cust1","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","8" "cust1","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","9" "cust1","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust1","MM2","SHIP_CUST_2d_AIR","10" "cust1","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"vmi1","MM1","VMI","0" "cust3","MM1","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","1" "cust3","MM1","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","2" "cust3","MM1","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","3" "cust3","MM1","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","4" "cust3","MM1","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","5" "cust3","MM1","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","6" "cust3","MM1","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","7" "cust3","MM1","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","8" "cust3","MM1","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","9" "cust3","MM1","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 15 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","1" "cust3","MM1","CUST","1" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","2" "cust3","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","3" "cust3","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","4" "cust3","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","5" "cust3","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","6" "cust3","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","7" "cust3","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","8" "cust3","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","9" "cust3","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_1d_AIR","10" "cust3","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"cust3","MM1","CUST","6" "SINK","-","-","100" cust_to_sink flow demand 0 null null 5
"vmi1","MM1","VMI","0" "cust3","MM1","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","1" "cust3","MM1","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","2" "cust3","MM1","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","3" "cust3","MM1","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","4" "cust3","MM1","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","5" "cust3","MM1","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","6" "cust3","MM1","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","7" "cust3","MM1","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 15 null null null
"vmi1","MM1","VMI","8" "cust3","MM1","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 15 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","2" "cust3","MM1","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","3" "cust3","MM1","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","4" "cust3","MM1","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","5" "cust3","MM1","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","6" "cust3","MM1","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","7" "cust3","MM1","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","8" "cust3","MM1","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","9" "cust3","MM1","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM1","SHIP_CUST_2d_AIR","10" "cust3","MM1","CUST","10" shipcust_to_cust flow null 0 null null null
"vmi2","MM2","VMI","0" "cust3","MM2","SHIP_CUST_1d_AIR","1" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","1" "cust3","MM2","SHIP_CUST_1d_AIR","2" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","2" "cust3","MM2","SHIP_CUST_1d_AIR","3" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","3" "cust3","MM2","SHIP_CUST_1d_AIR","4" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","4" "cust3","MM2","SHIP_CUST_1d_AIR","5" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","5" "cust3","MM2","SHIP_CUST_1d_AIR","6" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","6" "cust3","MM2","SHIP_CUST_1d_AIR","7" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","7" "cust3","MM2","SHIP_CUST_1d_AIR","8" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","8" "cust3","MM2","SHIP_CUST_1d_AIR","9" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","9" "cust3","MM2","SHIP_CUST_1d_AIR","10" vmi_to_shipcust flow null 1.8 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","1" "cust3","MM2","CUST","1" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","2" "cust3","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","3" "cust3","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","4" "cust3","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","5" "cust3","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","6" "cust3","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","7" "cust3","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","8" "cust3","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","9" "cust3","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_1d_AIR","10" "cust3","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"cust3","MM2","CUST","8" "SINK","-","-","100" cust_to_sink flow demand 0 null null 7
"vmi2","MM2","VMI","0" "cust3","MM2","SHIP_CUST_2d_AIR","2" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","1" "cust3","MM2","SHIP_CUST_2d_AIR","3" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","2" "cust3","MM2","SHIP_CUST_2d_AIR","4" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","3" "cust3","MM2","SHIP_CUST_2d_AIR","5" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","4" "cust3","MM2","SHIP_CUST_2d_AIR","6" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","5" "cust3","MM2","SHIP_CUST_2d_AIR","7" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","6" "cust3","MM2","SHIP_CUST_2d_AIR","8" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","7" "cust3","MM2","SHIP_CUST_2d_AIR","9" vmi_to_shipcust flow null 1.8 null null null
"vmi2","MM2","VMI","8" "cust3","MM2","SHIP_CUST_2d_AIR","10" vmi_to_shipcust flow null 1.8 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","2" "cust3","MM2","CUST","2" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","3" "cust3","MM2","CUST","3" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","4" "cust3","MM2","CUST","4" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","5" "cust3","MM2","CUST","5" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","6" "cust3","MM2","CUST","6" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","7" "cust3","MM2","CUST","7" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","8" "cust3","MM2","CUST","8" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","9" "cust3","MM2","CUST","9" shipcust_to_cust flow null 0 null null null
"cust3","MM2","SHIP_CUST_2d_AIR","10" "cust3","MM2","CUST","10" shipcust_to_cust flow null 0 null null null
"SINK","-","-","100" "source","-","-","0" closed_loop flow null 0 null null null
My R code is as below: You have to take above csv data and paste it in file C:/dumy_network.csv.
My R code might not be very efficient but it serves the purpose!
library("lpSolve", lib.loc="C:/Users/njog/Documents/R/win-library/3.0")
#get the data from CSV file
mydata <- read.csv("C:/dumy_network.csv", header=TRUE)
#build list of nodes (no repetition)
nodes=unique(c(as.character(mydata$startlink),as.character(mydata$endlink)))
#list of all links
links=mydata[,1:2]
#cost of moving material on each link - optimization problem objective coefficients
cost=mydata$cost
#decision variable is flow on each link. Objective is to minimize product of (cost on each link*flow on the link). Therefore, count of decision variable is equal to count of links.
#constraints matrix: for each node in nodes, incoming quantity should be equal to outgoing quantity.
constraints=matrix(0,sum(mydata$max_const!='null')+sum(mydata$equality_const!='null')+length(nodes),length(mydata$endlink))
for (i in 1:length(nodes) ) {
constraints[i,]=t(1*(nodes[i]==links[,1])-1*(nodes[i]==links[,2]))
}
#get constraints for equality constraints- in some cases we have to ship material exactly same as this quanity.
constraint1=matrix(mydata$equality_const,1,length(mydata$equality_const))
constraint1[constraint1=="null"]=0
constraint1=as.numeric(constraint1)
constraint1_length=which(constraint1!=0)
constraint1_final=matrix(0,length(constraint1_length),length(mydata$equality_const))
for (i in 1:length(constraint1_length) ) {
constraint1_final[i,constraint1_length[i]]=1
}
start=length(nodes)+1
end=length(nodes)+length(constraint1_length)
constraints[start:end,]=constraint1_final
#get constraints for maxconstraints - in some cases we cannot ship material exceeding this quanity.
constraint2=matrix(mydata$max_const,1,length(mydata$max_const))
constraint2[constraint2=="null"]=0
constraint2=as.numeric(constraint2)
constraint2_length=which(constraint2!=0)
constraint2_final=matrix(0,length(constraint2_length),length(mydata$max_const))
for (i in 1:length(constraint2_length) ) {
constraint2_final[i,constraint2_length[i]]=1
}
start=end+1
end=end+length(constraint2_length)
constraints[start:end,]=constraint2_final
#building direction of constraints
direction=c(rep("=",length(nodes)),rep("=",sum(mydata$equality_const!='null')),rep("<=",sum(mydata$max_const!='null')))
#building right hand side of constraints
b1=as.numeric(as.character(mydata$equality_const[constraint1_length]))
b2=as.numeric(as.character(mydata$max_const[constraint2_length]))
b=c(rep(0,length(nodes)),b1,b2)
res = lpSolve::lp('min', cost, constraints, direction, b, all.int = TRUE)
res$solution
answers1=data.frame(res$solution)
mydata=cbind(mydata,answers1)
=====================================update 2====================================== Not getting any answers, so trying to simplify my question:
Examples section of the page gives a simple problem. Does anyone has ideas how to solve it using Mapreduce? I mean let's say I have a similar problem ,but with a huge number of variables and constraints then is there a way to achieve faster processing?
回答1:
In short, you want to do linear programming (lp), in large scale and you are not happy with the performance of your solver.
Approaches
I suggest the following approaches.
Solver
Do you need to use this solver? Here are some alternatives:
- Gurobi
- Mosek
- CPlex
Parallel execution
A weak approach to achiev parallel processing is to run process in parallel. Can you launch several optimations at once? (Here, you could write a simple exectuion task in Hadoop)
Improved inputs
The solvers can perform much better if you hand in the data accordingly (sparse matrix, ...)
In your code you use lpSolve::lp. According to the CRAN page of lpSolver 5.6.7 and its PDF, there is a special mode for the transportation probelm lpSolve:lp.transport. (I haven't worked with R and I am not familiar with the syntax. I am a Matlab "enthusiast".)
Latest research
Maybe some engergy is spent in academia. You might find a paper (cost some dollars) on sciencedirect.com.
Hardware
The faster the CPU, the faster your get a soluation (answer). Do you mind giving us some insights on your environment (OS, CPU)?
Final thought
Have you tried to raise your question in quant.stackexchange.com because numerical optimization is closely related to mathematics and "does not involve that much programming".
Parallel mode
We use the CLP. According some colleagues you can set a flag or use a different implementation to run the solver in parallel (1 problem executed on many cores). Take a look at SYMPHONY.
Keep us posted and do you mind giving us some insights on your environment (OS, CPU)?
My bounty
On Quora you find your answer that Hadoop is good in data processing, fast in I/O, not specially designed for optimization or calculation: Quora
Please let me know, if you do not want to award me with the bounties.
Missing bounty of +100
I have not received any bounties even though I did some research... This thread comes to a point where you do not give us the necessary details and we should solve it for you - this never gonna work out!
来源：https://stackoverflow.com/questions/22868923/how-to-augment-lpsolve-r-optimization-solution-to-run-on-a-hadoop-cluster