Effacer les filtres
Effacer les filtres

Simulink: database toolbox connection fails in Parallel Computing parfor loop?

3 vues (au cours des 30 derniers jours)
Hua Kun Tan
Hua Kun Tan le 29 Sep 2023
Commenté : Raymond Norris le 29 Sep 2023
I am trying to use mutlithreaded database updates in Simulink using the update function inside parfor
My code is something like this:
parfor i = 1:numLoad
updateQuery = strcat("{""$set"":", jsonencode(document(i)), "}");
findQuery = strcat("{""name"": """, document(i).name, """}");
temp_conn = dbConnections{i};
update(temp_conn, "collection", findQuery, updateQuery);
end
where dBConnections is initialised like this:
for i = 1:5
dbConnections{i} = mongoc("localhost",27017,"database"); % Replace with your connection setup
end
However, this crashes.
It will not crash if I either (1) change parfor to for, or (2) remove the update function.
Is the mongoc connection just not usable in parfor?
The error messages are:
Warning: A worker aborted during execution of the parfor loop. The parfor loop will now run again on the remaining workers. > In distcomp/remoteparfor/handleIntervalErrorResult (line 246) In distcomp/remoteparfor/getCompleteIntervals (line 396) In parallel_function>distributed_execution (line 746) In parallel_function (line 578) In my_model>Outputs (line 118)
An error occurred while running the simulation and the simulation was terminated
Caused by:
  • Error evaluating registered method 'Outputs' of MATLAB S-Function 'my_model' in 'my_model/Level-2 S-Function For MongoDB'. The following is the MATLAB call stack (file names and line numbers) that produced this error: ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [195] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [259] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [396] ['C:\Users\me\MATLAB\Projects\learning_simulink\work\my_model.m'] [118]
  • All workers aborted during execution of the parfor loop
Any ideas for what the issue is?

Réponses (1)

Raymond Norris
Raymond Norris le 29 Sep 2023
To summarize:
  • You're running a single Simulink model (my_model), in which you have a block (Level-2 S-Function for MongoDB) that's running MATLAB code
  • In this MATLAB code, you have serial code that runs a for-loop to create an array of database connections (but really all the same db connection)
  • And you then run a parfor-loop, such that each loop iteration updates a collection with a set of documents a Mongo DB
A couple of thoughts:
  2 commentaires
Hua Kun Tan
Hua Kun Tan le 29 Sep 2023
Hi, thank you, you have summarised it quite well.
The array of db connections was an attempt to figure out if the issue was caused by multiple threads using the same connection.
I attempted to use parallel.pool.Constant to solve my issue, however, the same errors persist.
conn = mongoc("localhost",27017,"database"); % ip, port, database name
constant_conn = parallel.pool.Constant(conn);
parfor i = 1:numLoad
updateQuery = strcat("{""$set"":", jsonencode(document(i)), "}");
findQuery = strcat("{""name"": """, document(i).name, """}");
update(constant_conn.Value, "collection", findQuery, updateQuery);
end
An error occurred while running the simulation and the simulation was terminated
Caused by:
  • Error evaluating registered method 'Outputs' of MATLAB S-Function 'my_model' in 'my_model/Level-2 S-Function For MongoDB'. The following is the MATLAB call stack (file names and line numbers) that produced this error: ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [195] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [259] ['C:\Program Files\MATLAB\R2022b\toolbox\parallel\distcomp\+distcomp\remoteparfor.m'] [396] ['C:\Users\me\MATLAB\Projects\learning_simulink\work\my_model.m'] [122]
  • All workers aborted during execution of the parfor loop.
The baffling thing is that replacing parfor with for runs perfectly.
Do you have any luck attempting to run mongodb updates in a parfor loop?
Raymond Norris
Raymond Norris le 29 Sep 2023
Change the creation, as such
constant_conn = parallel.pool.Constant(@() mongoc("localhost",27017,"database")); % ip, port, database name
This will create the connection on the workers, instead of creating it from the client and then passing it to the workers.

Connectez-vous pour commenter.

Catégories

En savoir plus sur Reporting and Database Access dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by