Community Profile

photo

Edric Ellis

MathWorks

Last seen: Today Actif depuis 2011

Developer at MathWorks, working on Parallel Computing Toolbox. Professional Interests: parallel computing, distributed arrays, gpu

Statistiques

All
  • 5-Star Galaxy Level 3
  • Personal Best Downloads Level 1
  • First Review
  • First Submission
  • 36 Month Streak
  • Thankful Level 1
  • Guiding Light
  • Revival Level 2
  • Knowledgeable Level 5
  • First Answer
  • Solver

Afficher les badges

Content Feed

Afficher par

A répondu
terminate program if worker aborts in parallel loop
If you create your parpool using the option SpmdEnabled=true, then this has the side-effect of making parallel pools terminate c...

5 jours il y a | 0

| A accepté

A répondu
How is the work distributed in parfor loop?
If you know in advance which iterations are the time-consuming ones, then you could consider running those together as a separat...

10 jours il y a | 1

| A accepté

A répondu
How can I debug a parallel script with spmd?
Unfortunately you cannot set breakpoints in code running on workers. If the problem is related to the messages you are sending u...

environ un mois il y a | 0

A répondu
how parfor distributes resources to each iteration if the number of cores is larger than the number of iterations
When a parfor loop executes on a parallel pool, PCT will divide up the entire loop range 1:N into a series of batches known as "...

environ un mois il y a | 3

A répondu
bug parfor: analyzing and transferring files to worker: not done: blocked
The "analyzing and transferring files to workers" message occurs only when the workers encounter a failure to find some code tha...

environ un mois il y a | 0

A répondu
parpool use existing slurm job
Further to @Venkat Siddarth Reddy's suggestion, what you might want to do is something like this: clus = parcluster('mySlurmPro...

environ un mois il y a | 0

A répondu
Store intermediate results of cost function when solver is run in parallel
You could use a parallel.pool.PollableDataQueue to let the workers send the data to the client. The idea is that you create the ...

environ 2 mois il y a | 0

| A accepté

A répondu
"Invalid or deleted object" when using clib object in parfor loop
I'm not terribly familiar with clibgen I'm afraid, but I think the problem here is almost certainly that the function myCppFunct...

environ 2 mois il y a | 2

| A accepté

A répondu
MATLAB workers can't access files in network shared folder
You could try using FileStore for this. You'll first need to save your figure to a temporary location on disk, and then you can ...

environ 2 mois il y a | 0

| A accepté

A répondu
Running parfor on multiple nodes (20 nodes = 960 cores) using Slurm (super computer)
To run across multiple nodes of a cluster, you need to set up MATLAB Parallel Server on the cluster. You might well need help fr...

3 mois il y a | 1

A répondu
Parallelization data organization/transfer
parfor is designed to "just work" for a wide range of cases, but it's true that sometimes you need a bit more control. parforOpt...

3 mois il y a | 0

A répondu
Access Violation Detected MatLAB
That crash dump clearly implicates your MEX file octtrees.query_index_moct. One of the common pitfalls when running MEX files in...

3 mois il y a | 0

A répondu
Is it possible to run multiple lines at once in a Matlab script?
To expand just a little on @Stephen23's comment - you can do this if you have Parallel Computing Toolbox. You should first restr...

3 mois il y a | 1

| A accepté

A répondu
CNN Performance: CPU Consistency vs. GPU Variance - Why?
I'm not certain if it will make everything consistent, but note that random state on the GPU is controlled by the gpurng functio...

3 mois il y a | 0

A répondu
Parfor with options = statset('UseParallel',true);
If you are able to "flatten" the parallelism so that it is all exposed to a single parfor loop with enough iterations to keep al...

3 mois il y a | 0

| A accepté

A répondu
Is it possible to use parfeval and backgroundPool to write to a Mongo database in a background thread?
In your first case using parfor, if you haven't changed any settings, you will be using a process pool. (The default profile is ...

3 mois il y a | 0

A répondu
Avoid memory copy in thread-based environment (parfeval)
In this case, thread-based workers already avoid copying the large array m to the workers. See this section in the doc. Note tha...

4 mois il y a | 2

| A accepté

A répondu
How to assign values to arrays inside the PARFOR loop in Parallel Computing Toolbox?
To assign into A here, you need to follow the rules of sliced variables in parfor. In this case, you need to modify your code to...

5 mois il y a | 1

| A accepté

A répondu
Is there a parallel version of splitapply()?
tall arrays support parallel execution of splitapply. Would that work for your case?

5 mois il y a | 0

| A accepté

A répondu
How do you organize data from a parfor loop?
You can do this more directly by flattening the parfor loop, and using a combination of ind2sub and reshape. The idea is as foll...

9 mois il y a | 0

A répondu
How to use parallel.pool.Constant to save on overheard for large array constant across all calls to parfeval?
You've got a subtle flaw in your first piece of code that allows the program to run, but it does not actually benefit from the u...

9 mois il y a | 0

| A accepté

A répondu
Read/Write Limitations with Parallel Computing Toolbox (parfor)
Generally speaking, parfor is conservative in what it allows to ensure that by default, all variable accesses are safe. It is s...

10 mois il y a | 0

| A accepté

A répondu
Parallel workers automatically shutting down in the middle of RL parallel training.
If workers are leaving the pool one at a time while the pool is busy, this almost certainly means that they are crashing. I reco...

10 mois il y a | 0

| A accepté

A répondu
Clear persistent variables of function on worker
Whenever you issue a clear command at the client, the same clear command is issued on the workers. So you can simply do clear p...

10 mois il y a | 1

A répondu
Error using parpool. Parallel pool failed to start with the following error. For more detailed information, validate the profile 'Processes' in the Cluster Profile Manager.
This is probably the following known problem in R2023a: https://www.mathworks.com/support/bugreports/2968710 . There are workaro...

11 mois il y a | 0

A répondu
Matlab spmd Collecting Composite Arrays
You missed a crucial clue from your question - where are you assigning hx. I'm going to guess you have something like this: spm...

12 mois il y a | 0

| A accepté

A répondu
Matlab spmd and CoolProp (CPython library)
You need to avoid transferring the objects to the workers, and instead simply build the objects directly on the workers. The sim...

12 mois il y a | 0

| A accepté

A répondu
How to shut down all running workers of paarpools?
You can shut down all remaining workers of the currently running pool by executing: delete(gcp('nocreate')) There should be no...

12 mois il y a | 0

A répondu
Assigning values to a vector out of order in parallel (parfor)
One of the constraints of parfor is that indexed outputs like myVec must be "sliced" - that means that you must assign only elem...

12 mois il y a | 0

| A accepté

A répondu
Is it possible to create multiple parallel pools simultaneously
There's no direct way to do what you're after here. (As @Walter Roberson points out, you can have a backgroundPool and a parpool...

environ un an il y a | 1

| A accepté

Charger plus