Matlab can't see a specific function when running in parallel

[poster]

<p>This has got me stumped.</p>

<p>I've written a function <code>parObjectiveFunction</code> that runs several simulations in parallel using <code>createJob</code> and <code>createTask</code>. It takes as an argument an <code>objectiveFunction</code> which is passed deeper into the code to calculate the objective function value for each simulation.</p>

<p>When I run <code>parObjectiveFunction</code> from the directory where <code>objectiveFunction</code> is found, it works as expected, but when I go one level up, it can no longer find <code>objectiveFunction</code>. The specific error I get is</p>

<pre><code>Error using parallel.Job/fetchOutputs (line 1255)
An error occurred during execution of Task with ID 1.

Error in parObjectiveFunction (line 35)
taskoutput = fetchOutputs(job);

Caused by:
Error using behaviourObjective/getPenalty (line 57)
Undefined function 'objectiveFunction' for input arguments of type 'double'.
</code></pre>

<p>(behaviourObjective is an object)</p>

<p>This is weird for several reasons. </p>

<p><code>objectiveFunction</code> is definitely in <code>path</code>, and when I try <code>which objectiveFunction</code>, it points to the correct function.
I have other components of the deeper code in other directories, and they are found without issue (they are objects rather than functions, but that shouldn't make a difference).
There's a line of code in <code>parObjectiveFunction</code> that runs the simulation, and when I run that directly in the matlab command window it finds <code>objectiveFunction</code> without issue.
I get the same results on my local machine and an HPC server.</p>

<p>My first thought was that the individual task might have its own <code>path</code> which didn't include <code>objectiveFunction</code>, but then that should cause problems for the other components (it doesn't). The problem is compounded because I can't work out how to debug the parallel code.</p>

<ol>
<li><p>What am I doing wrong? Code that produced the issue is below.</p></li>
<li><p>Are there any known issues where matlab can't find functions when
using parallel processing with <code>createJob</code>, <code>createTask</code>, <code>submit</code>
and <code>fetchOutputs</code>?.</p></li>
<li>How can you debug in matlab when the issue is
only when operating in parallel? None of my print statements appear.</li>
</ol>

<hr>

<p>To make something work for external testing would take quite a bit of hacking, but for the sake of the question, the parallel function is:</p>

<pre><code>function penalty = parObjectiveFunction(params, objectiveFunction, N)

% Takes a vector of input parameters, runs N instances of them in parallel
% then assesses the output through the objectiveFunction

n = params(1);
np = params(2);
ees = params(3);
ms = params(4);
cct = params(5);
wt = params(6);
vf = params(7);

dt = 0.001;
bt = 10;

t = 10;


c = parcluster;
job = createJob(c);


testFunction = @(run_number)behaviourObjective(objectiveFunction,n,np,ees,ms,cct,wt,vf,t,dt,bt,run_number);


for i = 1:N

createTask(job, testFunction, 1, {i});

end

submit(job);
wait(job);
taskoutput = fetchOutputs(job);

pensum = 0;
for i = 1:N
pensum = pensum + taskoutput{i}.penalty;
end

penalty = pensum/N;

end
</code></pre>



More answer... (https://stackoverflow.com/questions/59045322/matlab-cant-see-a-specific-function-when-running-in-parallel)