14
$\begingroup$

How to monitor and/or print partial progress of Apply, e.g. f @@@ list?

I have found answers using Map (e.g. this nice trick using Echo, or this general trick), but none of these answers work for Apply.

All I want is some kind of feedback to let me know that the computation is progressing and hasn't stalled (think of something that take 12+ hours to run). The Echo trick above worked perfectly when I used Map, but not Apply.

(I understand that I can rewrite my code to avoid using Apply and monitor things manually, but this is not my question. I am curious about Apply specifically.)

Improve this question
$\endgroup$
1
  • $\begingroup$ I would remind very old solution with ShowStatus[] function, which allows to place status string in the bottom left corner of the window. This avoids contamination of notebook content.mathematica.stackexchange.com/questions/26438/… $\endgroup$
    – Acus
    Commented May 11, 2022 at 5:43

4 Answers 4

Reset to default
8
$\begingroup$

Probably many ways to do this. For instance, imagine you got a list of large integers

list = Table[{RandomInteger[{10^20, 10^21}]}, 10]

for which you need to check the number of prime factors using @@@:

In[]:= Length@*FactorInteger@@@list
Out[]= {3,6,3,5,4,5,3,3,2,5}

you could just add, for instance, EchoTiming, which would also time each computation in addition to monitoring the overall progress:

EchoTiming@*Length@*FactorInteger@@@list

enter image description here

Improve this answer
$\endgroup$
16
$\begingroup$

You can always transform an expression of the form Apply[f,list,level] to the (almost) equivalent Map[Apply[f],list,level]. You can then use any of the solutions for Map, such as ResourceFunction["MonitorProgress"]:

ResourceFunction["MonitorProgress"][Apply[Pause] /@ Table[{0.2}, 10]]

enter image description here

If you want to automate the process of this transformation for the case of MonitorProgress, you can add the following definitions:

mp = Symbol@# &@ResourceFunction["MonitorProgress", "SymbolName"];
mpt = Symbol[Context@Evaluate@mp <> "MonitorProgressTransform"];

mpt[HoldPattern@Apply[func_, list_, level_], o : OptionsPattern[mp]] :=
 mpt[Map[Apply[func], list, level], o]

This effectively adds the missing transformation rules for expressions of the for Apply[f,list,level to the internals of MonitorProgress, until I get around to adding them to the official version of the function.

Now, MonitorProgress accepts an untransformed Apply[f,list,level] as input:

ResourceFunction["MonitorProgress"][Pause @@@ Table[{0.2}, 10]]
(* {Null, Null, Null, Null, Null, Null, Null, Null, Null, Null} *)
Improve this answer
$\endgroup$
3
  • $\begingroup$ What do the three colors in the plot represent? $\endgroup$
    – Greg Hurst
    Commented May 10, 2022 at 17:46
  • $\begingroup$ @GregHurst the horizontal width gives the fraction of steps completed, just like a normal progress bar. As for the vertical part: the solid line shows the total time taken so far. And the dashed line shows the estimated total time at that point. The three colors are just there to differentiate the areas between the lines. So if all steps take the same amount of time, the dashed line is ideally constant, while the solid line is straight, just like in the animation above. If steps are getting slower, both lines will bend upwards, similarly they will bend downwards when speeding up. $\endgroup$
    – Lukas Lang
    Commented May 10, 2022 at 22:15
  • $\begingroup$ Thanks! It’s an impressively thorough function! $\endgroup$
    – Greg Hurst
    Commented May 11, 2022 at 1:34
6
$\begingroup$

Practically speaking, this is what I actually do:

First, if I need a variable to be dynamically updated, I use a global foo. Global side effects are generally a questionable programming practice, but we're basically just monitoring a command, not programming. In my practice, foo functions as a reserved symbol that is unsafe to use in a program but fine to assign a value at anytime. Substitute your own reserved symbol as you wish. I have a list of them that I would never use in a program.

Second, I use PrintTemporary@Dynamic@{Clock[Infinity], foo & other data} to get just one temporary, dynamically-updated cell. You might want to use Short[foo] so that you don't accidentally try to display a gigabyte array. Monitor does something like this, but I want something more flexible often enough that I just always start with PrintTemporary@Dynamic@{Clock[Infinity],...} and think about what I want shown. And having the running clock is also often convenient (for instance, if it stops running, or minutes after something that should take only seconds.)

Examples:

foo=0;
PrintTemporary@Dynamic@{Clock[Infinity], foo}
(foo++;f[##])& @@@ list

foo=0;
PrintTemporary@Dynamic@{Clock[Infinity], i, Short[foo]}
Table[f @@ (foo = list[[i]]), {i, Length@list}]

foo = "init";
PrintTemporary@Dynamic@{Clock[Infinity], foo};
f[foo = ##] & @@@ RandomInteger[9, {20000000, 4}];

Bonus (watch the timefront evolve instability):

PrintTemporary@Dynamic@{Clock@Infinity, foo};
NDSolve[{D[u[t, x], t, t] == 2 D[u[t, x], x, x]^3,
  u[0, x] == Sin[Pi x], Derivative[1, 0][u][0, x] == Sin[Pi x], 
  u[t, 0] == 0, u[t, 5] == 0},
 u, {t, 0, 10}, {x, 0, 5}, Method -> "MethodOfLines",
 StepMonitor :> (foo = ListLinePlot[Head@u[t, x], PlotLabel -> t])]
Improve this answer
$\endgroup$
0
5
$\begingroup$

You can do something like:

Module[{i},
 result = Monitor[
  Table[
   Pause[0.1]; f @@ i,
   {i, RandomReal[1, {20, 3}]}
   ],
   i
  ]
]

The Module is optional, but I like it to make the iterator in Table lexically scoped.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.