Dataflow in App Designer

What is a good way to implement a dataflow paradigm in App Designer?

My app consists of processing stages, each of which take input data and generate output data based on settings specified by GUI components. Each stage depends on other stages in a chain or tree (directed graph) being executed to provide its input data. The first stage is typically reading a file and the final stages typically display plots, images or tables of results. Because some stages take a significant time to execute, it makes sense whenever possible not to recompute intermediate results data. However, I need to find a clean way to code this behaviour.

Any suggestions welcome!

[J. Alex Lee]

in the object oriented paradigm, perhaps it is less productive to think about "data flow" versus "properties"...but I'm not sure exactly what is your question, and other computer science experts may shoot my comment down.

[Mario Malic]

That's a familiar name, hello again. Since I don't have programming background, i have not looked at it without some understanding of OOP. I took these so-called "helper-functions" as functions, and now I have a tiny mess of an application.

If you can program your data flow within a function or script externally, you can call it within App Designer.

However, it's best if you start typing.

[fburton]

Thanks, I have something that works. I just wondered if there were people out there who knew the best/right way of implementing these kind of programs, where only data that need recalculation actually are recalculated.

[J. Alex Lee]

it's difficult for me to understand, once you decided "only calculate data that need to be recalculated", what is preventing you from actually executingt? i'm not sure what you mean by the question "how to do that"? perhaps an example problem and why you believe it is challenging, or better yet an actual solution where you still have redundant calculations?

and also, does your question relate only to app designer, or does the problem persist in any kind of data processing workflow?

[fburton]

Okay, let's take a simple example. The app reads a file containing an image stack/movie of moving objects (Stage A), does some preprocessing to reduce noise (Stage B), and identifies the objects and their locations in each frame (Stage C). The GUI contains components to specify the parameters of each stage. So stage A's parameter would simply be the name of the file, stage B's parameters pertain to the noise reduction, and stage C's determine how objects are detected and localized. Each of these stages takes an input and processes it to produce an output. The GUI also contains a button to show plots of object location over time, which could also have associated parameters.

Pushing the plot button would cause stages A, B and C to be executed in turn and the output of stage C plotted as appropriate. Then one of the parameters associated with stage C is changed and the plot button pushed again. What happens? You could just run everything in the pipeline again, but only stage C needs to be executed because the output of stage B hasn't changed. If there are many stages and/or they take a long time to execute, it's better if the least amount of work is done to get the requested output. (If the file name changes then all stages would need to be executed when the plot button is pushed, of course.)

That's the problem I'd like to solve in a clean, readable, maintainable etc. way. It relates to App Designer because the program needs to find out which stages' parameters have changed. How the intermediate data and the state of each execution stage are stored is an issue of proper/idiomatic MATLAB language use. For example, would it be better to use the OOP facilities of MATLAB or could this be done just as easily and well procedurally?

In practice, the final app could have many such stages, each with their own set of parameters. Furthermore, the stages could be arranged in a tree rather than a simple linear chain, i.e. a stage might depend on the output of more than one other stage, and multiple stages might depend on a single ancestor stage.

Hope that clarifies what I'm asking about!

[Sourabh Kondapaka]

Hi fburton,

I think what you are looking for is a multi-windows app whilst passing data between each window.

These links could be helpful :

Creating a Multi-windowed App

Share data in Multi-windowed App

[Mario Malic]

I'd separate the calculation part out of plotting, for a complex app. I guess creating classes or having properties for each of the calculation stages would be the way to go. Then, the plotting part is where you'd have multiple options to choose what you want to plot. Within those options you can program what data you need, from which properties/classes and do the plotting.

[J. Alex Lee]

i completely agree about separating calculation from plotting!

Answer 1

[J. Alex Lee]

Yes, much clearer explanation of your question.

All due respect to Sourabh, I do not think you are looking for multi-window app with data passing.

First let's assume that you have the modular functions required on your back end squared away and available to do all the atomic steps of your workflow. [If you don't have that, focus on that first, do not even think about app-izing it].

If you can break your steps that cleanly, then conceptually why not have a "execute" button for each step? On file load, you can have the program go through all the steps, but then say if you want to change a parameter in step C, change it and press the "execute step C" button. Then you can have a "plot results" button, or executing step C can trigger execution of the plot. You can make the user experience a bit more seemless by associating each of your parameters with a step (A,B,C), and automatically executing the relevant step on ValueChanged. This would be easiest if your workflow is truly serial, so that changes in A trigger B, trigger C. If you have a more complicated workflow, like you may be able to change A, but actually wouldn't need to re-run B and can skip to D, but that DOES require triggering F, etc., then I guess you need to think about any kidn of automatic triggers more, or stick with manual step execution buttons or something like that.

To your specific question about the app "knowing" what step has changed, that just sounds like a logical variable to switch on or off depending on if a step has been executed or needs to be executed based on detecting a ValueChanged on something.

[J. Alex Lee]

it may help to use "events and listeners" in addition to callbacks