# Adding a couple Information keys to enable temporal filters to be run in situ

**URL:** <https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356>\
**Category:** Development\
**Tags:** proposal\
**Created:** [May 2, 2023, 2:44pm UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356 "2023-05-02T14:44:32Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yohann\_Bearzi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/yohann_bearzi/32/671_2.png) [@Yohann\_Bearzi](https://discourse.vtk.org/u/Yohann_Bearzi)\
**Post date:** [May 2, 2023, 2:44pm UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356/1 "2023-05-02T14:44:32Z")

</div>

There are problems currently when running some temporal filters in situ. Since the entire time series is not available up straight, temporal filters need to be able to produce a correct output up to the available timesteps, and then to be run on upcoming timesteps using some internal cache. An example of a filter failing in situ is `vtkTemporalStatistics`.

However, we should, for performance reasons, still be able to run a filter on the entire time series without having to generate intermediate states, typically when using ParaView without catalyst.

As a consequence, in filters for which it applies, there should be 2 modes: a regular mode (compute all the time steps at once), and an in situ mode (access to incomplete temporal data). In the regular mode, one can just call `Update()` to get the output. In situ, one needs to call `UpdateTimeStep(double t)` for each time step.

Ideally, there shouldn’t be extra parameters in temporal filters to avoid users producing wrong outputs because they forgot to turn ON the in situ mode inside a filter that needs it. Instead, I propose that we add a new Information key that should be set on source output information in situ. The key would be passed downstream and all filters needing to change how they behave would know when to. We could call it `INCOMPLETE_TIME_STEPS`. When set to true, filters for which it applies can cache whatever data they need to cache in order to produce a complete dataset for each time step. An implementation is proposed [here](https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10104).

This key would be automatically set on all sources when using ParaView Catalyst (see implementation [here](https://gitlab.kitware.com/paraview/paraview/-/merge_requests/6292)).

An issue arises when setting ON `INCOMPLETE_TIME_STEPS`: the filter doesn’t know when to reinitialize its cache. The temporal cache held by concerned filters needs wiped when the user wants to rerun all the timesteps as he changes a parameter in the visualization for example. There is no way to know when a filter gets updated to add a new timestep, or when it does from scratch. To give awareness of such context, I propose to add a new information key that gets set by the executive automatically: `WIPE_ALGORITHM_CACHE`. It is set to 1 when any upstream filter has called `Modified()`. It gets propagated downstream by the pipeline. When a concerned temporal filter sees it set, it knows to wipe its temporal cache. You can find how it would be implemented and how to use the key [here](https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10104).

Does anyone have any comments or suggestions?

---

<div class="post-metadata">

**Author:** ![mwestphal](https://discourse.vtk.org/user_avatar/discourse.vtk.org/mwestphal/32/19_2.png) [@mwestphal](https://discourse.vtk.org/u/mwestphal)\
**Post date:** [May 2, 2023, 2:52pm UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356/2 "2023-05-02T14:52:33Z")

</div>

@nicolas.vuaille @Francois_Mazen @Timothee_Couble @lgivord

---

<div class="post-metadata">

**Author:** ![danlipsa](https://discourse.vtk.org/user_avatar/discourse.vtk.org/danlipsa/32/390_2.png) [@danlipsa](https://discourse.vtk.org/u/danlipsa)\
**Post date:** [May 2, 2023, 2:56pm UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356/3 "2023-05-02T14:56:48Z")

</div>

To me, WIPE\_ALGORITHM\_CACHE is too generic. I would rename that and link it to the first key. So have something like the following instead:

WIPE\_INCOMPLETE\_TIME\_STEPS\_CACHE

---

<div class="post-metadata">

**Author:** ![Yohann\_Bearzi](https://discourse.vtk.org/user_avatar/discourse.vtk.org/yohann_bearzi/32/671_2.png) [@Yohann\_Bearzi](https://discourse.vtk.org/u/Yohann_Bearzi)\
**Post date:** [May 2, 2023, 3:04pm UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356/4 "2023-05-02T15:04:53Z")

</div>

Good point. Now that I think about it, this key as I implemented it so far always gets set when an upstream filter called `Modified()`, even when `INCOMPLETE_TIME_STEPS` is not set.

So maybe I should either set it in this context and rename it `WIPE_INCOMPLETE_TIME_STEPS_CACHE`, or leave the code as is and rename it `UPSTREAM_CALLED_MODIFIED` so this key is available if someone needs this information for another reason?

---

<div class="post-metadata">

**Author:** ![danlipsa](https://discourse.vtk.org/user_avatar/discourse.vtk.org/danlipsa/32/390_2.png) [@danlipsa](https://discourse.vtk.org/u/danlipsa)\
**Post date:** [May 2, 2023, 3:18pm UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356/5 "2023-05-02T15:18:18Z")

</div>

In that case UPSTREAM\_CALLED\_MODIFIED seems fine. I cannot think of a case when an upstream filter get its Modified() called and you should not clear the cache.

---

<div class="post-metadata">

**Author:** ![nicolas.vuaille](https://discourse.vtk.org/user_avatar/discourse.vtk.org/nicolas.vuaille/32/5363_2.png) [@nicolas.vuaille](https://discourse.vtk.org/u/nicolas.vuaille)\
**Post date:** [May 3, 2023, 8:08am UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356/6 "2023-05-03T08:08:43Z")

</div>

@jfausty It may be of interest for the Particles Tracking filters

---

<div class="post-metadata">

**Author:** ![Jens\_Munk\_Hansen](https://discourse.vtk.org/user_avatar/discourse.vtk.org/jens_munk_hansen/32/1215_2.png) [@Jens\_Munk\_Hansen](https://discourse.vtk.org/u/Jens_Munk_Hansen)\
**Post date:** [May 3, 2023, 11:06pm UTC](https://discourse.vtk.org/t/adding-a-couple-information-keys-to-enable-temporal-filters-to-be-run-in-situ/11356/7 "2023-05-03T23:06:53Z")

</div>

I’m personally struggling a bit to find a solution for client receiving data that needs to expose TIME\_STEPS downstream. The problem is that it doesn’t know a time range from start and further downstream, I need a temporal cache, which should have a certain depth. My current solution is some dirty hack of calling a home-made update time range for each Update(). I guess this solution could be of use for me.