Skip to content

Conversation

@ktf
Copy link
Member

@ktf ktf commented Dec 10, 2020

This introduces:

  • --fairmq-send-buffer-size
  • --fairmq-recv-buffer-size
  • --fairmq-ipc-prefix

to customize the associated fairmq channels options.
Moreover it defaults --fairmq-rate-logging to 0.

This introduces:

* --fairmq-send-buffer-size
* --fairmq-recv-buffer-size
* --fairmq-ipc-prefix

to customize the associated fairmq channels options.
Moreover it defaults --fairmq-rate-logging to 0.
@ktf ktf requested a review from a team as a code owner December 10, 2020 21:14
@ktf
Copy link
Member Author

ktf commented Dec 10, 2020

@jgrosseo --fairmq-ipc-prefix to decide where to put the files
@ironMann --fairmq-recv-buffer-size 1 should make DPL not drop any message.

@ktf
Copy link
Member Author

ktf commented Dec 11, 2020

@TimoWilken I am not sure what failed here:

https://ali-ci.cern.ch/alice-build-logs/AliceO2Group/AliceO2/5048/66ce76daf2fafd0ea83d28f06a165abd763152a9/build_O2_o2-dataflow/pretty.html

could it be some issue with generating the web page?

@ktf ktf merged commit e6eb662 into AliceO2Group:dev Dec 11, 2020
@ktf ktf deleted the channel-cli-interface branch December 11, 2020 09:04
@ironMann
Copy link
Contributor

@ktf Is this a workaround or long term solution? More importantly, can this simply be the default behavior for all synchronous processing?

@TimoWilken
Copy link
Contributor

@ktf there was an error in the most recent builds:

2020-12-11@06:51:32:DEBUG:O2Suite:O2Suite:5048: ++ source /mnt/mesos/sandbox/sandbox/o2suite-dataflow/sw/slc7_x86-64/Configuration/v2.5.0-1/etc/profile.d/init.sh
2020-12-11@06:51:32:DEBUG:O2Suite:O2Suite:5048: /mnt/mesos/sandbox/sandbox/o2suite-dataflow/sw/slc7_x86-64/Control-OCCPlugin/v0.18.90-1/etc/profile.d/init.sh: line 9: /mnt/mesos/sandbox/sandbox/o2suite-dataflow/sw/slc7_x86-64/Configuration/v2.5.0-1/etc/profile.d/init.sh: No such file or directory

I'm guessing the O2 build succeeded, which is why the test results were included, but O2Suite failed with that error, so no log for that build was produced. So the overall run failed, but the error wasn't in any log that report-pr-errors could find.

The build succeeded last night though.

@ktf
Copy link
Member Author

ktf commented Dec 11, 2020

@ironMann The better way will be to have full req/reply but for that I need a bit more refactoring / thinking.
I cannot make this the default behaviour, at least for now, as I think it will affect the performance of the analysis framework (because multiple readers will stomp on each other). We could however have a special runSynchronousReconstruction.h which is included in place of runDataProcessing.h and provides proper customisation, but again if a component is used both in the sync and async it might lead to performance issues (assuming we want to process multiple files in parallel in async as well, if not, it should be less of an issue).

@ironMann
Copy link
Contributor

@kft I agree with the req/rep scheme. It's also needed to drive 2 independent DPL workflows with a single TF source on EPN. I'll add a jira subtask to track this feature.

@ktf
Copy link
Member Author

ktf commented Dec 11, 2020

@ironMann ok. keep in mind that you will also have to provide the proper config to the readout-proxy channel, since that's managed outside DPL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants