Skip to content

TOF DCS Processor #5047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shahor02 merged 1 commit into from
Dec 16, 2020
Merged

TOF DCS Processor #5047

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions Detectors/DCS/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ o2_add_executable(
PUBLIC_LINK_LIBRARIES O2::Framework
O2::DetectorsDCS)

o2_add_executable(
sim-workflow
COMPONENT_NAME dcs
SOURCES testWorkflow/dcs-sim-workflow.cxx
PUBLIC_LINK_LIBRARIES O2::Framework O2::DetectorsDCS)

if(OpenMP_CXX_FOUND)
target_compile_definitions(${targetName} PRIVATE WITH_OPENMP)
target_link_libraries(${targetName} PRIVATE OpenMP::OpenMP_CXX)
Expand Down
16 changes: 14 additions & 2 deletions Detectors/DCS/src/DataPointGenerator.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
#include "DetectorsDCS/DataPointCreator.h"
#include "DetectorsDCS/DataPointCompositeObject.h"
#include "DetectorsDCS/StringUtils.h"
#include "Framework/Logger.h"
#include <fmt/format.h>
#include <random>
#include <utility>
Expand All @@ -23,16 +24,27 @@ namespace
{
std::pair<uint32_t, uint16_t> getDate(const std::string& refDate)
{

uint32_t seconds;
if (refDate.empty()) {
auto current = std::time(nullptr);
auto t = std::localtime(&current);
uint32_t seconds = mktime(t);
seconds = mktime(t);
Copy link
Collaborator Author

@chiarazampolli chiarazampolli Dec 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphecetche : this is the fix.

} else {
std::tm t{};
std::istringstream ss(refDate);
ss >> std::get_time(&t, "%Y-%b-%d %H:%M:%S");
seconds = mktime(&t);
if (ss.fail()) { // let's see if it was passed as a TDatime
Copy link
Collaborator Author

@chiarazampolli chiarazampolli Dec 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphecetche : this is to accept also the TDatime format

std::tm tt{};
std::istringstream sss(refDate);
sss >> std::get_time(&tt, "%a %b %d %H:%M:%S %Y");
if (sss.fail()) {
LOG(ERROR) << "We cannot parse the date";
}
seconds = mktime(&tt);
} else {
seconds = mktime(&t);
}
}
uint16_t msec = 5;
return std::make_pair(seconds, msec);
Expand Down
1 change: 1 addition & 0 deletions Detectors/DCS/src/DetectorsDCSLinkDef.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,6 @@
#pragma link C++ class std::unordered_map < o2::dcs::DataPointIdentifier, o2::dcs::DataPointValue> + ;
#pragma link C++ function o2::dcs::expandAlias(const std::string&);
#pragma link C++ function o2::dcs::expandAliases(const std::vector<std::string>&);
#pragma link C++ class std::unordered_map < o2::dcs::DataPointIdentifier, std::string> + ;

#endif
68 changes: 54 additions & 14 deletions Detectors/DCS/testWorkflow/DCSRandomDataGeneratorSpec.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@
#include "Framework/DeviceSpec.h"
#include "Framework/Logger.h"
#include "Framework/Task.h"
#include <TDatime.h>
#include <random>
#include <variant>
#include <string>
#include <algorithm>

using namespace o2::framework;

Expand Down Expand Up @@ -54,9 +56,18 @@ using HintType = std::variant<DataPointHint<double>,
std::vector<int> generateIntegers(size_t size, int min, int max)
{
std::uniform_int_distribution<int> distribution(min, max);
std::mt19937 generator;
std::vector<int> data(size);
std::generate(data.begin(), data.end(), [&]() { return distribution(generator); });
std::mt19937 generator(std::random_device{}());
Copy link
Collaborator Author

@chiarazampolli chiarazampolli Dec 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphecetche : this is to change the subset of DPs that are generated at every TF

std::vector<int> data;
Copy link
Collaborator Author

@chiarazampolli chiarazampolli Dec 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphecetche , this is to make the generation of the DPs in the delta map such that there are no duplicated DPs - the delta map will contain one value only per DP.

while (data.size() != size) {
data.emplace_back(distribution(generator));
std::sort(begin(data), end(data));
auto last = std::unique(begin(data), end(data)); // make sure we do not duplicate
data.erase(last, end(data));
}
std::shuffle(begin(data), end(data), generator);
for (auto i = 0; i < data.size(); ++i) {
LOG(INFO) << "Generating randomly DP at index " << data[i];
}
return data;
}

Expand All @@ -68,12 +79,20 @@ std::vector<int> generateIntegers(size_t size, int min, int max)
* @returns a vector of DataPointCompositeObjects
*/
std::vector<o2::dcs::DataPointCompositeObject> generate(const std::vector<HintType> hints,
float fraction = 1.0)
float fraction = 1.0,
uint64_t tfid = 0)
{
std::vector<o2::dcs::DataPointCompositeObject> dataPoints;

auto GenerateVisitor = [](const auto& t) {
return o2::dcs::generateRandomDataPoints({t.aliasPattern}, t.minValue, t.maxValue);
TDatime d;
Copy link
Collaborator Author

@chiarazampolli chiarazampolli Dec 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphecetche : this is to be able to change the timestamp at every map that we send, using the TF id.

auto dsec = d.Convert();
dsec += tfid;
d.Set(dsec);

std::string refDate = d.AsString();

auto GenerateVisitor = [refDate](const auto& t) {
return o2::dcs::generateRandomDataPoints({t.aliasPattern}, t.minValue, t.maxValue, refDate);
};

for (const auto& hint : hints) {
Expand All @@ -84,7 +103,8 @@ std::vector<o2::dcs::DataPointCompositeObject> generate(const std::vector<HintTy
}
if (fraction < 1.0) {
auto indices = generateIntegers(fraction * dataPoints.size(), 0, dataPoints.size() - 1);
auto tmp = dataPoints;
std::vector<o2::dcs::DataPointCompositeObject> tmp;
tmp.swap(dataPoints);
dataPoints.clear();
for (auto i : indices) {
dataPoints.push_back(tmp[i]);
Expand Down Expand Up @@ -114,11 +134,29 @@ class DCSRandomDataGenerator : public o2::framework::Task
mMaxCyclesNoFullMap = ic.options().get<int64_t>("max-cycles-no-full-map");

// create the list of DataPointHints to be used by the generator
mDataPointHints.emplace_back(DataPointHint<char>{"TestChar_0", 'A', 'z'});
// each detector should create his own when running the tests

/*mDataPointHints.emplace_back(DataPointHint<char>{"TestChar_0", 'A', 'z'});
mDataPointHints.emplace_back(DataPointHint<double>{"TestDouble_[0..3]", 0, 1700});
mDataPointHints.emplace_back(DataPointHint<int32_t>{"TestInt_[0..50000{:d}]", 0, 1234});
mDataPointHints.emplace_back(DataPointHint<bool>{"TestBool_[00..03]", 0, 1});
mDataPointHints.emplace_back(DataPointHint<std::string>{"TestString_0", "ABC", "ABCDEF"});
*/
// for TOF
// for test, we use less DPs that official ones
mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_vp_[00..02]", 0, 50.});
mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_vn_[00..02]", 0, 50.});
mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_ip_[00..02]", 0, 50.});
mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_in_[00..02]", 0, 50.});
mTOFDataPointHints.emplace_back(DataPointHint<int32_t>{"TOF_FEACSTATUS_[00..01]", 0, 255});
mTOFDataPointHints.emplace_back(DataPointHint<int32_t>{"TOF_HVSTATUS_SM[00..01]MOD[0..1]", 0, 524287});
// for TOF, official list
//mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_vp_[00..89]", 0, 50.});
//mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_vn_[00..89]", 0, 50.});
//mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_ip_[00..89]", 0, 50.});
//mTOFDataPointHints.emplace_back(DataPointHint<double>{"tof_hv_in_[00..89]", 0, 50.});
//mTOFDataPointHints.emplace_back(DataPointHint<int32_t>{"TOF_FEACSTATUS_[00..71]", 0, 255});
//mTOFDataPointHints.emplace_back(DataPointHint<int32_t>{"TOF_HVSTATUS_SM[00..17]MOD[0..4]", 0, 524287});
}

void run(o2::framework::ProcessingContext& pc) final
Expand All @@ -135,12 +173,13 @@ class DCSRandomDataGenerator : public o2::framework::Task
// fraction is one if we generate FBI (Full Buffer Image)
float fraction = (generateFBI ? 1.0 : mDeltaFraction);

auto dpcoms = generate(mDataPointHints, fraction);
TDatime d;
auto dpcoms = generate(mDataPointHints, fraction, tfid);
auto tofdpcoms = generate(mTOFDataPointHints, fraction, tfid);

// the output must always get both FBI and Delta, but one of them is empty.
std::vector<o2::dcs::DataPointCompositeObject> empty;
pc.outputs().snapshot(Output{"DCS", "DATAPOINTS", 0, Lifetime::Timeframe}, generateFBI ? dpcoms : empty);
pc.outputs().snapshot(Output{"DCS", "DATAPOINTSdelta", 0, Lifetime::Timeframe}, generateFBI ? empty : dpcoms);
LOG(INFO) << "***************** TF " << tfid << " has generated " << tofdpcoms.size() << " DPs for TOF";
Copy link
Collaborator Author

@chiarazampolli chiarazampolli Dec 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shahor02 , @aphecetche : DCS will send only 1 map at a time, either FBI or delta.

pc.outputs().snapshot(Output{"DCS", "DATAPOINTS", 0, Lifetime::Timeframe}, dpcoms);
pc.outputs().snapshot(Output{"DCS", "TOFDATAPOINTS", 0, Lifetime::Timeframe}, tofdpcoms);
mTFs++;
}

Expand All @@ -150,6 +189,7 @@ class DCSRandomDataGenerator : public o2::framework::Task
uint64_t mMaxCyclesNoFullMap;
float mDeltaFraction;
std::vector<HintType> mDataPointHints;
std::vector<HintType> mTOFDataPointHints;
};

} // namespace
Expand All @@ -159,7 +199,7 @@ DataProcessorSpec getDCSRandomDataGeneratorSpec()
return DataProcessorSpec{
"dcs-random-data-generator",
Inputs{},
Outputs{{{"outputDCS"}, "DCS", "DATAPOINTS"}, {{"outputDCSdelta"}, "DCS", "DATAPOINTSdelta"}},
Outputs{{{"outputDCS"}, "DCS", "DATAPOINTS"}, {{"outputDCSTOF"}, "DCS", "TOFDATAPOINTS"}},
AlgorithmSpec{adaptFromTask<DCSRandomDataGenerator>()},
Options{
{"max-timeframes", VariantType::Int64, 99999999999ll, {"max TimeFrames to generate"}},
Expand Down
35 changes: 35 additions & 0 deletions Detectors/DCS/testWorkflow/dcs-sim-workflow.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Copyright CERN and copyright holders of ALICE O2. This software is
// distributed under the terms of the GNU General Public License v3 (GPL
// Version 3), copied verbatim in the file "COPYING".
//
// See http://alice-o2.web.cern.ch/license for full licensing information.
//
// In applying this license CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.

#include "DetectorsDCS/DataPointIdentifier.h"
#include "DetectorsDCS/DataPointValue.h"
#include "Framework/TypeTraits.h"
#include <unordered_map>
#include "Framework/DataProcessorSpec.h"
#include "DCSRandomDataGeneratorSpec.h"

using namespace o2::framework;

// we need to add workflow options before including Framework/runDataProcessing
void customize(std::vector<o2::framework::ConfigParamSpec>& workflowOptions)
{
// option allowing to set parameters
}

// ------------------------------------------------------------------

#include "Framework/runDataProcessing.h"

WorkflowSpec defineDataProcessing(ConfigContext const& configcontext)
{
WorkflowSpec specs;
specs.emplace_back(getDCSRandomDataGeneratorSpec());
return specs;
}
1 change: 1 addition & 0 deletions Detectors/TOF/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@ add_subdirectory(reconstruction)
add_subdirectory(compression)
if(BUILD_TESTING)
add_subdirectory(prototyping)
add_subdirectory(calibration/macros)
endif()
add_subdirectory(workflow)
1 change: 1 addition & 0 deletions Detectors/TOF/base/include/TOFBase/Geo.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ class Geo
static void getPos(Int_t* det, Float_t* pos);
static void getVolumePath(const Int_t* ind, Char_t* path);
static Int_t getStripNumberPerSM(Int_t iplate, Int_t istrip);
static void getStripAndModule(Int_t iStripPerSM, Int_t& iplate, Int_t& istrip); // Return the module and strip per module corresponding to the strip number per SM

static Float_t getAngles(Int_t iplate, Int_t istrip) { return ANGLES[iplate][istrip]; }
static Float_t getHeights(Int_t iplate, Int_t istrip) { return HEIGHTS[iplate][istrip]; }
Expand Down
29 changes: 29 additions & 0 deletions Detectors/TOF/base/src/Geo.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -723,3 +723,32 @@ Int_t Geo::getIndexFromEquipment(Int_t icrate, Int_t islot, Int_t ichain, Int_t
{
return 0; // to be implemented
}

void Geo::getStripAndModule(Int_t iStripPerSM, Int_t& iplate, Int_t& istrip)
{
//
// Convert the serial number of the TOF strip number iStripPerSM [0,90]
// in module number iplate [0,4] and strip number istrip [0,14/18].
// Copied from AliRoot TOF::AliTOFGeometry
//

if (iStripPerSM < 0 || iStripPerSM >= NSTRIPC + NSTRIPB + NSTRIPA + NSTRIPB + NSTRIPC) {
iplate = -1;
istrip = -1;
} else if (iStripPerSM < NSTRIPC) {
iplate = 0;
istrip = iStripPerSM;
} else if (iStripPerSM >= NSTRIPC && iStripPerSM < NSTRIPC + NSTRIPB) {
iplate = 1;
istrip = iStripPerSM - NSTRIPC;
} else if (iStripPerSM >= NSTRIPC + NSTRIPB && iStripPerSM < NSTRIPC + NSTRIPB + NSTRIPA) {
iplate = 2;
istrip = iStripPerSM - NSTRIPC - NSTRIPB;
} else if (iStripPerSM >= NSTRIPC + NSTRIPB + NSTRIPA && iStripPerSM < NSTRIPC + NSTRIPB + NSTRIPA + NSTRIPB) {
iplate = 3;
istrip = iStripPerSM - NSTRIPC - NSTRIPB - NSTRIPA;
} else if (iStripPerSM >= NSTRIPC + NSTRIPB + NSTRIPA + NSTRIPB && iStripPerSM < NSTRIPC + NSTRIPB + NSTRIPA + NSTRIPB + NSTRIPC) {
iplate = 4;
istrip = iStripPerSM - NSTRIPC - NSTRIPB - NSTRIPA - NSTRIPB;
}
}
12 changes: 11 additions & 1 deletion Detectors/TOF/calibration/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@ o2_add_library(TOFCalibration
src/LHCClockCalibrator.cxx
src/TOFChannelCalibrator.cxx
src/TOFCalibCollector.cxx
src/TOFDCSProcessor.cxx
PUBLIC_LINK_LIBRARIES O2::DataFormatsTOF O2::TOFBase
O2::CCDB
O2::DetectorsCalibration
O2::DetectorsDCS
ROOT::Minuit
ms_gsl::ms_gsl)

Expand All @@ -28,7 +30,8 @@ o2_target_root_dictionary(TOFCalibration
include/TOFCalibration/LHCClockCalibrator.h
include/TOFCalibration/TOFChannelCalibrator.h
include/TOFCalibration/TOFCalibCollector.h
include/TOFCalibration/CollectCalibInfoTOF.h)
include/TOFCalibration/CollectCalibInfoTOF.h
include/TOFCalibration/TOFDCSProcessor.h)


o2_add_executable(data-generator-workflow
Expand Down Expand Up @@ -73,3 +76,10 @@ o2_add_executable(tof-collect-calib-workflow
PUBLIC_LINK_LIBRARIES O2::Framework
O2::TOFCalibration
O2::DetectorsCalibration)

o2_add_executable(tof-dcs-workflow
COMPONENT_NAME calibration
SOURCES testWorkflow/tof-dcs-data-workflow.cxx
PUBLIC_LINK_LIBRARIES O2::Framework
O2::TOFCalibration
O2::DetectorsDCS)
Loading