September 2017 Offline Leads Meeting Notes

The September Offline Leads ‘meeting’ was handled via email and a google document. We received reports from: Erica Snider, Thomas Junk, Robert Sulej, Andrzej Szelc and Wes Ketchum. We did not hear from: ICARUS or LArIAT.

LArSoft Report – Erica Snider

  1. Status of LArG4 – Hans has delivered a draft data product that will store energy depositions created by Geant4 that can then be used as input for ionization / scintillation modeling, electron and photon transport modeling, etc.  We are requesting your feedback on the item sent by Erica via email on September 8th.  Once we have agreement on this data product, we can use it as the basis for extracting the various LArTPC simulation effects into pluggable components to be used in the replacement of LArG4. Some coordination on the relevant interfaces will be needed, so please talk to us (Hans, Gianluca and Erica) before making changes.
  2. Status of SPACK migration. Jim Amundson has turned development over to Chris Jones and discussed the details on September 14th.
  3. Phase II track work – Phase 1 changes to recob::Track completed in June included added track fit information, introduced policies for trajectory points, etc. This provides the basis for a uniform definition of tracks from different algorithms, treatment of fts, etc. Phase 2 changes will introduce helpers to write tracks with all requires associations, a track “proxy” (previously called “facade”) that hides complications of finding associated hits and clusters. This will be extended to include a number of other associated data structures in Phase 3. Specification of the interfaces are being finalized and will be presented at a LArSoft Coordination meeting.
  4. A recob::SpacePoint re-design effort has been launched, motivated by the several active 3D reconstruction efforts. Goals:
    • Provide  a uniform interface between 3D SpacePoint finding and downstream PR
      • Include all information needed to store results of / drive 3D pattern recognition
      • Complements recent discussion with pixel-based L/GAr TPC efforts
        • Discussing how to integrate pixel reconstruction into LArSoft
        • Enabled by 3D efforts!  
      • Include charge (new!), covariance matrix, chisq etc, by implicit association
        • Via parallel vectors
        • just need to know the index to navigate to associated elements
      • Plan to use “proxy” concept to simplify finding associated data
      • Straw proposal under discussion at recent architecture meeting. A number of issues raised in that discussion related to these choices are under review (e.g. track fitting)
      • Will get to recob::Vertex soon

When setting priorities, would like to hear opinions on items that are in the accepted redmine issues, not assigned to see if they are a priority for next year, or if any of them should be rejected because the need is no longer there.

DUNE & ProtoDUNE – Thomas Junk, Robert Sulej, Andrew John Norman

Answer to documentation question: We will distribute the source code documentation guideline wiki page to our developers (both DUNE & ProtoDUNE) via email.

Current issue:  Robert Sulej is interested in using TensorFlow with his CNN in the LArSoft environment.  Robert is working with Lynn, Patrick, and Erica to get TensorFlow as a UPS product.  The problem is that TensorFlow is its own little world and has its own build system. Tensorflow based code for the CNN inference mode is up and running. The EM/track/Michel selection model is ready for it. Tensorflow UPS is still in preparations (the library size needs to be addressed by Lynn, need to turn on “core2” optimizations). Hope to have the full machinery ready by 1-2 weeks

Hans’s new data product for persisting G4 steps will help in the WireCell integration as well as the FLUKA interface.

Hans’s data product (actually refactorization of the G4 step) enables the factorization of the dual-phase detector simulation. However, due to the DUNE TDR timeline, we proceed with the simulations performed with the modified current codes (on a branch not planned to merge with LArSoft develop).

The feature ‘Global wire’ coordinate within LArSoft geometry, is important for both ProtoDUNE and DUNE. For the purpose of neutrino selection for DUNE TDR people are developing their own ways of imaging the events (on the hit level) contained in multiple TPC. We will need the same for ProtoDUNE at the level of ADC. The feature would really facilitate reconstruction, analysis codes and also event displays.

ICARUS – Daniele Gibin

No Report

LArIAT – Jonathan Asaadi

No Report

MicroBooNE – Herbert Greenlee, Wesley Robert Ketchum

Answer to documentation question: We will inform developers via email and meetings on the quidelines, and encourage that use for new code. We can start to work through modules in use in production to make sure it meets the guidelines. New modules going into production we can check to make sure this is included.

Hans data product looks good, with just one concern that the number of photons may be misleading should it be updated later. Perhaps “initial_photons” or something like that? Not sure.  Also trust that “G4double” is ok, or will be changed to C++ “double”.  We’ve identified Bill S. as someone from MicroBooNE who may be able to help implement further downstream changes working with the project. Encourage LArSoft project to reach out to him and formulate a plan.

MicroBooNE has recently tagged its reconstruction software release for the near-term, with a potential update planned for late fall/early winter that would be used for summer conferences. It’s currently anticipated we will need continued long-term support for the v06_26_01 branch of LArSoft, likely with potential changes to LArSoft releases near the end of the calendar year, and potentially urgent bug fixes after that point.

Tracy Usher is the new MicroBooNE Analysis Tools Co-convener, replacing Wes.

SBND – Roxanne Guenette, Andrzej Szelc

We have finally updated the geometry into a working state, and used it to launch a larger scale data production. Depending on the availability of Gustavo V., who first observed this problem, we might go back and test whether the issue is still there. We have a student who wants to start with the Geometry, this might be a good starting project for her as well.

We are now trying to understand storage to tape, given the size of our production and perhaps any data size reducing measures that MicroBooNE has developed.

Garcia-Gamez and V. Basque are testing the optical light simulation that allows using optical foils in the newest versions of LArSoft. In particular, they are testing the effects of changing the Rayleigh scattering length on the timing parametrizations that were developed – this is a work in progress currently.

Per Documentation we will send it out the SBND-software mailing list and discuss it at one of our Software/Physics meetings.

They are hosting a LArSoft tutorial for UK (and maybe European students) in Manchester a month from now similar to a self-organized one they did two years ago. Information is at: 

Open issues from previous meetings:

  1. From 3/22/17 meeting: Since SBND has been trying to include files inside GDML, have run into problems. Gianluca has been helping debug this. New version of ROOT may be a solution.
    • 4/18/17 update: The version of root used by LArSoft has changed a few times over the last few weeks.  Andrzej Szelc said they haven’t looked into this because they were focusing on getting the basic geometry in. Once that is done, will look at this and hope the new ROOT version may have fixed it. So, no ticket has been written yet (to ROOT or LArSoft) about this as waiting on whether it is still an issue.
    • 5/25/17 update – still waiting.
    • 6/14/17 update – Once SBND gets the latest version of ROOT in, they’ll see if this fixes the issue.
    • 7/27/17 update – SBND is still finalizing the new Geometry idea – expert moving. Once this is done, we have a student starting up that could follow up on whether the bugs are gone in the newest version of ROOT. In the meantime Gianluca with Vito’s help is putting sbndcode in the continuous integration framework – this will be very useful.
    • 8/16/17 – have basic geometry working. Still debugging before mounting full scale production.
    • 9/28/17 – SBND said the geometry is working and they’re using it, but they may go back and test if there is still an issue.

2. From May: Feedback at a recent SBN analysis meeting on proposed restructuring of G4 simulation step was that these were potentially very important changes. Is it possible to see all of this in place this summer for ICARUS large-scale processing and potentially MicroBooNE processing campaign?  Along with the work itself, how one updates/does the backtracking remains an unanswered question: not technically difficult, but could imply significant changes in downstream code based on how it is done. This may take another or a few more people working on it to really see it through.

    • The project and schedule are being tracked in issue 14454,
    • 8/16/17 update – Up until June, thought could get it done by mid-summer. The original plan was to incrementally replace parts of the LArG4 code with the updated Geant4 functionality and interfaces. But it looks like it is easier to do a full replacement. Since we changed the scope, we’re awaiting a good estimate on completion.
      • A question about the design arose, which was whether the ionization and scintillation modeling is downstream of the energy deposition, or embedded in the Geant4 stepping. ICARUS (and presumably other experiments) would like to be able to run ionization and scintillation, and all other downstream simulation in a separate job from the energy deposition simulation. It is not clear whether this is possible if the ionization and scintillation is done by Geant4. We (LArSoft) believe this will be possible with the re-factored code, but should be verified with Hans.  Will set up a meeting with Hans, Wes and Erica when he’s back to clarify design details and get a reasonable end-date or if additional resources are needed.
    • 9/15/17 update – Hans delivered a draft of the data product which we passed along to Offline Leads on September 8. Requesting feedback.
      • 9/28/17 update – MicroBooNE feedback: Hans data product looks good, with just one concern that the number of photons may be misleading should it be updated later. Perhaps “initial_photons” or something like that? Not sure.  Also trust that “G4double” is ok, or will be changed to C++ “double”.

3. From 6/14/17 meeting: MicroBooNE asked about how to give credit to authors, experiments and institutions for LArSoft code written.

  • 7/27/17 update
    1. Have updated the recommendation on documenting code at
    2. We committed to presenting a proposed documentation template for header files at a LArSoft Coordination Meeting.  (This was done on 6/27/17).
    3. LArSoft has a place to give author credit for algorithms and services on – as well as in the code itself. This is not being done enough, so suggest a multi-prong education campaign to encourage people to sign their name on the code they write and to contribute to the page. This will be highlighted in a LArSoft Coordinate Meeting with follow up but this needs to be done by all experiments, not just LArSoft.
    4. Presented proposal at June 27 Coordination Meeting. The action items from that discussion
      1. Add institution and experiment to the template
      2. Publish the template on a web page
      3. What are the plans from the experiments to push this practice out to the code?
        1. Recommend adding template documentation whenever code is updated?
        2. Make it the policy of the experiments
  • 8/10/17 – update – have institution and author in the template and published it at:
  • 9/14/17 update – asking each experiment to report on their plan for pushing this practice out to the code. Closing this issue.

Please email Katherine Lato or Erica Snider for any corrections or additions to these notes.