August 2020 Offline Leads Meeting Notes

leads

Attendees: Tom Junk (DUNE), Tracy Usher (ICARUS), Erica Snider, Katherine Lato

LArSoft

  • 2021 Workplan – we will be contacting each experiment about their plans for 2021
    • We expect that the major focus will continue to be modernizing the code to run multi-threaded, or to run in multi-threaded environments. Will also continue to facilitate work toward using LArSoft on HPC resources and platforms as they become available.
    • Deep learning
      • Expect that interest in using DL for LAr reconstruction will continue to grow. Already an effort in MicroBooNE to use DL to perform vertex reconstruction, a significant addition to the DL reconstruction chain.
      • LArSoft will want to facilitate using these methodologies within art / LArSoft jobs. At the very least, image translation for input to DL algorithms, and translation code to port the results back into art / LArSoft.
    • Ideas welcome.

DUNE

  • Asked DUNE whether the FD requires similar photon visibility features to that of ICARUS. They need to have visibility services understand that light can travel between TPCs, and through a semi-transparent cathode plane. Does DUNE have to deal with this as well?
    • Tom: If you want light to go from one TPC to the next, that is a huge volume. Therefore, a parameterized solution would probably work better that a voxelized lookup library in general.
    • The field cages have holes in them, so can admit light from outside the sensitive volume. Probably not a serious concern.
    • The photo-detectors are in the APAs, and the cathode planes are opaque and non-reflective, so even though APAs are transparent, light generated on one side will not be visible to other TPCs on the other side of the APA plane. Light can travel across TPC boundaries vertically, and horizontally within the confines of the enclosed APA and CPA planes.
    • Visibility parameterization will need to understand this detail of the geometry.
  • Discussed the need to integrate pixels and alter the geometry to accommodate them. This is relevant to the ND, and to some level, the somewhat strained way of handling the strip readout in dual-phase detectors.
    • Tom mentioned a student, or postdoc, who is working on their own Python ND simulation, doing diffusion and re-inventing other effects by themselves, giving the output for deep learning. Does not use Geant4, so is missing a lot of physics.
    • Also commented that several groups are working on alternatives to the gas detector. Everyone doing their own thing. 
      • Talking about using edep-sim to join the effort of several groups. This is an interface to Geant4 originally used by a non-liquid argon detector. It can handle the entire detector complex, in principle, so people have been making GDML to describe it all. At present, LArSoft can’t do that.
      • Not clear the experiment wants to build the GAr detector. May just get a stack of magnetized iron/scintillator. A MINERvA-like scintillator detector is also on the table. At the last review the MPD wasn’t even on the agenda. Only talked about an iron range-stack. Looking for international contributions for a GAr detector. Thinking about reconfiguring on day 1 so it is scintillator instead of gas to start, then change later. Things are up in the air. 
      • People are using simpler software chains for all of this work — not LArSoft, not art. They have little time and are afraid that the things they are working on might not pan out, so don’t want to make large investments (eg, in LArSoft). One drawback to this approach is that people may encounter a limitation that requires a huge effort that they aren’t able to tackle. Yet, this limitation may have already been addressed in LArSoft.
      • DUNE ND people include Clark McGrew who is the owner of edep-sim: https://github.com/ClarkMcGrew/edep-sim. The DUNE near detector software integration leader is Mat Muether (Wichita State). The DUNE ECAL/GEANT/Software expert is Eldwan Brianne (DESY). ND-LAr software person is  Andy Mastbaum.  Kazu Terao (SLAC) is on reconstruction.
    • DUNE still requests that pixels be integrated in LArSoft, since the pixel LArTPC is baselined for the ND. 
    • Noted that if you don’t have the gas in the detector,  not doing the physics that the gas lets us do.

ICARUS 

  • Two methods for doing light. 
    • Photon library. (Look up table). Effective, known solution, but large
    • Semi-analytic. Smaller, but then need to deal with details of handling optically non-trivial boundaries between TPCs, starting with the semi-transparent cathode plane
    • Still relying on photon library lookup method now. Gianluca has all the pieces to take into account the effects that weren’t in the first one (photons crossing the cathode plane). 
    • There is the SBN PDS group with coveners Diego Garcia and Alessandro Menenegolli. The group has as a goal to have the experiments use the same approach. Not only the cathode effect, but also being on the other side of the PMTs and outside the field cage, which they think will be an issue. These pose serious (but solvable)  problems for the semi-analytic approach. 
    • They are setting up to generate a new photon library. They are working on the semi-analytic approach, but not working full time on it. They are learning how to do this.
    • The experiment needs someone to generate the photon library with all the changes that Gianluca has made. Should happen in a month or so.
    • Diego is pushing that. Alexandra is co-creator of the group. They have a program that they laid out to compare the programs side-by-side. 
  • ICARUS started taking data. 
    • Already at full-field. Ramped up to 50% voltage a week ago. All with zero bias voltages, so the first induction plan is acting as a collection plane. On Tuesday, went up to full field. When they went to 50%, saw tracks. Had a few mapping issues, sorted them out before going to full field. 
    • The events are amazing to look at for having just turned on. Things are looking good. Will turn off on Friday since there is a power outage. Then they are going to turn on again next week. Will see first full events with bias voltages set. Happy to see how quickly it turned on and are producing. Work with MicroBooNE helped. 
  • One other thing that they haven’t been working on because of taking data is the database. 
    • Aside:  Tracy was initially unable to read from it using his laptop for first data. Turned out to be a slow internet connection. It was trying to push the data faster than he could take it, so it gave up. Fermilab fixed some parameters and he no longer has problems. They were very responsive to his initial report, and quickly fixed it.
    • Tracy anticipates problems with DB scaling on the grid, particularly when launching a lot of jobs at once.
    • MicroBooNE observed a DB scaling problem on the grid, particularly when launching a lot of jobs. They solved it by introducing an SQLite back end to the relevant services, and exporting those files with the grid jobs. That solved their problem. 
    • Tracy anticipates that ICARUS will have the same problem because they have 8x more channels. Wants to avoid hitting the same issue by adopting the MicroBooNE solution, written by Herb Greenlee.
    • Herb’s solution is sitting on a feature branch. ICARUS would like that integrated into larevt.
      • Erica will check on the status of this code and see if we can start the process of integrating into LArSoft.
      • Noted that this places an operational burden on the production staff, by requiring that the right files get transported with the jobs. Herb might (or might not) have helpful scripts. Will inquire.
  • Inquired as to status of v09. Need this to run their production in multi-threaded mode.
    • Plan was for this week. Right  now just working through merge conflicts in experiment code.

Please email Katherine Lato or Erica Snider for any corrections or additions to these notes.