January 2020 Offline Leads Meeting Notes

leads

The January 2020 LArSoft Offline Leads status update was handled via a live meeting on January 22  with attendees: Tracy Usher, Andrzej Szelc, Herb Greenlee, Erica Snider, Katherine Lato

  1. GitHub migration – 
    • Have started the migration. 
    • People who have said they will be Level 2 managers (from the experiments) will be in that role. The GitHub side of things is working properly with our scripts. 
    • We expect that there will be test failures. 
      • It’s up to the L2 managers to resolve these if related to problems with their experiment. If it’s some other experiment, everyone will be notified and can work with the author to try and resolve it. 
      • L2 managers should not approve changes that break other experiments before consulting with those experiments. LArSoft will mediate as best we can. We’ll be monitoring the system closely.
    • Feature branches cannot be pushed to the central repositories. They need to be shared out of someone’s private area  or from a GitHub organization if multiple people are using it.
    • Currently trailing white space causes a failure during “code checks”. 
      • We have a script that strips trailing white space that will fix this. Usually we recommend committing cosmetic changes separately from other changes.
      • The script is not context sensitive, so there is a chance that in some cases, using it will change the semantics
    • Herb Greenlee – we have production branches. Do we make our own fork? 
      • They will be migrated. All branches that have names that start with “v” have been migrated. But yes, then you need to make your own fork. 
    • Tracy – SBND and ICARUS have everything mirrored to GitHub already. Just waiting on LArSoft’s migration.
  2. Licensing and copyright 
    • We’ve wanted to do this for some time, and with moving to GitHub it seems like this is a good time. 
    • We discussed licensing issues with Aaron Sauers of the Office of Partnership and Technology Transfer. He has advised that we can model a copyright on that used by CERN for CMS code, which states, “Copyright CERN for the benefit of the CMS Collaboration”, with appropriate references. He also confirmed that we can license a portion of LArSoft under GPL v3 (the Pandora interface packages that depend explicitly on GPL v3-licensed Pandora SDK), and the balance of LArSoft under Apache v2, our preferred choice, since the balance of the code does not depend in any way on the Pandora interface components. We plan to move forward with licensing under this guidance.  Wording will be ‘Copyright Fermilab for the benefit of experiments in the LArSoft Collaboration.’
    • Only Pandora was identified as a problem with licenses by the Technology Transfer people. 
    • Apache v2 is in a class that is least restrictive. 
    • GPL v3 forces you to make all the code that touches it to be GPL v3. There’s no reason why someone that touches LArSoft code should be forced to be GPL v3. 
  3. Weekly releases vs some alternative 
      • The intent of weekly releases was to provide stability for people developing code. Because the dependency versioning is tightly coupled to the source code, experiments are forced to move their repositories forward as integration releases appear.
      • Weekly releases creates a burden. (Not just on LArSoft, but on the experiments as well to keep up.) 
      • Are the weekly releases valuable? Or will some other model work as well or better? We’d like to have a discussion about this. 
        • Andrzej would be fine with them less often, but will check. 
        • Herb thinks could have less, but not just on-demand, since scheduled releases are good. 
        • Tracy agrees with Herb. Maybe go to having them every other week, the same week at the LArSoft Coordination meetings.
  4. Round-table
      • Andrzej – SBND
        • Doing work parallelizing. 
          • Getting code running to run on HPC computers, the Theta supercluster at Argonne. They have an allocation there.
          • Also running some Director’s Allocation for MicroBooNE stuff. Ran Genie and Corsika- had to change how random numbers are generated, per event, not per job. 
          • They can run LArSoft out of containers. Have run a reconstruction job. Ran into problem with database web servers. The problem might be on the Argonne side. They don’t allow enough connections to go out to connect to the web servers, not enough slots to go out? Tried to downgrade the number of connections. This caching feature of web servers doesn’t happen since everyone is providing timestamps in nanoseconds so each event has to get its own data rather than sharing. Tried various things, such as down-sampling timestamps. Got it working with a new version that reads a cache file locally, but the feature branch is now two versions higher than the production version.
      • Tracy – ICARUS
        • Still pushing forward to the idea that ICARUS will be taking data soon. Had a mini-collaboration meeting in mid-January. Expect the end of April before ready to turn on high voltage. Getting the  first data out of DAQ system the week of Jan 20th. Starting to show stress points, even reading only 4608 out of 53,000 channels. Bulk of focus is converting data to LArSoft format. Starting to see the first issues, so getting a good head start. 
        • LArSoft Event display isn’t going to be up to 53,000 channels in ICARUS. So far, they only write out (in MC) the channels that have signal. With larger sets, it’s been clear that running it over the network, it doesn’t work. People have given up on that. Over winter break, worked on Event Display based on QT with Gianluca Petrillo providing an interface to services within a gallery environment. So no longer need to hardcode the geometry. Also enables use with other detectors. Marco Del Tutto  has been working on that. Corey Adams and Marco have named their event display “TITUS”. They have released version 1.0 and it has been shown to work with all three of SBND, ICARUS and MicroBooNE detectors. It is not integrated into LArSoft/art proper but, rather, is integrated into gallery. Probably one could change this if desired. 
        •  Umut Kose has continued to develop an Eve based event display. This seems more oriented at the 3D side of things.
      • Herb – MicroBooNE
        • We are migrating to python 3. There is a “2to3” script that does most of it. Plan on running that on everything. It doesn’t make things backwards compatible — it’s just a straight conversion. He’s trying to make things compatible with 2 and 3 for some stuff. LArCV and LArLite need to work for both python 2 and 3 due to analyses working from existing production branches. This conversion is just for integration releases. 
        • Question:  Noted that python 2 won’t be supported after LArSoft migrates to art 3.05. Is it possible that a security issue with python 2 could arise that would make python 2 unusable, and that would not get patched (which would be a problem for the analyses noted above)? Yes, this is possible, but seems unlikely. Otherwise, things that require Python 2 don’t require a lot of maintenance.

Please email Katherine Lato or Erica Snider for any corrections or additions to these notes.