Skip to main content

2020-08-12 conda-forge core meeting

Zoom link What time is the meeting in my time zone

Attendees​

Agenda​

Standing items​

  • intros for new folks on the call
  • (CJ) budget

Your new agenda items​

  • (MRB) CFEP-13 and team updates are done

  • (CJ) What is the desired UX for abandoned feedstocks? How do we get there?

    • Our current UX is pretty bad, since there is a large disconnect between our users and the maintainance of the feedstocks. This means that many users may not be aware of the feedstock being unmaintained or may be unable to maintain the feedstock themselves. Many users may only find out there is a problem when we don't ship a critical fix or security patch.
    • My proposal is to verify the status of the feedstock and provide our best effort support while not consuming any additional time from our maintainers:
      1. add an issue asking if the feedstock is unmaintianed (if 3 bot version bump PRs haven't been addressed for a month)
      2. If the issue isn't addressed (closed and bot PRs merged/closed) in a timely manner (a month?) it is considered abandoned
      3. Remove the maintainers and add a caretaker unmaintained team. This team's only job is to merge any prs that add maintainers (and remove themselves)
      4. Add a line at the top of the Readme stating that the feedstock is unmaintained and any new maintainers are welcome/encouraged/needed
      5. Add automerge and any other automation that is available at the time (eg. depedency updating)
      6. We can make carve outs for systemicly important packages (eg ruamel.yaml)
    • Need to provide feedback on install to state that the feedstock is unmaintained
    • Add no-maintainer feedstocks (for those who haven't accepted invitations)
    • Don't remove maintainers
    • CVEs? What do we do when a new CVE crops up on a feedstock that is unmaintained? Can we generate a list of these things?
    • Provide
    • Automerging?
      • R is fine with tightly managed metadata
      • Python, on the other hand, is a mess for dependency resolution. If we add grayskull auto-updating of metadata then most of the concerns around auto-merging are gone.
      • What about other languages?
    • What about packages that are abandoned upstream?
    • Feedstocks with no maintainers is another good place to start
    • https://github.com/conda-forge/cfep/pull/15
    • TODO: Capture the overarching strategy on a hackmd.
      1. Push through the CFEP on identifying what unmaintained means
      2. ???

  • (FF) Pay for Heroku via a NumFOCUS card that will make a direct debit on our funds. NumFOCUS (Leah) is also getting in touch with Heroku to see if they can get us some special free stuff.

  • (FF) AWS credits: we have 1k and we may get more. We must approve 2 plans, one for what we have and a second one with the extrars that I will ask Andy if we can execute too.

  • (IF) Windows server from AWS. This would make debugging windows recipes much easier than debugging on the CI servers. Pricing at https://aws.amazon.com/workspaces/pricing/?nc=sn&loc=3

  • AWS results

    • We are going to make one big request for a lot since that is what is useful.
    • Going to ask for a lot and let them knock it down.
    • Total will be 1600 + cost of windows server
    • use this server:
      • 8 vCPU, 32 GB Memory 80 GB 50 GB $130.00 $9.75/month + $1.53/hour

  • (IF) macOS arm is under way.

    • Currently blocked by CDN not supporting osx-arm64 downloads
    • Started building python dependencies.
      • zlib - had to guard make check
      • bzip2/libffi - worked fine (turned off test_on_native_only on libffi because tests are only existence tests)
      • xz/ncurses - had to run autoreconf to get new config.sub and config.guess
      • ncurses - had to set BUILD_CC instead of standard CC_FOR_BUILD. (We should probably set that as well)
      • ncurses - Need ncurses from build. https://github.com/conda/conda-build/pull/4011
    • Mini-migrators for some tasks above
      • Guard make check with a conditional on CONDA_BUILD_CROSS_COMPILATION env variable.
      • Change cmake . to cmake ${CMAKE_ARGS} .
    • macOS Arm migrator improvements
      • Determine if the source tarball has config.sub and config.guess and if so replace them with new ones from libtool.
      • If the tests are only existence checks like test -f, turn off test_on_native_only.

  • (CHL) conda 4.8.4 released to "defaults" 2020-08-12; conda-build release coming in next week or two.

  • (AS) qgpu - GPU build agents.

    • Drone or Azure? Drone is a simple go executable and you can run it in docker. Azure build agent is heavy weight?
    • Pick one and go

Stuff from last week that we didn't get to​

Who is taking these action items?

Active votes​

Subteam updates​

Bot​

  • Bot now closes PRs with conflicts if it is the only committer
  • Bot had an outage on Monday but should be resolved now

ARM​

POWER​

CUDA​

Docs​

staged-recipes​

website​

security+systems​

See above

CI infrastructure​

Compiler upgrade​

CFEP updates​

Open PRs​

  • cfep-04 X11 and CDT policy

    • INACTIVE - Merge in with some inactive-esque status?
    • Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020

  • cfep-06 Staged-recipes review lifecycle

    • INACTIVE - Merge in with some inactive-esque status?
    • Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020
    • (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently.

  • cfep-10 Feedstock statuses, unmaintained

    • INACTIVE - Merge in with some inactive-esque status?
    • Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020

  • cfep-12 Removing packages that violate the terms of the source package

    • Stalled since May 26, 2020
    • Active debate about moving to "broken" vs deleting from conda-forge channel
    • Active vote, ends on 2020-03-11
    • What were the results of the vote?
    • Did we hear back from NumFOCUS?

  • cfep-17 Handling pin backports and dependency rebuilds

    • Stalled debate about implementation details between Isuru, CJ and Matt
    • UPDATE 2020-07-22: We in principle have agreement to render the extra pinnings needed directly in the feedstock on a temporary basis (i.e., until the migration has ended).

Discussion​

Check in on previous action items​

Copy previous action items from last meeting agenda.

This meeting​

Last meeting​

  • Figure out how to communicate breaking changes to users. Likely should open up an issue immediately for futher discussion. Ping @kkraus, plus capture notes from further up in these meeting notes

  • (Eric) TODO: Make strict an option in conda_forge.yaml and turn it on by default. Open issue in conda-smithy

2 meetings ago​

  • Eric to add a new page to our docs around how to engage with conda-forge and affiliated in a commercial relationship.
  • Eric will get the NVBug link from Keith and archive it in the conda-forge google drive.
  • John K. will update the cuda toolkit feedstock on the git repo to note the NVBug link to the internal NVIDIA issue tracker
  • Jonathan will update docs to note that some non-exhaustive list of packages (like cuda-toolkit, MKL, etc.)
  • Jonathan will review this PR

3 meetings ago​

Move to Issue Tracker​

  • (Kale) schedule conda working group
  • cfep-10 next steps: CJ to call a vote for feedback
  • cfep-06 next steps: Ask staged recipes team to champion this CFEP and move it forward
  • jakirkham & CJ-wright to sync on adding CUDA to the migration bot
  • (Eric) Scheduling Anaconda <-> conda-forge sync on anaconda.org requirements gathering
    • Will try and get this scheduled in the next month.
  • (Anthony) Reach out to NumFocus to figure out legal ramifications of not including licenses in files.
  • (Eric) check internally for funding levels for hotels & flying folks from the community in?
  • (Eric) Figure out finances of conda-forge to support themselves?
  • (jjhelmus) Open up CFEP for which python's we're going to support
  • (jakirkham) write a blog post on CUDA stuff we discussed today
  • (jakirkham) update docs on how to add CUDA support to feedstocks
  • (jakirkham) will open an issue on conda-smithy to investigate Drone issues. (ping the aarch team)
  • (ED) Who we are page? Some combination of a FAQ and a who is everyone. FAQ things like:
    • who's the POC for CF <> Anaconda, CF <> NumFocus, CF <> Azure
    • who's the POC for the various subteams?
    • Informal information: roles, day jobs, bios, the whole nine yards, why you're here, etc.
    • Public or internal? I don't really care either way. Anyone feel strongly one way or the other?
    • opt-in to public bios
    • software carpentry has a large number of instructors and has https://carpentries.org/instructors
    • some concern about "yet another place to keep stuff up to date"
  • (CJ) Form finance subteam
  • (ED) document strategies for reproducible environments using conda-forge
  • (UK) Static libraries stuff
    • Add linting hints to builds to find them
    • Recommend how to package them -> CFEP-18
    • We should write docs saying we don't provide support and this is a bad idea. -> CFEP-18