Repository navigation
Survey: Frontend Language Support #625
Description
Dear @openPMD/general-contributors,
this issue is pinned to keep track of your needs for our reference API in terms of minimum supported language features.
So far we support Python 3.6+ and C++14 or newer. But as compilers and languages progress, we would love to drop old language revisions in newer releases to keep maintenance at a minimum and make use of newer language features in public interfaces. Please vote in the following comments according to the minimum requirements you have in your downstream code-bases. This will help us to plan potentially breaking changes with minimum hazzle.
Voting in the C++ and Python frontend will help us to ping and interact with you in case we would like to drop, e.g. C++14 bindings (for C++17 and newer) or older Python bindings in the future.
Voting on new language bindings does not implicate someone will find the time to provide such implementations, but it potentially motivates volunteers to step forward and add further bindings or share their own implementations where needed.
Activity
ax3l commentedon Dec 9, 2019
C++ Frontend
As of today, I (only) need support for
You can update your vote at any time, e.g., if your code-base updated.
Note: C++ compiler defaults and C++ compiler support.
ax3l commentedon Dec 9, 2019
Python Frontend
As of today, I (only) need support for
You can update your vote at any time, e.g., if your code-base updated.
ax3l commentedon Dec 9, 2019
C Frontend
We do not yet provide a C-frontend. If one would spend time developing one, I (only) need support for
You can update your vote at any time, e.g., if your code-base updated.
Note: C compiler defaults.
Potential library usage: shroud
ax3l commentedon Dec 9, 2019
Fortran Frontend
Currently, there are at least two implementations of a HDF5-Fortran openPMD library [ref1] which we hightly recommend to check out and use.
We have no plans yet to do so, but just to mention the technical possibilities: one could in principle, based on implementing a C-Frontend first, use the former to implement Fortran bindings herin as well. This would widen the supported backends to include, e.g., ADIOS1 and ADIOS2 for all Fortran users. Alternatively, one could also add these backends via their native Fortran bindings to the aforementioned libraries.
If one would add Fortran bindings to openPMD-api, I (only) need support for
Note: Fortran bindings prior to the 2008 standard are extremely cumbersome (read as: not worth spending time developing), since iso-C-bindings were first standardized in version 2008.
Potential library usage: shroud
You can update your vote at any time, e.g., if your code-base updated.
ax3l commentedon Dec 9, 2019
More Exotic Bindings
It's in principle possible to add bindings for further (modern) languages in case there is a sufficiently large user-base for it. Just tell us what you currently use in production workflows, maybe someone wants to team up with you and build bindings!
I use the following languages in my workflows and I would benefit from adding bindings to
(Multiple answers are welcome, too.)
mccoys commentedon Dec 9, 2019
I know I should use the front-end already ... Will do when I have time. I envision only c++11 & py3.5+ for now
ax3l commentedon Dec 9, 2019
Sounds great, feel free to ping us if you need help with anything when adopting to the API (reviews, more examples, etc.). Since Python 3.5's EOL is in Q3 2020 I do not expect any drop of our supported versions soon until it runs into this.
Conda-Forge already transitioned to ship Python 3.6+ only, but that's just one of our many release channels.
ax3l commentedon Jun 5, 2020
Announcement: since most user-level distributions already dropped Python 3.5 and it will not receive any updates after September 2019, we anticipate to drop Python 3.5 support in upcoming releases.
This approach is on par with other major projects such as Numpy dropping Python 3.5 in upcoming releases (1.19.0+). We continue to support Python 3.6-3.8 and anticipate addition of Python 3.9 after its October release.
Please see: https://devguide.python.org/#status-of-python-branches
Update: Python 3.5 dropped and 3.9 added in #828 (0.13.0 release). We already ship Python 3.9 via conda-forge for 0.12.0.
williamfgc commentedon Jun 5, 2020
I see a label for Julia, are you guys invested on it? I'm currently evaluating its use and seems very promising now that it's been stable for a year.
ax3l commentedon Jun 6, 2020
Unfortunately not, I know exactly one person in our community that does Julia @berceanu
bussmann commentedon Jun 8, 2020
Hi William, yes, Julia support would be lovely. As ax3l pointed out, community-wise the interest is unfortunately limited, so the manpower is as well. Julia is IMHO quite o.k., but the parallelism does not come out of the box as promised (it never did). Why would you got for Julia? What would be the features you are missing with Python?
williamfgc commentedon Jun 8, 2020
@ax3l @bussmann thanks for chiming in. Python is great, nothing is missing with the added ecosystem (numpy, scipy, matplotlib, etc.), still dependency versions must be tracked when packaging with conda, pip, etc.. We are mostly exploring Julia for the long run (still young in 2020, I just hit a wall myself with IDE support) as it presents interesting concepts built into the language (type declarations, the parallelism you mentioned, math) targeting the kind of work we do in their roadmap. I just got mostly curious as I saw the label in this issue and I wanted to learn more about your view/experiences. Thanks.
[-]Vote: Frontend Language Support[/-][+]Survey: Frontend Language Support[/+]ax3l commentedon Sep 30, 2020
With our main downstream consumers now upgraded to C++14 code bases, we'll soon migrate to C++14, too. This will simplify the maintenance of the library and we can stop testing some ancient compilers that only came with now deprecated operating systems.
Many compilers already switched to build with C++14 by default and some already set C++17 as their default compilation language. C++ is generally forward-compatible and we make sure in our CI that our C++ standard (e.g. 14) requirement will built with C++17 and 20 flags, too. We will generally aim to continue to adopt new C++ standards, as soon as vendor compilers and HPC systems provide stable support for them; and thus once our main downstream users can make the switch.
Update: transition to C++14 and newer will happen with the 0.13.0 release #825
eschnett commentedon May 19, 2021
@axl3 Incidentally, I started working on a Julia package
openPMD_api_jlltoday.eschnett commentedon May 25, 2021
C++: C++17
Python, C, Fortran: n/a
Julia: yes please!
ax3l commentedon May 25, 2021
Awesome, thanks for the update & contributions @eschnett! :)
ax3l commentedon Dec 17, 2021
C++ Frontend Announcement:
With WarpX and PIConGPU both upgrading to C++17, we will likely transition to require C++17 #1164 in openPMD-api with release
0.15and newer 🎉ax3l commentedon Dec 20, 2021
Python Frontend Announcement:
Since Python 3.6 is EOL in three days, the last release line that will support it is
0.14.*. It's likely that major user-level package managers, like conda-forge, and release support software, like cibuildwheel, will drop support soon after (to make room in build resources for newer releases, like Python 3.10).https://endoflife.date/python
ax3l commentedon Dec 21, 2021
@skuschel do you still use Python 3.6 in production? Just checking in, feel free to update your current needs here by updating your reaction emoji :)
berceanu commentedon Sep 28, 2023
A short update regarding
Julia. While the maturity and diversity of the ecosystem still lags somewhat behind that ofPython, I feel that the community has progressed a lot in recent years, especially on issues related to parallelism and GPU kernel generation. Here are two recent comparative papers:Julia as a unifying end-to-end workflow language on the Frontier exascale system
Evaluating performance and portability of high-level programming models: Julia, Python/Numba, and Kokkos on exascale nodes
Probably @williamfgc has more details on this, as first author on both studies
DavidSagan commentedon Sep 28, 2023
@williamfgc Note: There is a Bmad (https://www.classe.cornell.edu/bmad/) port to Julia project in development and part of this project will be a Julia front end for the EXT_BeamPhysics.md extension.