Repository navigation
Python 3 compatibility #3195
Description
A lot of Python code inside CloudStack is still Python 2 and we have just 10 months left before Python 2 is no longer supported: https://pythonclock.org/
Various items need to be looked at and checked for Python 3 compatibility. A list, but probably not complete:
- KVM Security Groups
- Xen Security Groups
- Virtual Router scripts (addressed in Update SystemVM scripts from python2 to python3 #3730)
- Various setup tools
- build tools (addressed in some minor changes to make the build run with python 3 #4030)
- marvin framework and smoke tests (Marvin3 #4727)
- marvin component test sets (component test ports/fixes in python3 #5082)
- packaging (cloud.spec) (Suggestions on Python processing of cloud.spec file in packaging directory #4513 is asking for it)
These are isolated projects and can usually be taken care of separately and also properly tested.
Creating this ticket so it can be referred to and also this is out there.
Activity
yadvr commentedon May 27, 2019
Is there an argument to say, these may be implemented purely in Java (if possible)? Currently, many of these scripts esp on KVM are forked via the cloudstack-agent and called.
For VR, I played with GRPC and I think it can be secured against the current CA-framework introduced in 4.11; here's a PoC: shapeblue#44
I'll propose more details on dev@ once I can put together design/idea details.
falcon78921 commentedon Jun 4, 2019
@rhtyd @wido I would love to help with moving over to Python 3, if that's still the plan.
wido commentedon Jun 5, 2019
Although I agree with @rhtyd that a lot can be done in Java, it's not always the easiest. Python is sometimes a lot easier to develop and talk directly with the system for executing commands.
We haven't started on this @falcon78921 , but any help is welcome! Just pick a Python script and start working on it.
yadvr commentedon Jun 5, 2019
@wido changing the env dependency is a big step, python3.x is not available on all distros by default. I think it's better to discuss on dev@ for future steps. In my proposal, for the VR for example I've proposed that we move to a lightweight agent and as PoC I've shared a Go-Java approach based on grpc.
I think the best time would be to discuss/start doing things after 4.13, as we've decided to deprecate CentOS6 after 4.13.
wido commentedon Jun 5, 2019
@rhtyd Python 3 is available on ALL distros. It has been for a very long time. It's on RHEL, CentOS, Debian and Ubuntu. No need to install anything :)
We can always talk about this, but the fact is that per January Python 2 is EOL and it's not so super difficult to write things in Python 3.
I'm not against an Agent in the VR for example, sounds like a good thing!
PaulAngus commentedon Jun 5, 2019
There are also knock-on effects, ie I suspect that the chances of running Python3 on XenServer 6.x is near zero.
wido commentedon Jun 5, 2019
@PaulAngus Python 2 is EOL in 2020, so any plans on what Xen is going to do?
But this doesn't stop us to fix other Python code before that time.
yadvr commentedon Jun 5, 2019
I think we'll need to deprecate all the XS 6.x, which we can discuss and aim to do in 4.14 and onwards. XS7 or xcp-ng 7.x with CentOS7 based dom0 may be more easy to support python3.
Porting network Python scripts from Python2 to Python3
Porting network Python scripts from Python2 to Python3
andrijapanicsb commentedon Jan 13, 2020
guys, any update on this one? I don't see it coming to 4.14 due to a needed effort?
9 remaining items
GabrielBrascher commentedon Sep 15, 2021
@nvazquez I need to roll a couple more tests.
Let's have it in the next release, for 4.16 it might be risky to have it.
Thanks!
DaanHoogland commentedon Sep 15, 2021
not to migrate but to fix. component tests have been neglected way too long.
DaanHoogland commentedon Oct 8, 2021
VR python scripts are not addressed and may not before 4.17. for now I'm keeping this open, just for the sake of the VR scripts.
DaanHoogland commentedon Jul 15, 2022
as there are no plans for doing this in 4.18 I'm moving this to 'unplanned'.
yadvr commentedon May 8, 2023
I think at some point our systemvmtemplate based OS (Debian 11) will EOL, so our runway may be limited by that. For this reason we may need to migrate the VR scripts to py3 or rework the VR agent approach.
Thanks @wido all, I think most of the items are done except migration of the VR scripts for which I've created a new ticket - #7500
Merge branch 'fix-42006' into '4.20.0.0-scclouds'