Repository navigation
[DEPR] Devstack #247
Description
Timeline
-
Proposed: 2022-03-01
-
Updated Acceptance Date: 2024-03-12
-
Archived: 2024-03-19 (pre-Redwood)
- Anyone wishing to continue to use devstack will have to make their own forks in order to make changes or keep it updated/maintained.
- 2U will likely maintain its own fork, but there will be a number of 2U-specific changes and will not be open to outside development. It is possible that this fork will be made private.
-
Removal of devstack support from application repositories:
Original deadline: 2024-10-10- New deadline: 30 days after lms/envs/development.py and cms/envs/development.py are merged into edx-platform master.
- Estimate: development.py will merge 2024-10-20, old devstack.py will be removed on 2024-11-20.
Replacement
The replacement for Devstack is Tutor. Tutor is "the Docker-based Open edX distribution designed for peace of mind". It is both a deployment tool and a development environment. Tutor has been the only community-support production deployment method since Maple, although the Open edX docs and website do not officially recommend it as a development environment yet.
Rationale
Devstack has several issues:
- It is not pluggable. This leads to two issues:
- It is large. Out of the box, it comes with a dozen IDAs and another dozen MFEs; new Open edX developers generally needs two IDAs (LMS and CMS) a few MFEs. In the past couple years, improvements have been made such that only a subset of IDAs are run by default. Still, cloning and provisioning are bound by the full weight of all services.
- Some IDAs like enterprise-catalog, in order to avoid being part of the large Devstack core, make a plugin-esque docker-compose.yml file that hooks into Devstack. This complicates the developer experience on those IDAs and also makes it harder to modify Devstack itself (because of the implicit plugin interface that such IDAs are using).
- Its images are slow to download. The Docker images used by devstack are built by running Ansible in a Dockerfile instead of using native Dockerfile commands. This means that the images are large and make poor use of the Docker layer cache, and thus take a long time for Devstack users to download.
- Many users find it difficult to diagnose and fix Devstack errors. Often, they find it more time/effort-efficient to destroy their volumes and provision from scratch instead of fixing the immediate issue.
- Provisioning takes a long time. This exacerbates the previous issue.
- There are several "upkeep" steps that must be independently done on a regular basis in order to keep Devstack working reliably. These include git-pulling devstack, git-pulling multiple app repos, pulling Docker images, and running migrations.
- Setting up a service in Devstack is completely tangential to setting it up for production deployment. Once it is set up, it takes attention and effort to keep the setup consistent between Devstack and production.
Tutor shows promise in many of these areas:
- It features a documented plugin API. Many IDAs (forums, discovery, credentials, et al) are implemented as plugins to Tutor. The core of Tutor includes just LMS and CMS, and there is even work being discussed to factor those out into plugins.
- Tutor's Dockerfiles are written using native Dockerfile syntax and are intentionally designed to be as small and cache-friendly as reasonably possible.
- Tutor has actively-maintained documentation including a Troubleshooting section. Beyond that, Tutor has a dedicated forum community where maintainers and users help each other. Tutor also makes a point to have a consistent, documented, and debuggable CLI. For example, the
tutor devsegment of the CLI is implemented entirely on top ofdocker-composeand transparently prints out the underlyingdocker-composecommands that it runs to aid in user debugging, using terminal colors to call out commands vs tutor logging vs actual output. - Once Tutor is installed, provisioning involves one command (
tutor local quickstart) and runs fairly quickly. Provisioning can be safely re-run without destroying existing data. - Because Tutor uses the code in the Docker images by default, the user only needs to regularly git-pull the Tutor repo and the repos that they host-mount in, if any.
- Since Tutor is also the community-supported deployment method, one can support development and and production usage of an IDA all in a single plugin. Furthermore, using Tutor for both dev and prod allows operators to keep their dev and prod environments very similar with less effort.
For areas where Tutor could be improved, there is an ongoing Tutor Adoption Initiative which aims to support the transition from Devstack to Tutor through a mix of education, plugin development, and core Tutor improvements. Within the initiative roadmap, the "Dev Quality-of-Life" epic contains stories that should ideally close any existing gaps between Tutor and Devstack.
Migration
Many developers can start using Tutor now, and many already are). Particularly, developers whose projects are encompassed by the LMS, CMS, MFEs, and IDAs covered by existing official plugins can generally transition their workflows over to Tutor. On the other hand, for teams that run IDAs with less community adoption (notably, teams at 2U), several new Tutor plugins will be needed in order for Tutor to fully replace Devstack.
There are no plans to help users migrate data from a Devstack instance to a Tutor instance. We recommend that folks start fresh when switching from Devstack to Tutor.
Deprecation & Removal
Deprecation steps
- Update the development page on open.edx.org to recommend Tutor instead of Devstack.
Ctrl+Fthe docs for "Devstack", and where appropriate, update them to point at Tutor.- Add a deprecation warning to devstack's readme
- Make devstack's
dev.upMakefile target print a deprecation warning - Update the configuration docker role's readme with a deprecation message.
- In the description of https://dockerhub.com/u/edxops, warn that all images in the org are deprecated.
Removal steps
- Archive https://github.com/openedx/devstack
- Update OEP-10, which still mentions "devstack" as a supported distro
- Update dev setup in README openedx-platform#34430
- Remove env.devstack files from frontend apps
- Scour our docs for other references to devstack; update them to Tutor.
- Delete the various
devstack.pyanddevelopment.pysettings files unless otherwise depended upon. - Delete the various
docker-compose.ymlfiles and Makefile targets that some IDAs (eg enterprise-catalog) use in order to integrate with Devstack. - Delete the
docker/role (incl. Dockerfiles) from github.com/openedx/configuration. - Follow up: Decide whether or not to keep the Dockerfiles that exist in various repos, which aren't used in Tutor.
- Update edx-cookiecutters
Activity
[-][DEPR]: Devstack[/-][+][DEPR] Devstack[/+]kdmccormick commentedon Feb 28, 2022
Relates to openedx/wg-build-test-release#138
dianakhuang commentedon Jul 13, 2023
Since this is primarily a 2U project right now, we are planning on announcing this and gathering feedback.
2U is still planning on using this for the near to medium term until we're confident we can migrate successfully onto Tutor, but we can move it out of the openedx GitHub org to minimize confusion.
kdmccormick commentedon Jul 13, 2023
In general, sounds great!
Only thing is... MFE development currently sucks in Tutor >.< So nearly all frontend devs still use Devstack when they're trying to change MFEs.
Brian Smith is actively working on rectifying this, with promising results. Would you be willing to delay the deprecation further until Tutor has MFE dev sorted out?
dianakhuang commentedon Jul 13, 2023
Yup! I think we're still trying to understand the requirements needed here. I'll push out the acceptance date for another few months.
dianakhuang commentedon Aug 28, 2023
Discuss posting: https://discuss.openedx.org/t/deprecation-devstack/10703
kdmccormick commentedon Aug 31, 2023
@dianakhuang So, I know I had originally been a proponent of deprecating both configuration and devstack "into the edx org", under the understanding that if only 2U is using some parts of the project, then it's easier for both sides if we transfer those parts over to 2U. Since then, I've talked about this with my team at Axim, and come to understand that the situation is more nuanced than that.
The code in the openedx org is built from a mix of Axim-owned contributions and community-contributor-owned contributions (licensed to Axim under the CLA). We hold the code in this org for the good of all current & former Open edX contributors and users, and as the project's stewards we have a mandate to maintain ownership and ensure availability of all its repositories, even unsupported repositories.
Our procedure of archiving repos by transferring to the openedx-unsupported org lets us mark a repo as "unsupported" while still guaranteeing the repo's availability and keeping its ownership clear. So, if/when devstack and configuration are deprecated, we'll need to move them there.
dianakhuang commentedon Aug 31, 2023
@kdmccormick that sounds reasonable! 2U/edX.org should be able to then clone the repositories for our own use case after they are put into unsupported, then?
Obviously, this is our first stab and understanding how this process is going to work going forward.
kdmccormick commentedon Aug 31, 2023
Yes, you could fork from openedx-supported after that full deprecation & archival process.
Even in that case, though, I'd be curious what we could do to help keep 2U involved with community tooling.
Would you be forking devstack & configuration as a temporary stop-gap until you're able to transition off of them, or with the intent of using them indefinitely?
Same here :) learning as we go!
dianakhuang commentedon Jan 11, 2024
We are going to block acceptance because devstack is still used heavily by people who need to do development on MFEs outside of 2U.
@arbrandes @kdmccormick could you add more context on this?
43 remaining items
feat!: Remove unsupported Devstack references and Docker tooling
robrap commentedon Jan 29, 2025
[inform]
kdmccormick commentedon Jan 29, 2025
@robrap
kdmccormick commentedon Feb 14, 2025
Update: While devstack itself is deprecated, there are still many tendrils to clean up in the openedx org , such as:
robrap commentedon May 16, 2025
@kdmccormick: I'm wondering if we could do away with the "blocked" status, and these tickets are helping me get more clear.
I think this ticket could move to Transition Unblocked and when updating to the new DEPR template, could get a task list for unblocking removal, and a task list for completing the removal. The first list would be required to get to Breaking Changes Unblocked. The Review After Date could simply be used as some future check-in date. Thoughts?
kdmccormick commentedon May 16, 2025
Yes, I think that sounds reasonable! I updated the ticket status. I can add that info to the the tasklist in the future (I plan to update all my tickets in one go when I have a couple spare hours).
dianakhuang commentedon Jun 26, 2025
Docker compose files still kicking around: https://github.com/search?type=code&q=(org%3Aopenedx)+NOT+is%3Aarchived+path%3Adocker-compose.yml
dianakhuang commentedon Jan 8, 2026
This is waiting for new development environment settings files to be created. Blocked on the edx-platform settings refactoring.
dianakhuang commentedon Jun 25, 2026
We need a new base development settings file before this can be completed. The creation of this settings file is no longer blocked.