Repository navigation
osc exec, Docker images and Software Collections #2001
Description
tl;dr: Couldn't we somehow build the images in such a way that the appropriate Software Collections that it uses are enabled globally?
I noticed that the Docker images we use that leverage SCL have a shortcoming when trying to use what might be the main software binaries the image was meant for.
After setting up a working environment with the sample-app, one might want to run a command in a container. Running a simple command like date works just fine:
$ osc exec -p frontend-1-8zeyh -c ruby-helloworld -- date
Thu Apr 30 09:09:54 UTC 2015But to run ruby in a Ruby Image:
$ osc exec -p frontend-1-8zeyh -c ruby-helloworld -- ruby --version
env: ruby: No such file or directory
Error: Error executing remote command: Error executing command in container: exit status 127
$ osc exec -p frontend-1-8zeyh -c ruby-helloworld -- scl enable ruby200 "ruby --version"
ruby 2.0.0p353 (2013-11-22) [x86_64-linux]The same for databases:
$ osc exec -p database-1-nsxjr -c ruby-helloworld-database -- mysql --version
env: mysql: No such file or directory
Error: Error executing remote command: Error executing command in container: exit status 127
$ osc exec -p database-1-nsxjr -c ruby-helloworld-database -- scl enable mysql55 "mysql --version"
mysql Ver 14.14 Distrib 5.5.37, for Linux (x86_64) using readline 5.1Couldn't we somehow build the images in such a way that the appropriate Software Collections that it uses are enabled globally?
As a consequence, this would enable us to simplify our .sti/* scripts, removing the need to enable SCL in each script, and, most importantly, would make it easier for users to customize their scripts, focusing on functionality rather than having to know and care about SCL.
For instance, a .sti/run script for a Python project could be as simple as:
#!/bin/bash
exec python app.py --db=$DATABASE_SERVICE_HOST:$DATABASE_SERVICE_PORTAs a reference, the behavior I'm looking for matches that of the official Ruby image from Docker Hub:
$ docker run --rm ruby:2.0.0 ruby --version
ruby 2.0.0p645 (2015-04-13 revision 50299) [x86_64-linux]
Activity
ncdc commentedon Apr 30, 2015
osc exec uses the container's configured path, which I'm guessing in this case doesn't include the SCL locations. We'll have to think about ways to handle this.
Sent from my iPhone
ncdc commentedon Apr 30, 2015
And I would also guess you'll see the same behavior with docker exec - can you try and report back?
Sent from my iPhone
rhcarvalho commentedon Apr 30, 2015
@ncdc yes yes, it's the very same thing with
docker exec.You're also right to the point with SCL. Enabling it changes the PATH, making new binaries available or take precedence over others from
/bin,/usr/bin, etc.I don't know what would be the best way to do that, but one possibility I see would be appending the path to the appropriate SCL binaries in
/etc/profile.rhcarvalho commentedon Jun 10, 2015
sclorg/s2i-base-container#35 solved this partially. SCL now is automatically enabled as long as you wrap commands with
bash -c:ncoghlan commentedon Nov 11, 2015
In https://github.com/ncoghlan/rf-apitest/blob/master/Dockerfile (which installs the upstream SCL directly) I handled this for the main command by using ENTRYPOINT to always enable the SCL, and then CMD to run the desired operation.
This handles
docker run(since that invokes ENTRYPOINT), but doesn't help withdocker exec(since that bypasses both ENTRYPOINT and CMD)rhcarvalho commentedon Nov 11, 2015
@ncoghlan any reason you preferred that over the approach in sti-base?
In sti-base we set a few env vars that are picked up by bash to make it enable SCL by sourcing a file.
For instance, this is what the file looks like for Python 2.7:
ncoghlan commentedon Nov 11, 2015
Why would I do that when ENTRYPOINT natively handles the use case? I'm OK with needing to enable the SCL manually under exec, since that lets me choose whether to access the system Python or the SCL one.
rhcarvalho commentedon Dec 18, 2015
Still today we need to be aware of SCL and whether it is enabled or not. This issue has driven us to implement mechanisms to give a better user experience. Today, wrapping commands through
bashorshautomatically enable the necessary collections for our images.I don't think we can do much better than that unless there would be a way to enable SCL globally and permanently.
Before I close this, @hhorak would it be reasonable to have SCLs globally enabled as a feature request?
hhorak commentedon Dec 18, 2015
There are ways (that are sometimes risky and thus not recommended officially) to enable Software Collections globally by having a file in
/etc/profiles.dwhich doessource scl_source enable <mycollection>. This enables the collection for all users outside of container.However, as you can expect, this mechanism depends on the same apparatus as we use already -- shell needs to read
/etc/profiles.din the first place. So, this won't help us in containers.Going one step back, the software collections is a concept that is build on this basic thing: we need to change environment variables by some logic (the variables content is not static). So, enabling a collection permanently in container means being able to change environment in the image by some logic. If we can make something like this to work:
... then we would do make it happen. However, something like that doesn't work in Dockerfile.
Anyway, the all above was to explain why I don't see anything we can do better right now. That doesn't mean somebody else can't find a better way though :)
rhcarvalho commentedon Dec 18, 2015
Summarizing my brainstorming with @hhorak on IRC earlier today:
PATHmanipulation is an incomplete solution, because there are many other environment variables that need to be set for things to work correctly. Forgetting some of them would show problems only under some specific circumstances, causing unexpected bugs. E.g.:LD_LIBRARY_PATH,X_SCLS./usr/bin, and so on. Apparently, this would be too complicated and require building the packages again.scl enable rh-python34 -- pip $@, we would have to have wrappers for all binaries that are located under /opt/.../usr/bin/"This doesn't work, as user installed libraries that include binaries would not know about the trick, e.g.,
pip install ipython-- theipythonexecutable would not have a wrapper to enable the SCL.docker exec ... container-entrypoint pip ...This is essentially the same as having to call
bash -c ..., and now people need to worry about the entrypoint script and what it does.With the lack of much viable improvements in the short term, I'm closing this.