Skip to content
This repository was archived by the owner on Feb 16, 2023. It is now read-only.
This repository was archived by the owner on Feb 16, 2023. It is now read-only.

Map all secrets from directory to environment variables #286

Closed
Closed
Map all secrets from directory to environment variables#286
Labels
enhancementNew feature or request

Description

@jpcoenen
jpcoenen
opened on May 11, 2020

Currently, a secrethub.env file has to be created to easily map multiple secrets to environment variables when using secrethub run.

However, in many cases the names of secrets already describe the names of the environment variables they should be mounted to. By adding the --secrets-dir path/to/dir flag, you could easily map load all secrets to the environment.

The mapping could follow a default set of rules, e.g. replace dirs (/) with _ etc. This could drastically decrease the setup time needed, as no templates need to be written.

Example

This tree:

org/repo/my-app/
    aws/
        access_key_id
        secret_access_key
    db/
        username
        password

Would map to:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
DB_USERNAME
DB_PASSWORD

by running:

secrethub run --secrets-dir=org/repo/my-app/ -- node app.js

Activity

  1. jpcoenen
    added
    enhancementNew feature or request
    on May 11, 2020

  2. mgrlabs commented on May 12, 2020

    @mgrlabs
    mgrlabs
    on May 12, 2020

    Hi there,

    How does this work if you were to reference org/repo/my-app/aws? Would you get?

    ACCESS_KEY_ID
SECRET_ACCESS_KEY

    Regards,
    Mark.

  3. jpcoenen commented on May 12, 2020

    @jpcoenen
    jpcoenen
    on May 12, 2020
    MemberAuthor

    Hi Mark,

    My current proposal would indeed be that using --secrets-dir=org/repo/my-app/aws results in:

    ACCESS_KEY_ID
SECRET_ACCESS_KEY

    Is that what you'd expect?

  4. mgrlabs commented on May 12, 2020

    @mgrlabs
    mgrlabs
    on May 12, 2020

    That's a pretty cool idea to map subordinate directories as he prefix for the environment variable. The example I would give is where I would store secrets required for Terraform to deploy into Azure in org/product/dev/arm containing:

    client_id
client_secret
tentant_id
subscription_id

    would map to:

    ARM_CLIENT_ID
ARM_CLIENT_SECRET
ARM_TENANT_ID
ARM_SUBSCRIPTION_ID

    This would work well in that case. Gives us flexibility. Looks good :)

    Regards,
    Mark.

  5. stavalfi commented on Jul 4, 2020

    @stavalfi
    stavalfi
    on Jul 4, 2020

    • I'm assuming that this feature will let us avoid writing a secrethub.env file (or removing some content from it). if yes, keep reading. if not, I'm wrong and just ignore this response).

    • I'm assuming that the community is allowed to have an opinion about this issue so I will write my own. if not, ignore it.

    when a project has a secrethub.env file, there are advantages of visibility and explicitness (maybe its the same) about which env-var are used. it's a single-source-of-truth for the secrets that are being used in the project.

    when developing in a project when the project made a use of this feature, I can't know which env-var that is being used, is a secret or not.

    a possible response can be:

    • a developer shouldn't be care if a env-var is a secret or not.
      • I'm not sure that I have much expirience to answer this but my intoition tells me that the developer should know if a env-var is a secret or not.
      • you guys can argue that the developer can list the secrets of a project in secrethub-cli. but what if he doesn't have a read permission in this repo? maybe he doesn't have have an account?
    • a project should have an abstraction over secrethub: a function that returns all secrets as variables instead of making use of env-var directly in the code. so the issue that I mentioned above, is not a problem.

    What do you guys think?

  6. SimonBarendse commented on Jul 6, 2020

    @SimonBarendse
    SimonBarendse
    on Jul 6, 2020
    Member

    Hi @stavalfi , yes, we welcome any input from the community!

    This feature is indeed meant to be used instead of a secrethub.env file.

    Note however, that we will still be supporting secrethub.env files and all other methods already available to pass environment variables (--envar flag, secret references, environment files and environment variables passed by the OS).

    So, if you prefer the secrethub.env files, feel free to keep using those!

    As you also noted, this --secrets-dir flag doesn't require explicit definition of the environment variables in the secrethub.env file or any other place.
    This can be seen both as an advantage (Your directory in SecretHub is already the source of truth of all secrets in your project, so using that to determine which environment variables need to be populated saves time of maintaining a secrethub.env file that just echos that state) or as a disadvantage (you no longer have a secrethub.env file in source control which exactly defines which secrets you'll be using, making your Git repo the source of truth and always having a commit for any changes to your environment). I believe it largely depends on whether you want your directory in SecretHub or your Git repo to be the source of truth about which secrets are being used.

    I imagine the preference and trade-offs between both methods to vary between projects and processes, which is why we aim to support both and leave it up to you to pick the solution that best fits your use.

    You mentioned using both secrets and non-secrets in the environment. We've seen many users store both of these in SecretHub. You can also store values in SecretHub that are not secret. We might add features later which make it possible to differentiate between these two types of values stored in SecretHub (e.g. no masking, other access levels etc.) but this is not something we've currently planned. However, please use the current set of features as you see fit, including storing other values than secrets.

    You mentioned a user without read permissions not being able to see what secrets will be used when the --secrets-dir flag is used. I can imagine adding a separate permission later (named e.g. list), which allows you to read the metadata (e.g. names of the secrets), but doesn't allow you to read the secret data. However, this is not on the roadmap yet.

    a project should have an abstraction over secrethub: a function that returns all secrets as variables instead of making use of env-var directly in the code. so the issue that I mentioned above, is not a problem.

    I'm not sure I quite understand what you mean. Do you mean to call SecretHub directly from your code? Could you please elaborate on this a bit? Maybe an example can help?

  7. stavalfi commented on Jul 6, 2020

    @stavalfi
    stavalfi
    on Jul 6, 2020

    the roadmap sounds very cool! thanks for the great answer :)

  8. SimonBarendse commented on Jul 21, 2020

    @SimonBarendse
    SimonBarendse
    on Jul 21, 2020
    Member

    This has been merged in #299 and will be included in the next release

  9. SimonBarendse commented on Sep 14, 2020

    @SimonBarendse
    SimonBarendse
    on Sep 14, 2020
    Member

    This has been released in v0.41.0

  10. SimonBarendse
    closed this as completedon Sep 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions