Limitations of Credentials Masking
In the Jenkins project, we ask that people report security issues to our private issue tracker. This allows us to review issues and prepare fixes in private, often resulting in better, safer security fixes.
As a side effect of that, we also learn about common misconceptions and usability problems related to security in Jenkins. This post is intended to address one of those: The goal and limitations of credentials masking.
The Problem
One very common example of that is the role of credentials masking in Jenkins, typically involving a pipeline snippet that looks like this:
withCredentials([usernamePassword(credentialsId: 'topSecretCredentials', passwordVariable: 'PWD', usernameVariable: 'USR')])
sh './deploy.sh' // requires PWD and USR to be set
}Credentials that are in scope are made available to the pipeline without limitation.
To prevent accidental exposure in the build log, credentials are masked from regular output, so an invocation of env (Linux) or set (Windows), or programs printing their environment or parameters would not reveal them in the build log to users who would not otherwise have access to the credentials.
The misconception here is that Jenkins will prevent other, perhaps deliberate ways to reveal the password. Some examples:
withCredentials([usernamePassword(credentialsId: 'topSecretCredentials', passwordVariable: 'PWD', usernameVariable: 'USR')])
sh 'echo $PWD | base64' // will print e.g. dDBwczNjcjN0Cg= which is trivially converted back to the top secret password
}withCredentials([usernamePassword(credentialsId: 'topSecretCredentials', passwordVariable: 'PWD', usernameVariable: 'USR')])
sh 'echo $PWD > myfile'
archiveArtifacts 'myfile' // then browse archived artifacts from the Jenkins UI
}Both of these snippets circumvent credentials masking in the build log, and show that people with control over the build script can use credentials in ways not necessarily intended or approved by admins.
Obviously these are just the most straightforward examples illustrating the problem. Others could involve the proc file system, sending it to an HTTP server in response to a 401 authentication challenge, embedding it in the (otherwise legitimate) build result, etc.
It would be great if Jenkins could allow the flexible use of credentials with no risk of exposing them through straightforward build script modifications, but realistically, it is impossible for Jenkins to police use of the credential by a build script without the support of a very specific environment setup (e.g. restrictive network configuration).
It should also be noted that credentials aren’t just at risk from users able to control the pipeline, typically by editing the Jenkinsfile.
Actual build scripts invoked by pipelines, either shell scripts as in the example above, or more standard build tools such as Maven (controlled by pom.xml) are just as much of a risk if they are run inside a withCredentials block, or executing on the same agent as another block that passed such credentials.
Disclosure of secrets can also happen inadvertently:
Jenkins will prevent exact matches of the password or other secret to appear in the log file.
Consider that the secret may contain shell metacharacters that bash +x would escape by adding a \ before those characters.
The sequence of characters to be printed is no longer identical to the secret, so would not be masked.
The Solution
Credentials can be defined in different scopes: Credentials defined on the root Jenkins store (the default) will be available to all jobs on the instance. The only exception are credentials with System scope, intended for the global configuration only, for example, to connect to agents. Credentials defined in a folder are only available within that folder (transitively, i.e. also in folders inside this folder).
This allows defining sensitive credentials, such as deployment credentials, on specific folders whose contents only users trusted with those credentials are allowed to configure: Directly in Jenkins using Matrix Authorization Plugin and by limiting write access to repositories defining pipelines as code.
Pipelines inside this folder can use the (e.g. deployment) credentials without limitation, while they’re inaccessible to pipelines outside the folder.
Those would need to use the build step or similar approaches to invoke the pipelines inside the folder to deploy their output.
Caveats
While the previous section outlines a solution to the problem of restricting access to credentials, care needs to be taken so that credentials are not captured anyway. For example, a deployment pipeline that allows its users to define where to deploy to as a build parameter might still be used to send credentials to a maliciously set up host to capture them. A blog post explaining the design of some Jenkins project infrastructure discusses some of these concerns around trust.
It should also be noted that credential domains are a UI hint only — defining a credential to only be valid for github.com does not actually prevent its use elsewhere.