According to research by GitGuardian and CyberArk, 79% of IT leaders reported that they have experienced a secret leakcompared to 75% in the previous year’s report. At the same time, the number of credential leaks has never been greater than There are 12.7 million hard-coded credentials in public GitHub repositories alone. One of the most disturbing aspects of this report is that over 90% of the real secrets found and reported remained valid for more than 5 days.
According to studies take an average of 27 days for organizations to fix a credential leak. Combine this with that non-human identities outnumber human identities by at least 45:1and it’s easy to see why many organizations realize that stopping the spread of secrets means finding a way to deal with the machine’s identity crisis. Unfortunately, the research also shows that many teams are confused about who owns the security of these individuals. It’s a perfect storm of risk.
Why is rotation taking so long
So why do we take so long to change credentials when we know it’s one of the easiest ways for adversaries to attack? One of the main factors is the lack of clarity about how our credentials are authorized. Permissions are what allow what specific things one object, such as a Kubernetes workload or microservice, can successfully request from another service or data source.
Let’s remember what it means to fix a secret leak incident: you need to safely replace the secret without breaking anything or giving new permissions that are too broad, potentially putting your company at greater security risk. If you already have a complete understanding of the lifecycle of your Inhumans and their associated secrets, it’s a fairly simple process to replace them with new secrets with the same permissions. This can take a long time if you don’t already have this information, as you have to hope that the developer who originally created it is still there and has documented what was done.
Let’s explore why permission management is especially challenging in an NHI-dominated environment, explore the challenges developers and security teams face in balancing access control and performance, and discuss how a shared responsibility model can help.
Who really owns the secrets?
Secret sharing generally refers to sharing access keys, passwords, and other sensitive credentials in development environments, repositories, and services like Slack or Jira. The latest GitGuardian Voice of the Practitioners report highlights that 65% of respondents place remediation responsibility directly on IT security teams. Meanwhile, 44% of IT leaders said developers don’t follow best practices to manage secrets.
Secret sharing and the underlying issues of over-authorized long-term credentials will continue to fall into this gap until we figure out how best to work together in a shared responsibility model.
A developer’s view of permissions
Developers face enormous pressure to build and deploy features quickly. However, carefully managing permissions while adhering to security best practices can be time-consuming. Each project or application often has its own unique access requirements that require time to research and properly establish, almost feeling like a full-time job on top of the work of building and deploying their applications. Best practices for creating and managing permissions are too often not applied uniformly across teams, rarely documented properly, or completely forgotten after a developer launches an application.
The problem is compounded by the fact that in too many cases developers simply give too broad permissions to these machines. One report found that only 2% of permits granted are actually used. If we take a closer look at what they’re up against, it’s easy to see why.
For example, consider managing permissions in Amazon Web Services. AWS Identity and Access Management (IAM) policies are known for their flexibility, but they are also complex and confusing to navigate. IAM supports different types of policies—identity-based, resource-based, and permission limits—all of which require precise configurations. AWS also offers multiple access paths for credentials, including IAM roles and KMS (Key Management Service) grants, each of which comes with its own unique access configurations. Learning this system is quite a task.
Another common example of a service where managing permissions can get tricky is GitHub. API keys can grant permissions to repositories in different organizations, making it difficult to ensure appropriate access boundaries. A single key can inadvertently provide excessive access across environments when developers are members of multiple organizations. The pressure mounts to get it right, while the clock is always ticking and the backlog grows.
Why can’t the security services alone fix this
It may seem logical to assign security teams responsibility for monitoring and communicating secrets; after all, it’s a security issue. The reality is that these teams often lack the detailed project-level knowledge needed to make changes safely. Security teams don’t always have the context to understand which specific permissions are important to keep applications running. For example, a seemingly minor change in resolution can disrupt the CI/CD pipeline, disrupt production, or even bring down an entire company cascading failure when the wrong service goes down.
The distributed nature of secret management across teams and environments also increases the attack surface. If no one is truly in charge, it becomes much more difficult to maintain consistency in access controls and audit trails. This fragmentation often results in credentials and associated permissions remaining active for too long, possibly forever. It can be difficult to know who has legal or illegal access to which secrets at any given time.
Shared responsibility model for faster rotation
Developers and security teams can help solve these problems by meeting in the middle and building a model of shared responsibility. In this model, developers are more responsible for consistently managing their permissions using appropriate tools such as Manager of CyberArk Conjur Secrets or Vault by HashiCorpand better document permissions and scope of required permissions at the project level. Security teams should help developers by working to automate secret rotation, investing in appropriate monitoring tools to gain clarity on the state of secrets, and working with IT to completely eliminate long-lived credentials.
When developers clearly document what permissions are required in their requirements, it can help security teams conduct faster and more accurate audits and speed up remediation. If security teams work to ensure that the easiest and fastest common path to implementing a new non-human identity secret is also the most secure and scalable path, then there will be far fewer incidents requiring emergency rotation, and everyone will win .
The goal of developers should be to ensure that the security team can confidently change or update credentials in their applications without compromising production.
The main issues to be resolved are related to the issuance of permits
When thinking about what to document, here are some specifics to help this cross-team work flow more smoothly:
- Who created the credentials? – Many organizations find it difficult to track ownership of credentials, especially if the key is shared or encrypted. This knowledge is essential to understanding who is responsible for changing or revoking authority.
- What resources does it have access to? – API keys can often access a range of services, from databases to third-party integrations, making it necessary to limit permissions to the bare minimum.
- What permissions does it give? – Permissions vary widely across roles, resource-based policies, and policy conditions. For example, in Jenkins, a user with `Overall/Read` permission can view general information, while `Overall/Administer` gives full control over the system.
- How can we withdraw or reverse it? – Ease of revocation varies by platform, and in many cases teams must manually track keys and permissions across systems, making remediation difficult and extending exposure to threats.
- Account active? – It is important to know if credentials are being used. When NHIs use long-term API keys, these credentials can remain active indefinitely if not properly managed, creating ongoing access risks.
Permissions are complex, but we can manage them together as a team
According to a GitGuardian report, while 75% of respondents expressed confidence in their secret management capabilities, the reality is often much different. The average fix time of 27 days reflects this gap between confidence and practice. It’s time to rethink how we implement and communicate secrets and their permissions as an organization.
While developers work hard to balance security and functionality, the lack of streamlined approval processes and non-centralized or non-standardized documentation paths only increase the risks. Security teams alone cannot effectively address these issues because of their limited understanding of specific project needs. They should work hand-in-hand with developers every step of the way.
GitGuardian builds the next generation of secret security tools, helping security and IT teams deal with the spread of secrets. Getting rid of this threat requires knowing what long-term credentials are exposed in your code and other environments. Get started with GitGuardian today.
