Skip to content

Conversation

@shjala
Copy link
Member

@shjala shjala commented Jan 15, 2026

Description

This change fixes the security issue where TLS root certificates were copied from the integrity-protected /config partition to the mutable /persist/certs directory and referenced via SHA256 indirection.

Changes:

  • Remove certificate copy logic from InitializeCertDir() - now only creates the certs directory without copying files
  • Update GetTLSConfig() to read certificates directly from types.V2TLSBaseFile (/config/v2tlsbaseroot-certificates.pem)
  • Update UpdateTLSProxyCerts() to read from V2TLSBaseFile instead of V2TLSCertShaFilename
  • Remove V2TLSCertShaFilename constant from locationconsts.go as it's no longer needed
  • Update getSecurityInfo() in handlemetrics.go to compute SHA256 hash directly from the V2TLSBaseFile content instead of reading a pre-computed SHA from persist

PR dependencies

None

How to test and validate this PR

  1. Verify TLS connections work after the change, device should connect to the controller, receive configurations, certs, etc.
  2. Verify certificates are read from /config (not /persist):
# Should NOT contain files like: 6edbc9851d43dc2cd77bcd60d758994663b9f4bf or v2tlsbaseroot-certificates.sha256
eve exec pillar ls -la /persist/certs/
# Verify the base cert exists in /config
cat /config/v2tlsbaseroot-certificates.pem | head -5
  1. Test with proxy certificates, proxy certs are dynamically added to cert pool (unchanged by this PR):
  • Configure a network proxy with custom certificates
  • Verify device still connects through proxy correctly
  1. Validate offline tampering protection:
  • modify cert on the /config directory
  • should result in vault lock up
  1. Test upgrade from a version without this PR:
  • Device running old version (e.g., 16.0.0) with certs in /persist/certs/]
  • Upgrade to version with this fix
  • Verify device continues to work (ignores old persist copies, uses /config)
  • Old SHA files in /persist/certs/ should is harmless (as it is not read anymore)

Changelog notes

Fixed security issue where TLS root certificates could be tampered with via physical disk access, now loading certificates directly from integrity-protected /config partition.

PR Backports

Possibly all :

  • 11.0-stable
  • 12.0-stable
  • 13.4-stable
  • 14.5-stable
  • 16.0-stable

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR

For backport PRs (remove it if it's not a backport):

  • I've added a reference link to the original PR
  • PR's title follows the template

And the last but not least:

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

Please, check the boxes above after submitting the PR in interactive mode.

This change fixes the security issue where TLS root certificates were
copied from the integrity-protected /config partition to the mutable
/persist/certs directory and referenced via SHA256 indirection.

Changes:
- Remove certificate copy logic from InitializeCertDir() - now only
  creates the certs directory without copying files
- Update GetTLSConfig() to read certificates directly from
  types.V2TLSBaseFile (/config/v2tlsbaseroot-certificates.pem)
- Update UpdateTLSProxyCerts() to read from V2TLSBaseFile instead of
  V2TLSCertShaFilename
- Remove V2TLSCertShaFilename constant from locationconsts.go as it's
  no longer needed
- Update getSecurityInfo() in handlemetrics.go to compute SHA256 hash
  directly from the V2TLSBaseFile content instead of reading a
  pre-computed SHA from persist

Signed-off-by: Shahriyar Jalayeri <shahriyar@posteo.de>
@shjala shjala self-assigned this Jan 15, 2026
@shjala shjala added stable Should be backported to stable release(s) security Provides a security fix labels Jan 15, 2026
Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that soon we should figure out how to deliver a new root CA file to deployed devices, do you have ideas how to update it?

We could do the eve config mount and then to a writerename to types.V2TLSBaseFile perhaps with a .bak file to handle a power outage? Or something different?

@shjala
Copy link
Member Author

shjala commented Jan 16, 2026

Given that soon we should figure out how to deliver a new root CA file to deployed devices, do you have ideas how to update it?

We could do the eve config mount and then to a writerename to types.V2TLSBaseFile perhaps with a .bak file to handle a power outage? Or something different?

If we start storing these in rootfs, then we don't have to change the process, a system update brings the new certs too, but if we want to allow surgical update of the pinned certs, then I think storing in config is good approach. But let's discuss all the options and have a security assessment of each before commiting.

@eriknordmark
Copy link
Contributor

Given that soon we should figure out how to deliver a new root CA file to deployed devices, do you have ideas how to update it?
We could do the eve config mount and then to a writerename to types.V2TLSBaseFile perhaps with a .bak file to handle a power outage? Or something different?

If we start storing these in rootfs, then we don't have to change the process, a system update brings the new certs too, but if we want to allow surgical update of the pinned certs, then I think storing in config is good approach. But let's discuss all the options and have a security assessment of each before commiting.

Not part of this PR, but we do have /etc/ssl/certs/ca-certificates.crt in the rootfs so we could have EVE compare and update /config/v2tlsbaseroot-certificates.pem from that file (e.g., by introducing some additional onboot container). That will ensure that we update it as we update the rootfs to new (Alpine) versions.

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but CI/CD has issues. Can you rebase on master to see if those issues go away?

Copy link
Contributor

@eriknordmark eriknordmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether we should just drop this file from /config and instead read from the rootfs i.e., read directly from /etc/ssl/certs/ca-certificates.crt and feed that to AppendCertsFromPEM().

@shjala
Copy link
Member Author

shjala commented Jan 19, 2026

@eriknordmark OK, I marked this as draft for now, let me finish the Flexible PCR feature and then get back to this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

security Provides a security fix stable Should be backported to stable release(s)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants