Skip to content

Error exporting https certificate on macOS Sonoma 14.4 #99735

New issue
New issue
Closed
#99768
Closed
Error exporting https certificate on macOS Sonoma 14.4#99735
#99768
Assignees
vcsjones
Labels
area-System.Securityin-prThere is an active PR which will close this issue when it is mergedos-macosmacOS aka OSX
Milestone
9.0.0

Description

@holm-dk
holm-dk
opened on Mar 11, 2024

Something changed in macOS 14.4 (compared to 14.3) that caused PEM cert export to start raising Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK).

The original repro uses dotnet dev-certs to generate and export a self-signed cert aspnetcore uses during local development, but I've included a self-contained repro below.

[Original description below]

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

Generating https development certificates using this commands throws an exception.

dotnet dev-certs https -v -ep ./test.pem --format Pem --no-password

Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK)

Full output:

[1] Listing certificates from CurrentUser\My
[2] Found certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[3] Checking certificates validity
[4] Valid certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[5] Invalid certificates: no certificates
[6] Finished listing certificates.
[1] Listing certificates from CurrentUser\My
[2] Found certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[3] Checking certificates validity
[4] Valid certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[5] Invalid certificates: no certificates
[6] Finished listing certificates.
[1] Listing certificates from LocalMachine\My
[2] Found certificates: no certificates
[3] Checking certificates validity
[4] Valid certificates: no certificates
[5] Invalid certificates: no certificates
[6] Finished listing certificates.
[8] Filtered certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[9] Excluded certificates: no certificates
[14] Valid certificates: 1 certificate
    1) 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[15] Selected certificate: 9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true
[23] Saving certificate '9633C9DBE09859140B6AE1051D35540809AD6293 - CN=localhost - Valid from 2024-03-11 23:31:20Z to 2025-03-11 23:31:20Z - IsHttpsDevelopmentCertificate: true - IsExportable: true' to ./test.pem with private key.
[24] Exporting certificate with private key but no password.
[25] Creating directory ..
[26] An error has occurred while exporting the certificate: Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK)
   at Interop.AppleCrypto.TrySecKeyCopyExternalRepresentation(SafeSecKeyRefHandle key, Byte[]& externalRepresentation)
   at System.Security.Cryptography.RSAImplementation.RSASecurityTransforms.ExportParameters(Boolean includePrivateParameters)
   at System.Security.Cryptography.RSA.WritePkcs1PrivateKey()
   at System.Security.Cryptography.RSA.TryExportRSAPrivateKey(Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.RSA.WritePkcs8PrivateKey()
   at System.Security.Cryptography.RSA.TryExportEncryptedPkcs8PrivateKey(ReadOnlySpan`1 password, PbeParameters pbeParameters, Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.AsymmetricAlgorithm.ExportArray[T](ReadOnlySpan`1 password, PbeParameters pbeParameters, TryExportPbe`1 exporter)
   at Microsoft.AspNetCore.Certificates.Generation.CertificateManager.ExportCertificate(X509Certificate2 certificate, String path, Boolean includePrivateKey, String password, CertificateKeyExportFormat format).
[26] An error has occurred while exporting the certificate: Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error -67738 - CSSM Exception: -2147415780 CSSMERR_CSP_INVALID_KEYATTR_MASK)
   at Interop.AppleCrypto.TrySecKeyCopyExternalRepresentation(SafeSecKeyRefHandle key, Byte[]& externalRepresentation)
   at System.Security.Cryptography.RSAImplementation.RSASecurityTransforms.ExportParameters(Boolean includePrivateParameters)
   at System.Security.Cryptography.RSA.WritePkcs1PrivateKey()
   at System.Security.Cryptography.RSA.TryExportRSAPrivateKey(Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.RSA.WritePkcs8PrivateKey()
   at System.Security.Cryptography.RSA.TryExportEncryptedPkcs8PrivateKey(ReadOnlySpan`1 password, PbeParameters pbeParameters, Span`1 destination, Int32& bytesWritten)
   at System.Security.Cryptography.AsymmetricAlgorithm.ExportArray[T](ReadOnlySpan`1 password, PbeParameters pbeParameters, TryExportPbe`1 exporter)
   at Microsoft.AspNetCore.Certificates.Generation.CertificateManager.ExportCertificate(X509Certificate2 certificate, String path, Boolean includePrivateKey, String password, CertificateKeyExportFormat format)
   at Microsoft.AspNetCore.Certificates.Generation.CertificateManager.EnsureAspNetCoreHttpsDevelopmentCertificate(DateTimeOffset notBefore, DateTimeOffset notAfter, String path, Boolean trust, Boolean includePrivateKey, String password, CertificateKeyExportFormat keyExportFormat, Boolean isInteractive).
There was an error exporting HTTPS developer certificate to a file.

.NET Version

8.0.201

Anything else?

> dotnet --info
.NET SDK:
 Version:           8.0.201
 Commit:            4c2d78f037
 Workload version:  8.0.200-manifests.3097af8b

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  14.4
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/8.0.201/

.NET workloads installed:
There are no installed workloads to display.

Host:
  Version:      8.0.2
  Architecture: arm64
  Commit:       1381d5ebd2

.NET SDKs installed:
  8.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  x64   [/usr/local/share/dotnet/x64]

Environment variables:
  Not set

global.json file:
  Not found

Activity

  1. cpboyd commented on Mar 13, 2024

    @cpboyd
    cpboyd
    on Mar 13, 2024

    Also facing the same issue. Removing --no-password allows the command to succeed, but this option is used in several SPA projects generated by Visual Studio.

    Edit: Ah, the issue is with exporting the private .key file. Without --no-password, it doesn't generate the .key file.
    Specifying --password test instead results in the same error.
    Even trying to run the commands as sudo still results in the same error.

  2. cpboyd commented on Mar 13, 2024

    @cpboyd
    cpboyd
    on Mar 13, 2024

    Ok. I've found a workaround.
    First, find the trusted cert file:

    ls ~/.aspnet/dev-certs/https

    Next, use openssl to extract the info (replace <cert-ID> with the unique ID):

    openssl pkcs12 -in ~/.aspnet/dev-certs/https/aspnetcore-localhost-<cert-ID>.pfx -out cert.pem -legacy -nodes

    When prompted for a password, just press Return/Enter.

    Then you need to further split the PRIVATE KEY block into its own file <filename>.key:

    -----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----

    And the CERTIFICATE as well into a matching <filename>.cert:

    -----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

    It's not very straight-forward, but it seems like dotnet dev-certs has been deprecated and hasn't seen updates in quite some time: https://www.nuget.org/packages/dotnet-dev-certs

  3. holm-dk commented on Mar 13, 2024

    @holm-dk
    holm-dk
    on Mar 13, 2024
    Author

    Ok. I've found a workaround.

    You can also generate the self-signed SSL certificate for localhost and key using openssl:

    openssl req -x509 -sha256 -nodes -newkey rsa:2048 -days 365 -subj '/CN=localhost' -keyout localhost.key -out localhost.pem

    It's not very straight-forward, but it seems like dotnet dev-certs has been deprecated and hasn't seen updates in quite some time: https://www.nuget.org/packages/dotnet-dev-certs

    Only that NuGet package is deprecated, dotnet dev-certs is not (it's no longer a "tool").

  4. cpboyd commented on Mar 13, 2024

    @cpboyd
    cpboyd
    on Mar 13, 2024

    @thj-dk I guess the main thing that the dotnet cert is supposed to add over that is that it's trusted by the system. (Which, yes, you could manually trust the cert generated via openssl as well.)

  5. amcasey commented on Mar 13, 2024

    @amcasey
    amcasey
    on Mar 13, 2024
    Member

    Yep, the in-box version of dotnet dev-certs is still very much supported, though the tool itself is fairly simple and doesn't change very often.

    The call in question appears to be https://github.com/dotnet/aspnetcore/blob/702ba39ace967037a3f726765732e3e63f80907e/src/Shared/CertificateGeneration/CertificateManager.cs#L506

    Unfortunately, Apple appears to use that error code for a number of different conditions, making it hard to debug by inspection. We may need to pull in https://github.com/dotnet/runtime for their understanding of the PAL.

  6. amcasey commented on Mar 13, 2024

    @amcasey
    amcasey
    on Mar 13, 2024
    Member

    I tried pulling out the important calls from dev-certs. Does this throw the same exception?

    using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;

using var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
var certs = store.Certificates.OfType<X509Certificate2>().ToList();
var cert = certs.Single(c => c.FriendlyName == "ASP.NET Core HTTPS development certificate");
var key = cert.GetRSAPrivateKey()!;
var eyBytes = key.ExportEncryptedPkcs8PrivateKey(string.Empty, new PbeParameters(PbeEncryptionAlgorithm.Aes256Cbc, HashAlgorithmName.SHA256, 1));

    I'd ask for a copy of the cert, but exporting seems to be the crux of the issue. 😆

  7. amcasey commented on Mar 13, 2024

    @amcasey
    amcasey
    on Mar 13, 2024
    Member

    Huh, a colleague happened to be on 14.3 and it didn't repro there, but it did start failing in this way after upgrading to 14.4. Didn't see that coming.

  8. amcasey commented on Mar 13, 2024

    @amcasey
    amcasey
    on Mar 13, 2024
    Member

    Self-contained repro

    using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using System.Text;

var cert = Repro.CreateCertificate();
var key = cert.GetRSAPrivateKey()!;
var eyBytes = key.ExportEncryptedPkcs8PrivateKey(string.Empty, new PbeParameters(PbeEncryptionAlgorithm.Aes256Cbc, HashAlgorithmName.SHA256, 1));

public class Repro
{
    internal const int CurrentAspNetCoreCertificateVersion = 2;
    internal const string AspNetHttpsOid = "1.3.6.1.4.1.311.84.1.1";
    internal const string AspNetHttpsOidFriendlyName = "ASP.NET Core HTTPS development certificate";

    private const string ServerAuthenticationEnhancedKeyUsageOid = "1.3.6.1.5.5.7.3.1";
    private const string ServerAuthenticationEnhancedKeyUsageOidFriendlyName = "Server Authentication";

    private const string LocalhostHttpsDnsName = "localhost";
    private const string LocalhostHttpsDistinguishedName = "CN=" + LocalhostHttpsDnsName;

    public const int RSAMinimumKeySizeInBits = 2048;

    public static X509Certificate2 CreateCertificate()
    {
        // shims
        var notBefore = DateTime.Now;
        var notAfter = DateTime.Now + TimeSpan.FromDays(365);
        var Subject = LocalhostHttpsDistinguishedName;
        var AspNetHttpsCertificateVersion = CurrentAspNetCoreCertificateVersion;


        var subject = new X500DistinguishedName(Subject);
        var extensions = new List<X509Extension>();
        var sanBuilder = new SubjectAlternativeNameBuilder();
        sanBuilder.AddDnsName(LocalhostHttpsDnsName);

        var keyUsage = new X509KeyUsageExtension(X509KeyUsageFlags.KeyEncipherment | X509KeyUsageFlags.DigitalSignature, critical: true);
        var enhancedKeyUsage = new X509EnhancedKeyUsageExtension(
            new OidCollection() {
                new Oid(
                    ServerAuthenticationEnhancedKeyUsageOid,
                    ServerAuthenticationEnhancedKeyUsageOidFriendlyName)
            },
            critical: true);

        var basicConstraints = new X509BasicConstraintsExtension(
            certificateAuthority: false,
            hasPathLengthConstraint: false,
            pathLengthConstraint: 0,
            critical: true);

        byte[] bytePayload;

        if (AspNetHttpsCertificateVersion != 0)
        {
            bytePayload = new byte[1];
            bytePayload[0] = (byte)AspNetHttpsCertificateVersion;
        }
        else
        {
            bytePayload = Encoding.ASCII.GetBytes(AspNetHttpsOidFriendlyName);
        }

        var aspNetHttpsExtension = new X509Extension(
            new AsnEncodedData(
                new Oid(AspNetHttpsOid, AspNetHttpsOidFriendlyName),
                bytePayload),
            critical: false);

        extensions.Add(basicConstraints);
        extensions.Add(keyUsage);
        extensions.Add(enhancedKeyUsage);
        extensions.Add(sanBuilder.Build(critical: true));
        extensions.Add(aspNetHttpsExtension);

        var certificate = CreateSelfSignedCertificate(subject, extensions, notBefore, notAfter);
        return certificate;
    }

    internal static X509Certificate2 CreateSelfSignedCertificate(
            X500DistinguishedName subject,
            IEnumerable<X509Extension> extensions,
            DateTimeOffset notBefore,
            DateTimeOffset notAfter)
    {
        using var key = CreateKeyMaterial(RSAMinimumKeySizeInBits);

        var request = new CertificateRequest(subject, key, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
        foreach (var extension in extensions)
        {
            request.CertificateExtensions.Add(extension);
        }

        var result = request.CreateSelfSigned(notBefore, notAfter);
        return result;

        RSA CreateKeyMaterial(int minimumKeySize)
        {
            var rsa = RSA.Create(minimumKeySize);
            if (rsa.KeySize < minimumKeySize)
            {
                throw new InvalidOperationException($"Failed to create a key with a size of {minimumKeySize} bits");
            }

            return rsa;
        }
    }
}
  9. amcasey
    changed the title [-]Error generating https certificate on macOS Sonoma 14.4[/-] [+]Error exporting https certificate on macOS Sonoma 14.4[/+] on Mar 13, 2024
  10. amcasey
    transferred this issue fromdotnet/aspnetcoreon Mar 13, 2024
  11. ghost added
    area-System.Security
    on Mar 13, 2024
  12. dotnet-policy-service
    added
    untriagedNew issue has not been triaged by the area owner
    on Mar 13, 2024
  13. vcsjones
    added
    os-macosmacOS aka OSX
    on Mar 14, 2024
  14. vcsjones
    added this to the 9.0.0 milestone on Mar 14, 2024

  15. vcsjones commented on Mar 14, 2024

    @vcsjones
    vcsjones
    on Mar 14, 2024
    Member

    Thanks for the self-contained repro @amcasey. I'll take a look.

  16. 4 remaining items

  17. jachin84 commented on Mar 14, 2024

    @jachin84
    jachin84
    on Mar 14, 2024

    I just hit this as well Sonoma 14.4.

  18. vcsjones commented on Mar 14, 2024

    @vcsjones
    vcsjones
    on Mar 14, 2024
    Member

    I think I got a fix. Doing validation.

  19. vcsjones commented on Mar 14, 2024

    @vcsjones
    vcsjones
    on Mar 14, 2024
    Member

    The gist of the problem is that Apple changed the error code returned here

    runtime/src/libraries/Common/src/Interop/OSX/System.Security.Cryptography.Native.Apple/Interop.SecKeyRef.cs

    Line 144 in b181ed5

    if (Interop.CoreFoundation.GetErrorCode(errorHandle) == errSecPassphraseRequired)

    Instead of errSecPassphraseRequired, we are getting errSecInvalidKeyAttributeMask back, so our error handling doesn't happen.

  20. waldekmastykarz commented on Mar 14, 2024

    @waldekmastykarz
    waldekmastykarz
    on Mar 14, 2024

    We're also affected by this issue in Dev Proxy, and it's preventing us from inspecting HTTPS traffic. Can't wait to try the fix. Thank you so much for patching it so quickly! 👏

  21. vcsjones
    closed this as completedin #99768on Mar 14, 2024
  22. vcsjones
    reopened this on Mar 14, 2024

  23. vcsjones commented on Mar 14, 2024

    @vcsjones
    vcsjones
    on Mar 14, 2024
    Member

    Reopened for backports.

  24. vcsjones commented on Mar 25, 2024

    @vcsjones
    vcsjones
    on Mar 25, 2024
    Member

    This has been fixed. The fix will be available in 8.0.4, 7.0.18, and 6.0.29. Since back ports are complete, I am going to close this out.

  25. vcsjones
    closed this as completedon Mar 25, 2024

  26. johndobrien commented on Apr 3, 2024

    @johndobrien
    johndobrien
    on Apr 3, 2024

    Where can I get 8.0.4?

  27. vcsjones commented on Apr 3, 2024

    @vcsjones
    vcsjones
    on Apr 3, 2024
    Member

    Where can I get 8.0.4?

    It is not released yet. It will be part of the April release of .NET, which I think is in the next week or two.

  28. aintabb commented on Apr 3, 2024

    @aintabb
    aintabb
    on Apr 3, 2024

    @johndobrien In the meantime, you could use the solution mentioned here temporarily, #99735 (comment)
    It worked for me.

  29. jaspreetsinghmahal commented on Apr 7, 2024

    @jaspreetsinghmahal
    jaspreetsinghmahal
    on Apr 7, 2024

    My God this issue has been driving me crazy for last 2 hours, thanks finally for this

  30. github-actions
    locked and limited conversation to collaborators on May 8, 2024
  31. dotnet-policy-service
    added
    in-prThere is an active PR which will close this issue when it is merged
    on Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-System.Securityin-prThere is an active PR which will close this issue when it is mergedos-macosmacOS aka OSX

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions