`PKIX Path Building Failed` Error and APIs Failing Although CA Certificates Are Added

Issue

The APIs are failing and,  although the CA certificates are in the ca-certs folder, the user gets the following error in the server logs: 

PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target 

However, when running the ls command inside the ca-certs folder, the certificates are displayed as expected. This article explains how to troubleshoot this issue further and provides a potential solution.

Note: If there is no ca-certs folder inside your stacks folder or you cannot find your CA certificates at all in the ca-certs folder, please follow the steps from this documentation to install the CA certificates. If you don't have the CA certificate, then this article explains how you can export it from Chrome. 

Resolution

Check the server logs to verify if the CA certificates have been loaded. If they have not been loaded, follow the below steps: 

  1. Run the ls -la command in the ca-certs directory to get detailed information on each file of this directory and also include hidden files. 
     
  2. Check the first character of the permissions field of the certificate files to identify what file type it is. If the first character in the permissions field is l, then it means that the file is a symbolic link (symlink).

    Here are some examples of how the output of ls -la might look like for different file types:
     
    • Regular File or  Executable File (-):
      -rw-r--r--  1 user  group   12345 Feb  6 10:00 example.txt
    • Directory (d):
      drwxr-xr-x  2 user  group   4096 Feb  6 09:30 my_directory
    • Symbolic Link (l):
      lrwxrwxrwx  1 user  group      8 Feb  6 11:15 link_to_file -> myfile.txt
    • Device File (Block or Character):
      brw-rw----  1 user  disk    8,   1 Feb  6 13:45 sda1
       
  3. In case you identify that the certificate files are symlinks, replace them with the full files to ensure they are picked up.
  4. Restart Appsmith after you added the actual files to the ca-certs folder.
  5. Check the logs to confirm that the certificates are picked up and try running again the APIs to verify if they run successfully.

We have a GitHub issue to address this issue and pick up the CA certificates even if they are symlinks: https://github.com/appsmithorg/appsmith/issues/30928.