What comes first - Certbot or nginx?

Question

I am trying to set up nginx to act as a reverse proxy on a small personal server; however, I am encountering some circular logic when I ask certbot to generate SSL certificates. From what I can tell, certbot is supposed to modify the nginx.conf file to use certbot's SSL certs, but certbot will not run if nginx.conf is not properly set up to use SLL certs. Here is my nginx.conf:

# /etc/nginx/nginx.conf

# Define the default server block to redirect all other traffic to a static HTML >
http{
        server {
            listen 80 default_server;
            server_name _;

            # Redirect all other traffic to the static HTML page
            location / {
                root /var/www/html;
                index index.html;
            }
        }

        # Server block for sub1.myDomain.org HTTP traffic
        server {
            listen 80;
            server_name sub1.myDomain.org;

            location / {
                proxy_pass http://localhost:60000;
            }
        }

        # Server block for sub1.myDomain.org HTTPS traffic
        server {
                listen 443 ssl;
                server_name sub1.myDomain.org;

                ssl_certificate /etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem
                ssl_certificate_key /etc/letsencrypt/live/sub1.myDomain.org/privatekey.pem
                location / {
                        proxy_pass https://localhost:60001;
                }
           }

        # Server block for sub2.myDomain.org HTTP traffic
        server {
            listen 80;
            server_name sub2.myDomain.org;

            location / {
                proxy_pass http://localhost:60600;
            }
        }

        # Server block for sub2.myDomain.org HTTPS traffic
        server {
                listen 443 ssl;
                server_name sub2.myDomain.org;

                ssl_certificate /etc/letsencrypt/live/sub2.myDomain.org/fullchain.pem
                ssl_certificate_key /etc/letsencrypt/live/sub2.myDomain.org/privatekey.pem
                location / {
                        proxy_pass https://localhost:60601;
                }
          }
}

When I run sudo certbot --nginx -d sub1.myDomain.org or sudo certbot certonly --nginx I get the following error:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Error while running nginx -c /etc/nginx/nginx.conf -t.

nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: configuration file /etc/nginx/nginx.conf test failed

The nginx plugin is not working; there may be problems with your existing configuration.
The error was: MisconfigurationError('Error while running nginx -c /etc/nginx/nginx.conf -t.\n\nnginx: [emerg] cannot load certificate "/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen(\'/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem\',\'r\') error:2006D080:BIO routines:BIO_new_file:no such file)\nnginx: configuration file /etc/nginx/nginx.conf test failed\n')

What am I missing? Thanks so much in advance for the help. The contents ofletsencrypt.log:

2023-08-06 11:02:40,026:DEBUG:urllib3.connectionpool:http://localhost:None "GET /v2/connections?snap=certbot&interface=content HTTP/1.1" 200 97
2023-08-06 11:02:40,414:DEBUG:certbot._internal.main:certbot version: 2.6.0
2023-08-06 11:02:40,414:DEBUG:certbot._internal.main:Location of certbot entry point: /snap/certbot/3024/bin/certbot
2023-08-06 11:02:40,414:DEBUG:certbot._internal.main:Arguments: ['--nginx', '-d', 'sub1.myDomain.org', '--preconfigured-renewal']
2023-08-06 11:02:40,414:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEn>
2023-08-06 11:02:40,431:DEBUG:certbot._internal.log:Root logging level set at 30
2023-08-06 11:02:40,433:DEBUG:certbot._internal.plugins.selection:Requested authenticator nginx and installer nginx
2023-08-06 11:02:40,449:ERROR:certbot.util:Error while running nginx -c /etc/nginx/nginx.conf -t.

nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such fi>
nginx: configuration file /etc/nginx/nginx.conf test failed

2023-08-06 11:02:40,450:DEBUG:certbot._internal.plugins.disco:Misconfigured PluginEntryPoint#nginx: Error while running nginx -c /etc/nginx/nginx.conf -t.

nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such fi>
nginx: configuration file /etc/nginx/nginx.conf test failed
Traceback (most recent call last):
  File "/snap/certbot/3024/lib/python3.8/site-packages/certbot_nginx/_internal/configurator.py", line 1003, in config_test
    util.run_script([self.conf('ctl'), "-c", self.nginx_conf, "-t"])
  File "/snap/certbot/3024/lib/python3.8/site-packages/certbot/util.py", line 125, in run_script
    raise errors.SubprocessError(msg)
certbot.errors.SubprocessError: Error while running nginx -c /etc/nginx/nginx.conf -t.

nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such fi>
nginx: configuration file /etc/nginx/nginx.conf test failed


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/certbot/3024/lib/python3.8/site-packages/certbot/_internal/plugins/disco.py", line 111, in prepare
    self._initialized.prepare()
  File "/snap/certbot/3024/lib/python3.8/site-packages/certbot_nginx/_internal/configurator.py", line 199, in prepare
    self.config_test()
  File "/snap/certbot/3024/lib/python3.8/site-packages/certbot_nginx/_internal/configurator.py", line 1005, in config_test
    raise errors.MisconfigurationError(str(err))
certbot.errors.MisconfigurationError: Error while running nginx -c /etc/nginx/nginx.conf -t.

nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such fi>
nginx: configuration file /etc/nginx/nginx.conf test failed

2023-08-06 11:02:40,452:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: Authenticator, Installer, Plugin
Entry point: nginx = certbot_nginx._internal.configurator:NginxConfigurator
Initialized: <certbot_nginx._internal.configurator.NginxConfigurator object at 0x7f8058ca2e50>
Prep: Error while running nginx -c /etc/nginx/nginx.conf -t.

nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/sub1.myDomain.org/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such fi>
nginx: configuration file /etc/nginx/nginx.conf test failed

Answer 1

Once I commented out the existing https servers in nginx.conf, certbot was able to run and write its own https blocks.