Updated a few seconds ago
Updated a few seconds ago
Status History
July 2024
- Partial Service Disruption in Staging
July 18, 2024 23:20 UTC
Partial Service Disruption in StagingPartial Service Disruption
Incident Status
Partial Service Disruption
Components
acme-staging-v02.api.letsencrypt.org (Staging)
Locations
High Assurance Datacenter 1
July 18, 2024 23:20 UTC[Resolved] This has been resolved.July 18, 2024 23:11 UTC[Investigating] Partial service disruption in staging environment.
- Let's Encrypt APIs unreachable from New Zealand
July 12, 2024 03:10 UTC
Let's Encrypt APIs unreachable from New ZealandPartial Service Disruption
Incident Status
Partial Service Disruption
Components
acme-v02.api.letsencrypt.org (Production), acme-staging-v02.api.letsencrypt.org (Staging)
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
July 12, 2024 03:10 UTC[Resolved] Network connectivity has been restoredJuly 11, 2024 18:44 UTC[Investigating] Let's Encrypt is largely unreachable from New Zealand due to an upstream network disruption between New Zealand and Let's Encrypt. We are still working with our network providers to find a way to resolve the situation.July 11, 2024 03:23 UTC[Investigating] The Let's Encrypt ACME API appears to be intermittently unreachable from New Zealand. We are working with our upstream networking provider to resolve this issue.
- Reports of "connection closed" errors to Let's Encrypt API
July 9, 2024 16:46 UTC
Reports of "connection closed" errors to Let's Encrypt APIDegraded Performance
Incident Status
Degraded Performance
Components
acme-v02.api.letsencrypt.org (Production)
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
July 9, 2024 16:46 UTC[Resolved] As of 14:17 UTC, network connectivity appears to be performing normally again.July 9, 2024 01:45 UTC[Investigating] We are investigating reports of users receiving "connection closed" or "EOF" errors from ACME clients connecting to Let's Encrypt APIs
- Partial service disruption to domain validation for .top TLDs
July 2, 2024 18:58 UTC
Partial service disruption to domain validation for .top TLDsPartial Service Disruption
Incident Status
Partial Service Disruption
Components
acme-v02.api.letsencrypt.org (Production)
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
July 2, 2024 18:58 UTC[Resolved] Error rates have dropped back to normal levels, and certificates for .top domains are once again issuing normally. The .top nameservers have resumed answering our queries.June 26, 2024 20:12 UTC[Investigating] We are investigating problems with the .top TLD nameservers failing to respond to Let's Encrypt's validation DNS queries, causing domain validation for .top domains to fail.June 25, 2024 20:56 UTC[Investigating] We are seeing an increase in failures in domain validation for .top TLDs in production.
June 2024
- Let's Encrypt incorrectly rejecting certificate requests with uppercase domain names
June 21, 2024 00:30 UTC
Let's Encrypt incorrectly rejecting certificate requests with uppercase domain namesPartial Service Disruption
Incident Status
Partial Service Disruption
Components
acme-v02.api.letsencrypt.org (Production)
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
June 21, 2024 00:30 UTC[Resolved] A fix has been deployedJune 20, 2024 17:18 UTC[Identified] Last week's release of Boulder, our CA software, has a bug where it incorrectly rejects uppercase identifiers in certificate requests. We have deployed a fix to our staging environment and will be deploying a production fix in 3 hours.
- [Staging] Staging downtime
June 13, 2024 20:33 UTC
[Staging] Staging downtimePlanned Maintenance
Description
Staging will be down for a few minutes for a storage upgrade.
Components
acme-staging-v02.api.letsencrypt.org (Staging), stg-{e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1
Schedule
June 13, 2024 20:30 - June 13, 2024 21:00 UTC
June 13, 2024 20:33 UTC[Update] Staging maintenance is completed.June 13, 2024 20:30 UTC[Update] Staging is going offline for storage maintenance now.
- [Staging] OCSP Service Disruption
June 6, 2024 23:46 UTC
[Staging] OCSP Service DisruptionService Disruption
Incident Status
Service Disruption
Components
stg-{e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1
June 6, 2024 23:46 UTC[Resolved] Staging's OCSP is functional.June 6, 2024 23:01 UTC[Identified] Staging's OCSP responders are undergoing maintenance.
- Infrastructure maintenance
June 3, 2024 20:55 UTC
Infrastructure maintenancePlanned Maintenance
Description
We're performing maintenance on our production infrastructure. We do not expect any service impact, but degraded performance may be possible.
Components
acme-v02.api.letsencrypt.org (Production), {e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 2
Schedule
June 3, 2024 18:05 - June 3, 2024 22:30 UTC
June 3, 2024 20:55 UTC[Update] The maintenance on our production infrastructure is complete.June 3, 2024 18:07 UTC[Update] We're performing maintenance on our production infrastructure. We do not expect any service impact, but degraded performance may be possible.
- API Database Maintenance
June 3, 2024 15:43 UTC
API Database MaintenancePlanned Maintenance
Description
We're performing database maintenance which may result in some ACME clients receiving timeouts for up to a 10 minute period.
Components
acme-v02.api.letsencrypt.org (Production)
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
Schedule
June 3, 2024 15:40 - June 3, 2024 15:50 UTC
June 3, 2024 15:43 UTC[Update] The database maintenance is complete.June 3, 2024 15:40 UTC[Update] We're performing database maintenance which may result in some ACME clients receiving timeouts for up to a 10 minute period.
May 2024
- Elevated Rate of 503 Service Busy Responses
May 31, 2024 21:22 UTC
Elevated Rate of 503 Service Busy ResponsesPartial Service Disruption
Incident Status
Partial Service Disruption
Components
acme-v02.api.letsencrypt.org (Production), {e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 2
May 31, 2024 21:22 UTC[Resolved] This issue has been resolved.May 31, 2024 19:23 UTC[Monitoring] Full service capacity has been restored and we are monitoring for any further issues.May 31, 2024 18:29 UTC[Identified] Due to an unexpected complication with ongoing maintenance, Let's Encrypt's API is returning an elevated rate of 503 Service Busy responses. We have identified the issue and are in the process of returning service to normal operation.
- Staging environment maintenance
May 31, 2024 20:37 UTC
Staging environment maintenancePlanned Maintenance
Description
We're performing maintenance in our staging environment, which will be unavailable for up to a few hours. We do not expect our production environment to be affected.
Components
acme-staging-v02.api.letsencrypt.org (Staging), stg-{e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1
Schedule
May 31, 2024 17:42 - May 31, 2024 21:00 UTC
May 31, 2024 20:37 UTC[Update] The maintenance in our staging environment is complete.May 31, 2024 17:43 UTC[Update] We're performing maintenance in our staging environment, which will be unavailable for up to a few hours. We do not expect our production environment to be affected.
- Elevated Rate of 503 Service Busy Responses
May 24, 2024 21:34 UTC
Elevated Rate of 503 Service Busy ResponsesDegraded Performance
Incident Status
Degraded Performance
Components
acme-v02.api.letsencrypt.org (Production)
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
May 24, 2024 21:34 UTC[Resolved] The excessive 503s have been remediated.May 16, 2024 22:42 UTC[Monitoring] We've applied a fix to the elevated rate of 503 responses as of 19:50 UTC, with no 503s served since then. We will continue monitoring through the next periods of high load before marking this incident resolved.May 16, 2024 15:49 UTC[Investigating] Starting 0900 UTC on 9 May, Let's Encrypt's API has returned an elevated rate of 503 Service Busy responses, which we return as a protection measure when request volume is high [1]. While absolute numbers are still low (<1% of requests) , we expect to return zero 503 Service Busy responses most of the time. Clients receiving a 503 Service Busy are given a Retry-After header which indicates how long they should wait before retrying. Many clients do not respect this header; please check that yours does. We have been investigating the source of this issue, and will continue to do so until we get it resolved. [1] https://community.letsencrypt.org/t/new-service-busy-responses-beginning-during-high-load/184174
- Service disruption
May 20, 2024 17:21 UTC
Service disruptionService Disruption
Incident Status
Service Disruption
Components
acme-v02.api.letsencrypt.org (Production), {e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
May 20, 2024 17:21 UTC[Resolved] The weekend's disruption is resolved.May 18, 2024 23:13 UTC[Identified] We have partially resolved the issue and restored service. However, we're aware of a condition that's likely to cause intermittent, brief disruptions. We're now working to mitigate that condition.May 18, 2024 22:47 UTC[Identified] Service is disrupted due to a problem with our infrastructure. We've identified the problem and are working to mitigate it ASAP.
- API Database Maintenance
May 17, 2024 23:03 UTC
API Database MaintenancePlanned Maintenance
Description
We're performing database maintenance which may result in some ACME clients receiving timeouts for up to a 10 minute period.
Components
acme-v02.api.letsencrypt.org (Production), {e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
Schedule
May 17, 2024 22:50 - May 17, 2024 23:00 UTC
May 17, 2024 23:03 UTC[Update] The database maintenance is complete.May 17, 2024 22:51 UTC[Update] We're performing database maintenance which may result in some ACME clients receiving timeouts for up to a 10 minute period.
- [Staging] Service Disruption
May 13, 2024 22:47 UTC
[Staging] Service DisruptionService Disruption
Incident Status
Service Disruption
Components
acme-staging-v02.api.letsencrypt.org (Staging), stg-{e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1
May 13, 2024 22:47 UTC[Resolved] The issue with Staging is resolved.May 13, 2024 19:27 UTC[Monitoring] We've remediated the problem with Staging, have brought it back online, and are now monitoring the fix.May 13, 2024 19:19 UTC[Investigating] Service is disrupted in Staging.
April 2024
- [Staging] Service Disruption
April 26, 2024 17:47 UTC
[Staging] Service DisruptionService Disruption
Incident Status
Service Disruption
Components
acme-staging-v02.api.letsencrypt.org (Staging), stg-{e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1
April 26, 2024 17:47 UTC[Resolved] Staging is online and healthy.April 26, 2024 17:36 UTC[Monitoring] The staging environment is operational, and we're monitoring it.April 26, 2024 17:30 UTC[Identified] We've identified the problem with Staging and are in-progress on the fix.April 26, 2024 17:26 UTC[Investigating] Staging is currently down, as of 17:08Z. We're investigating the outage.
March 2024
- Staging Maintenance
March 4, 2024 16:35 UTC
Staging MaintenancePlanned Maintenance
Description
Staging will be down for maintenance for several minutes during this window. Clients will receive error messages.
Components
acme-staging-v02.api.letsencrypt.org (Staging), stg-{e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1
Schedule
March 4, 2024 16:00 - March 4, 2024 17:00 UTC
March 4, 2024 16:35 UTC[Update] Staging maintenance is completed.March 4, 2024 16:06 UTC[Update] We're beginning Staging maintenance. Staging will be down for a short period of time.
February 2024
- Emergency infrastructure maintenance
February 8, 2024 22:12 UTC
Emergency infrastructure maintenancePartial Service Disruption
Incident Status
Partial Service Disruption
Components
acme-v02.api.letsencrypt.org (Production), {e,r}[1-14].o.lencr.org, Website, acme-staging-v02.api.letsencrypt.org (Staging), stg-{e,r}[1-14].o.lencr.org, oak.ct.letsencrypt.org (Production), sapling.ct.letsencrypt.org (Staging)
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2, Public Data Center
February 8, 2024 22:12 UTC[Resolved] The emergency maintenance is complete.February 8, 2024 21:59 UTC[Identified] We're performing emergency infrastructure maintenance which is causing several brief (up to 90 second) service disruptions over an approximately 15-minute period. The maintenance will be completed shortly.
- STAGING Database Maintenance
February 2, 2024 20:36 UTC
STAGING Database MaintenancePlanned Maintenance
Description
[STAGING] We're performing database maintenance which may result in some ACME clients receiving timeouts periodically over the next several hours.
Components
acme-staging-v02.api.letsencrypt.org (Staging), stg-{e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1
Schedule
February 2, 2024 19:00 - February 2, 2024 22:00 UTC
February 2, 2024 20:36 UTC[Update] Staging maintenance completed.February 2, 2024 19:45 UTC[Update] [Staging] Our staging environment's database maintenance is beginning. Staging clients may see timeouts periodically over the next two hours.
January 2024
- Production Database Maintenance
January 30, 2024 16:07 UTC
Production Database MaintenancePlanned Maintenance
Description
We're performing database maintenance which may result in some ACME clients receiving timeouts for short periods during the maintenance window.
Components
acme-v02.api.letsencrypt.org (Production), {e,r}[1-14].o.lencr.org
Locations
High Assurance Datacenter 1, High Assurance Datacenter 2
Schedule
January 30, 2024 15:00 - January 30, 2024 17:00 UTC
January 30, 2024 16:07 UTC[Update] We have finished maintenance.January 30, 2024 15:00 UTC[Update] We are beginning maintenance on the database now.
Back to current status
Subscribe to receive status updates by email
Manage Existing SubscriptionCreate New Subscription
Subscribe to receive status updates in Microsoft Teams
Enter your Microsoft Teams webhook. View Instructions
Email address for managing subscription
Manage Existing SubscriptionCreate New Subscription
Slack channel ID
Find the channel ID: Select the channel in your Slack workspace. The channel ID is displayed in the browser URL.
Example: https://app.slack.com/client/T04SJBK1C/C03SKGJ1P
Email address
Manage Existing SubscriptionCreate New Subscription