Notice #504402
Recovering

[S3] Location es34 connection issues

Began:

Further investigation revealed an additional potential cause. We will apply a fix and closely monitor the situation. Once the systems remain stable over the next 24 hours, we will mark this incident as resolved.

The issue arose again. We are in contact with the vendor.

Our deepest apologies. We are working hard to resolve the issue asap.

We have identified the root cause: a process [in the storage layer] got stuck, and a reboot was required to restore service. Together with the vendor, we are improving the monitoring for this component and have defined a playbook to resolve similar issues faster in the future. There is currently no indication that this issue will re-occur. We consider this incident resolved.

Buckets are reachable again. We are still investigating the root cause together with the vendor and monitoring the situation. Once we are confident the issue is fully resolved, we will post a final update. If you continue to experience issues, please reach out to support.

All buckets in es34 are affected and all services that use S3 are affected. Including: Loki, Docker Registry

We are investigating and will get back to you

  • General
  • Virtualization
  • Managed Kubernetes
  • Network
  • External
  • E-Mail
  • Deploio
Previous Incidents

[Resolved] [S3 Buckets] Some custom host buckets certificates are stuck in "Pending"

Began: Ended: Duration:
  • General
  • External
  • Deploio

Custom host buckets might show a fake TLS cert.

We identified a bug in the certificate provisioning and are working on a fix.

We've fixed the core issue, and are waiting for things to recover.

We've now resolved the incident. Thanks for your patience.

[Complete] Maintenance of the ES34 Virtualization Infrastructure

Began: Ended: Duration:
  • Virtualization

We will be performing scheduled maintenance on our ES34 virtualization environment. No customer impact is expected during this maintenance window.

We appreciate your understanding and will inform you once the maintenance is complete.

The scheduled maintenance is now underway. We'll keep you updated on our progress.

The maintenance is now complete. Thanks for your patience.

No further notices from the past 30 days.