Markertplace APIs and Integration APIs are unresponsive

Incident Report for Sharetribe

Postmortem

Today, Friday 29th Dec at 4:20 UTC - 5:22 UTC we had a major system wide outage, affecting all the APIs, Console and CLI. At 5:22 UTC the system became partially operational. At 5:34 the system because fully operational.

At 4:21 UTC our team was alerted and started immediate investigation. We identified a memory issue in our main database and after identifying it, the issue was resolved.

To prevent this happening in the future, we took actions and allocated more resources to our database instances.

In addition, we will continue investigating the root cause and we will implement additional alert system to identify the issue quicker in the future.

Posted Dec 29, 2023 - 08:14 UTC

Resolved

This incident has been resolved.

Posted Dec 29, 2023 - 08:11 UTC

Update

We are continuing to monitor for any further issues.

Posted Dec 29, 2023 - 06:05 UTC

Update

We are continuing to monitor for any further issues.

Posted Dec 29, 2023 - 05:30 UTC

Monitoring

The APIs are up and running again; we will continue investigating the root cause and monitoring the system.

Posted Dec 29, 2023 - 05:29 UTC

Investigating

We are currently investigating an issue with Marketplace API and Integration APIs being unresponsive.

Posted Dec 29, 2023 - 04:39 UTC

This incident affected: Marketplace API, Integration API, and Authentication API.