Investigating a potential outage

Incident Report for WorldTicket Operational Status

Postmortem

Incident start: 15:39 CET
Incident end: 16:31 CET
Duration: 53 minutes
Database connection pool exhaustion for user-management-service due to misconfiguration led to very high failure rate. This led to situation when all http threads for customer instances were busy waiting for response from user-management-service.
Initially this was identified as DDOS or huge spike of load. Further investigation showed that service exhausted its database connection pool and after failing to obtain connection from a pool for 20 seconds was returning error response.
Root-cause – due to misconfiguration, service was using default values for database connection pool size instead of explicitly configured size.

Best Regards,
Worldticket Team

Posted Sep 08, 2021 - 16:10 CEST

Resolved

This incident has been resolved.

Posted Sep 08, 2021 - 14:32 CEST

Monitoring

A fix has been implemented and we monitoring results

Posted Sep 07, 2021 - 17:35 CEST

Update

We are continuing to investigate this issue.

Posted Sep 07, 2021 - 15:54 CEST

Investigating

Dear Customers,

We are currently investigating reports of a potential service interruption. We apologize for any inconvenience and will post another update as soon as we learn more.

Kind regards ,

Worldticket Team

Posted Sep 07, 2021 - 15:54 CEST

This incident affected: SMS Modules (Reservations, Queues, Pricing, Payment & Ticketing, Inventory & Schedules, Security (Roles & Users), Reporting, Localization).