Resolved
Following additional monitoring, no further errors in webhook FIFO event deliveries were observed. The incident was marked resolved.
Monitoring
Our engineering team made necessary updates to the database configurations to rectify the issue.
Identified
Issue was identified that the automated backup configuration had impacted the database, leading to failures in webhook delivery.
Investigating
A network monitoring alert revealed a decrease in webhook delivery rates, specifically impacting FIFO events, prompting an investigation by our Engineering team.
Investigating
Webhook delivery disruption started. An issue with the real-time monitoring/alerting system caused a delay in identification by an hour.
