Incidents reported on status page for Productlane https://status.productlane.com/ https://d1lppblt9t2x15.cloudfront.net/logos/d78b134d7e90f69467c24db631f782a2.png https://status.productlane.com/ en https://status.productlane.com/incident/845286 Mon, 09 Mar 2026 20:20:00 -0000 https://status.productlane.com/incident/845286#32b59b2e3736ecbfe724691c17a0bad76bb672220ee7dd48c0d6a005ff8387cf Summary Earlier today we experienced a service disruption after deploying a database migration related to email handling. The migration triggered a failure in a background subprocessor responsible for synchronizing database views used by the Productlane client. This caused a schema mismatch between the client and server and resulted in errors in the Productlane interface. No data was lost during the incident. All incoming events and messages were safely queued and processed once the system recovered. The issue was fully resolved at 9:12 PM. We are terribly sorry for the experience this caused. Timeline 1:10 PM We deployed a database migration related to emails. 1:14 PM The migration attempted to create a unique index that included a nullable field. The database rejected this because unique indexes cannot be created on nullable columns in this configuration. 1:18 PM We removed the index and redeployed the migration. 1:27 PM Shortly after the migration, one of our background subprocessors responsible for syncing database views began failing. This component runs as part of our infrastructure built on Zero and keeps client-side views in sync with the database. 2:05 PM Because the view synchronization process failed, the system entered an inconsistent state where the server schema had changed but the client schema had not updated accordingly. This resulted in frontend errors due to the schema mismatch. 3:12 PM We investigated the failure and identified that the issue originated inside the view synchronization layer. The underlying problem was difficult to debug because the failure occurred inside a core infrastructure component and was not directly visible from the migration itself. 4:36 PM We prepared a mitigation by restoring compatibility between the client and server schema. 6:18 PM We deployed the fix which downgraded the client schema to match the server state again, restoring compatibility and resolving the frontend errors. The application was back up, but was not able to write data, still due to the schema mismatch. 6:48 PM We decided to wipe and rebuild the entire Replica database, which we wanted to avoid, as it can take hours to rebuild. 9:12 PM All services are restored. Root cause The incident was caused by a combination of a database migration that introduced an invalid index configuration and a bug in the view synchronization layer running on top of Zero. While the migration itself was routine, it triggered a failure in this subsystem that caused the view sync workers to stop processing updates. Because the client views rely on this system to stay in sync with the database, the result was a schema mismatch between client and server. What we will improve We are taking several steps to reduce the likelihood and impact of similar incidents in the future. We will add more redundancy to our infrastructure, ensuring that failures in individual subprocessors or sync workers cannot interrupt the system. In addition, we are speeding up our deployment pipeline and introducing instant rollback capabilities so that problematic deployments can be reverted immediately. We again sincerely apologize for the disruption and appreciate your patience. https://status.productlane.com/incident/845286 Mon, 09 Mar 2026 18:19:00 -0000 https://status.productlane.com/incident/845286#809e345c0e5169f2721f869c5fc2c3df03a056f2c8279012fbdf4dd8a4b0732a After several attempts, we have identified the root cause of the issue and are deploying a fix https://status.productlane.com/incident/845286 Mon, 09 Mar 2026 14:22:00 -0000 https://status.productlane.com/incident/845286#4607033092fe56a9711142d27a37c8d6bd5977316fc386b74e15b312cb57b643 We’re currently experiencing a temporary service disruption that may affect parts of Productlane. Our team is investigating the issue and working to restore full service as quickly as possible. We’ll share another update as soon as we have more information. You can also follow the current status at our status page if needed. Thanks for your patience while we resolve this. https://status.productlane.com/ Fri, 06 Mar 2026 02:46:53 +0000 https://status.productlane.com/#54bddd68fbb2800ec670e8626f9f6a8ed2841eeb3ee6c8ba2cf08c27c273afee Productlane Core & API recovered https://status.productlane.com/ Fri, 06 Mar 2026 02:44:11 +0000 https://status.productlane.com/#54bddd68fbb2800ec670e8626f9f6a8ed2841eeb3ee6c8ba2cf08c27c273afee Productlane Core & API went down https://status.productlane.com/incident/610139 Fri, 27 Jun 2025 10:36:00 -0000 https://status.productlane.com/incident/610139#9af8e06b38a37932f358ab86b8abaebf8a86220215f0e479d94f7a448f546c3c Productlane experienced approximately one hour of downtime due to our primary database reaching its allocated storage limit. An expected automatic storage scaling process in AWS failed because of a misconfiguration, which prevented the database from expanding storage as needed. • Service was unavailable for approximately 60 minutes • No data was lost • All queued emails have been preserved and will be delivered • All queued requests were processed after recovery We are truly sorry for this incident and the disruption it caused. We are putting additional safeguards in place to prevent similar issues in the future. If you have any concerns or questions, please reach out to our team at hello@productlane.com! https://status.productlane.com/incident/374522 Sat, 25 May 2024 09:00:00 -0000 https://status.productlane.com/incident/374522#6a10aa081c10da0b347af7451ea98f9af77fe5d2d2913094616078f5d06aadca Maintenance completed https://status.productlane.com/incident/374522 Sat, 25 May 2024 08:00:00 -0000 https://status.productlane.com/incident/374522#1438c31677ef036d143288f8eba03cbf272bef69ce01ef8942d02d1c2c328b53 Some DNS changes are necessary which could disrupt service temporarily.