Thursday, November 8

Basecamp 3 had trouble for about 5 hours.


We've verified that all data was safe and intact across all databases. We brought Basecamp back online, but are still seeing intermittent issues dealing with the load. Should not be much longer.

Basecamp 3 is still in read-only mode as we finalize the verification of the 8 databases we have that runs across 2 data centers. We need 100% confidence that all data is in proper sync. We're almost there, but almost doesn't count. We will update more rapidly as we get updates from the operations team.

Work is still ongoing to make sure that all the data is as it should be for Basecamp 3. The database migration has finished, but the verification process is still ongoing. We're working as fast as we can and hope to be back fully shortly.

We have now completed the database change to fix the underlying problem that forced us into read-only mode today. Our ops team is working to verify all the data is safe, and make the necessary configuration changes to bring us back online. We will continue to keep you posted until everything is in order, and you will get a full, detailed report once it's over. Again, we humbly ask for a little more of your patience.

Basecamp 3 continues to be in read-only mode. We're in the homestretch on the database change to fix the underlying issue with some twenty minutes remaining. There's still work to verify that all data is safe after that, as well as other configuration changes. But we look to be getting close to the end. We ask your patience, and are so sorry for the interruption caused over the last several hours.

Basecamp 3 is still in read-only mode. Our current estimate for when we will be back has moved to 11:30am CST (17:30 UTC). We're continuing to keep everyone updated on the longer write-up as well. Again, we are deeply sorry for interrupting your day like this. We know that you depend on Basecamp. We're doing everything we can to get back as soon as possible.

Basecamp 3 remains in read-only mode. We've summarized everything we know and have been doing so far in this write-up. We estimate that we will be back to normal in just about an hour. We are so sorry for this prolonged outage!

Basecamp 3 is still in read-only mode. That means you can't change anything in your account (no new messages, moving todos, etc). We've identified the problem, and the fix is being applied. We currently estimate to be back online between 10:30-11:00AM CST (GMT+7).

We're still investigating a fix for Basecamp 3. We'll update here and on Twitter as we resolve, but we believe it'll be about 2 hours more until we have a fix (10:35am CST).

Basecamp 3 is still in read-only mode. Our ops team is working on a fix, and will keep you updated!

Basecamp 3 is in read only mode right now. We are online and working to get the site back fully online.

Go back to Basecamp Status