PostgreSQL v18 updates; Database restores#720
Conversation
While this is a bit funky to shoehorn this into this Python app, we have other components of our deployment process that use this library to provide easy database restores, so by leveraging this library, this is the easiest way to integrate that process with this app.
Since this is an optional container only used in local development, shift it to a profile so that it doesn't start by default. This container will only work for NLR developers, so we don't want it starting/building by default in the CI environment or for other users.
Also update the nojulia docker compose variant to align with the main docker compose file that uses the default postgres superuser now (to better align with other environments).
|
@GUI when trying to do "docker compose run --rm db-syncer rake db:data:restore" locally, I get SSL certificate problem: self-signed certificate in certificate chain. |
|
Thoughts on why this deploy failed? Was this deployed with deploy-db-restore label? I'm not sure how I can tell when it's removed after. I was trying to GET a run_uuid result from the production db that happened within the last 15 days through the staging pg-v18 branch, but it said the run_uuid was not in the db. I do know we had a migration added after the May 15th deploy where you had to manually fix some migrations for the db snapshot, so we will need to do what you did before again. |
|
I just wanted to confirm that the celery (+julia) and julia pods are restarting nightly still. It doesn't look like they are based on the age of the pods there. Where if I compare to the master branch on staging, the pods seem to have restarted at ~1am as expected. |
Bill-Becker
left a comment
Bill-Becker
left a comment
There was a problem hiding this comment.
This looks good, but I left some comments separately/outside of this review to follow up on.
If the user doesn't have CREATE privileges at the database level, then even if the schema already exists, the `CREATE SCHEMA IF NOT EXISTS` will fail.
Ah, I forgot to document this, but you'll need a
🤦♂️ My bad... I made a final commit to fix the CI test suite, but then I missed that inadvertently broke the deploys. This should be fixed by 513c114, though, so deploys should be running again. Regarding the But happy to chat more about any of this if there are still any questions.
I believe only |
Align everything on the `reopt` database name as the default. This matches the other `DB_NAME` environment variables being passed into the Celery and Django containers.
Bill-Becker
left a comment
Bill-Becker
left a comment
There was a problem hiding this comment.
We've worked though everything and I understand the updated process. Thanks!
2 participants
This builds upon the deployment updates in #714, but with 2 main additions:
docker compose run --rm db-syncer rake db:data:restoredeploy-db-restorelabel to the deployment PR, then on the next deployment, it will restore with the latest snapshot once (the label will then be removed, but you can add it again to trigger another restore at a later date).docker compose run --rm db-syncerlocally and not worry about Ruby or anything else).