Data migration across different machines

Data migration across different machines is achieved by mongodump and mongorestore. The process should work pretty seamlessly as long as the mongo versions are the same in the source and destination environment. The running versions of Mongo can be easily checked, e.g., with docker ps.

Data Migration Steps (all data)

The steps below assumes the default naming of the Mongo volume, i.e., deploy_askcosv2_mongo_data. If it has been configured to a different name, please modify accordingly in deploy.sh. The migration process only makes use of 2 subroutines: mongodump(), mongorestore().

Step 1. Save the Mongo dump on the source server

$ cd askcos2_core
$ bash deploy.sh mongodump -d /absolute/dir/on/src

The services on the source machine do not need to be stopped. Once the script finishes execution, you will have the compressed Mongo dump at /absolute/dir/on/src/mongo_dump.gz.

Step 2. Copy the dump file from the source server to the destination server

$ scp /absolute/dir/on/src/mongo_dump.gz ztu@dest-machine:/absolute/dir/on/dest/mongo_dump.gz

Step 3. Load the Mongo dump on the destination server

Note: you might want to back up the data before overwriting.

$ cd askcos2_core
$ make stop
$ bash deploy.sh mongorestore -d /absolute/dir/on/dest

It's important to stop all services accessing Mongo (with make stop) on the destination machine. Once the script finishes execution, the user data would have been migrated on the destination server. You can then restart ASKCOS with

$ make start

Data Migration Steps (results and users only)

The steps below assumes the default naming of the Mongo volume, i.e., deploy_askcosv2_mongo_data. If it has been configured to a different name, please modify accordingly in deploy.sh. The migration process only makes use of 4 subroutines: mongodump-result-only(), mongodump-user-only(), mongorestore-result-only(), mongorestore-user-only().

Step 1. Save the Mongo dump on the source server

$ cd askcos2_core
$ bash deploy.sh mongodump-result-only -d /absolute/dir/on/src
$ bash deploy.sh mongodump-user-only -d /absolute/dir/on/src

The services on the source machine do not need to be stopped. Once the script finishes execution, you will have the compressed Mongo dump at /absolute/dir/on/src/mongo_dump_{results,users}.gz.

Step 2. Copy the dump file from the source server to the destination server

$ scp /absolute/dir/on/src/mongo_dump_results.gz ztu@dest-machine:/absolute/dir/on/dest/mongo_dump_results.gz
$ scp /absolute/dir/on/src/mongo_dump_users.gz ztu@dest-machine:/absolute/dir/on/dest/mongo_dump_users.gz

Step 3. Load the Mongo dump on the destination server

Note: you might want to back up the data before overwriting.

$ cd askcos2_core
$ make stop
$ bash deploy.sh mongorestore-result-only -d /absolute/dir/on/dest
$ bash deploy.sh mongorestore-user-only -d /absolute/dir/on/dest

It's important to stop all services accessing Mongo (with make stop) on the destination machine. Once the script finishes execution, the user data would have been migrated on the destination server. You can then restart ASKCOS with

$ make start