Back

upgrading / restoring a backup / loading fixtures

Backup files in Andes take one of the following two forms: SQL or JSON. The preferred method is to use SQL since this is the fastest and most reliable method. However, one advantage of the JSON backups is that they are completely agnostic of the databases used on the backends. Accordingly, if transferability between databases is important, this might be the format to use. The export format of an Andes instance is specified in the Mission:

Backup File Naming Convention

All backup files, regardless of their format, will have the following basic form: andes_v<MAJOR>.<MINOR>_<SHA>_<DATE>.[FILETYPE].zip, where the [FILETYPE] is either .sql or .json.

The <SHA> represents the git hash of the code responsible for the creation of the backup. This allows one to checkout the same code branch before loading the fixtures and ensures that the database contents matches the source code.

If there is an active mission at the time of backup creation, the mission number will be included in the filename, e.g., andes_v2.2_0f9cbbc5_IML2023027_2023_08_17_17_26_12.json.zip.

If the backup file is SQL, the flavor of SQL (i.e., mysql or sqlite3 ) will be labelled appended to the end of the filename preceded by an underscore. For example, a SQL backup exported from a sqlite3 database would look like this: andes_v2.2_0f9cbbc5_IML2023027_2023_08_17_17_26_12_sqlite3.sql.zip.

SQL Backups

The background process used for dumping the data differs according to which database is being used on the backend of a given instance; resulting in differently flavoured .sql files. Unfortunately these files are not transferable between types of database servers (e.g., mysql to sqlite3) without prior manipulation. This section will outline various processes for re-importing .sql exports.

Restore mysql-flavoured SQL backup into mysql database

1. Create a fresh target database:

   CREATE DATABASE andesdb_restored CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;
   

2. Import content of SQL file into database:

   sudo mysql -h localhost -u root -p andesdb_restored < /path/to/backup.sql
   

Restore sqlite3-flavoured SQL backup into sqlite3 database

1. Import content of SQL file into new database:

   cat /path/to/backup.sql | sqlite3 db_restored.sqlite3 
   

Restore mysql-flavoured SQL backup into sqlite3 database (LINUX ONLY)

1. Download this great tool and make sure it is accessible from the command line (if not already done):

   # make sure that sqlite3 cmd line tool is installed
   sudo apt install sqlite3
   cd ~
   git clone https://github.com/dumblob/mysql2sqlite
   ~/mysql2sqlite/mysql2sqlite backup.sql | sqlite3 db_restored.sqlite3
   

Django JSON Backups

Starting from en empty database, checkout the matching source code version having (for example, SHA=ft5cebd)

git checkout ft5cebd

The fixtures can be loaded with

python manage.py loaddata andes_v3.2_ft5cebd_2023_06_28.json.zip

Then with the data layer initialized, the code can also be updated (as long as the major and minor revisions are the same—see below)

git checkout stage

python manage.py makemigrations
python manage.py migrate

The <MINOR> revision denotes dependency changes (like changes to requirements.txt or settings). This means that re-installation of dependencies should follow the checkout of the source code before the fixtures can be loaded.

pip install -r requirements.txt
pip install -r special_requirements.txt

and/or make needed changes to the .env or other settings. This should be performed every time the source code changes minor revisions (whether upgrading or downgrading).

Lastly, the <MAJOR> version is incremented whenever migrations are squashed —or “crushed”—into a single file per app. This entails that the migrations pertaining to a different “major” version, cannot be applied (i.e., makemigrations and migrate cannot be performed between codes differing in major revisions).