full backup/restore workflow

[radtriguez]

Problem

What is the workflow for a fully fledged data backup?
The API backup/restore endpoint provides just the schematics and projects.

No nodes, no binaries.

Anything special?

Gentics Mesh Version, operating system, or hardware.

• v0.18.x

Operating System

• Linux
• MacOSX

[Jotschi]

The following parts need to be included in a backup:

• GraphDB - This can be backuped on demand via the backup endpoint or via the orientdb backup job using automatic-backup.json. This will automatically create a full backup. Note that this backup will block the application. We also plan to support OrientDB Enterprise and thus allow differential live backup of the graph database.
  The graph database contains all contents (also nodes, projects, roles etc). Except binaries.

• Binary files - A backup job would need to take these files into account by handling the binary files folder (e.g. data/binaryFiles)

Optionally you can also backup the Elasticsearch index but this can be recreated anytime using the GraphDB data.

The restore endpoint is currently a bit tricky. You are required to restart Mesh once the backup has been restored.

[radtriguez]

Thanks for the quick reply.

Using the restore endpoint and including graphDB and the binaryFiles folder works partially.
After restarting Mesh, all data is accessible via API but not visible in the Mesh-UI.

Any tricks?

[Jotschi]

You need to refresh the search index via POST /api/v1/search/sync endpoint - The current UI requires the index for a lot of operation. We are currently addressing this limitation with the development of a new UI.

[radtriguez]

Hi,

so far it's not running.

Worflows

• starting 0.18.1 fresh
• testing /api/v1/search/sync endpoint -> working
• create test project
• add binary
• get token for admin user via curl POST
• using api/v1/admin/graphdb/backup
• starting 0.18.1 fresh on a new machine
• get token for admin user via curl POST
• testing /api/v1/search/sync endpoint on new machine -> working
• copy backup, binaryFiles folders to new machine -> data/
• using api/v1/admin/graphdb/restore on new machine -> working
• restarting mesh on new machine
• get a new token via curl POST on new machine
• trying to sync via /api/v1/search/sync on new machine -> error

Error

from now on it's not possible to access any endpoint anymore.
{
"message": "You are not authorized to access the requested resource."
}

[radtriguez]

After waiting 30min, I have access again and now it's working.

• making a sync via /api/v1/search/sync endpoint
• restarting mesh
• done

[Jotschi]

@radtriguez The sync is executed asynchronously and you can check the state of the sync using /api/v1/search/status endpoint. The next release of Gentics Mesh will drastically increase the sync performance. I'll also add a guide which explains the backup workflow (this is currently missing in the docs). If I find the time I also change the restore endpoint so that no restart is required.

[radtriguez]

@Jotschi nice!

[Jotschi]

@radtriguez 0.18.2 is released. Restart after calling restore is no longer needed and I updated the backup / restore docs. https://getmesh.io/docs/beta/administration-guide.html#_backup_recovery