migrate mesh 0.22.0 to 0.27.2 #546

lilian-benoit · 2018-11-13T22:54:47Z

Gentics Mesh Version, operating system, or hardware.

v0.22.0 (old version)
v0.27.2 (new version)
I have two instances but i will migrate one first instance.

Operating System

Linux 64bit

JVM

Oracle JDK 1.8.0_60

Problem

orientdb haven't migrate after startup. It see releaseimpl in orientdb's studio. But normally Releases is renamed to Branches (v0.23.0)

Reproducer

I have backuped directory graphdb
I have started mesh with new war

2018-11-13 08:41:35:219 WARNI Requested command 'Committing the active transaction to create the new type 'BranchRootImpl' as subclass of 'V'. The transaction will be reopen right after that. To avoid this behavior create the classes outside the transaction' must be executed outside active transaction: the transaction will be committed and reopen right after it. To avoid this behavior execute it outside a transaction [OrientGraph]

To be sure, i stopped mesh
I have restored directory graphdb
I have started mesh with new war.

mesh's log => https://framabin.org/p/?9fc41ac37d0e4b39#+Q1+NqMQXtKBoq/oDNIdWJD0o1jZXyNquZI5A54zYiI=
orientdb's log => https://framabin.org/p/?5593e50b86b3f321#QNj+pOMGH1zPd9P+A2zeXhr0lbcE4ObBLJLYKJ0DBoI=

Expected behaviour and actual behaviour

orientdb should be migrate.
I should see branchimpl in orientdb's studio

I launched endpoint /api/v1/admin/consistency/check. Result is InternalServeError.

2018-11-13 22:16:08.662 [app1_cms0] ERROR [vert.x-eventloop-thread-2] - Error for request in path: /api/v1/admin/consistency/check
2018-11-13 22:16:08.663 [app1_cms0] ERROR [vert.x-eventloop-thread-2] - Error:
java.lang.IllegalArgumentException: Cannot find class 'BranchRootImpl' in database schema
        at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getVerticesOfClass(OrientBaseGraph.java:729) ~[mesh-server-0.27.2.jar:na]
        at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getVerticesOfClass(OrientBaseGraph.java:713) ~[mesh-server-0.27.2.jar:na]
        at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.getVertices(OrientBaseGraph.java:750) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.graphdb.OrientDBDatabase.getVerticesForType(OrientDBDatabase.java:639) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.core.endpoint.admin.consistency.check.BranchCheck.invoke(BranchCheck.java:30) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.core.endpoint.admin.consistency.ConsistencyCheckHandler.lambda$invokeCheck$0(ConsistencyCheckHandler.java:76) ~[mesh-server-0.27.2.jar:na]
        at com.gentics.mesh.graphdb.spi.Database.lambda$null$6(Database.java:175) ~[mesh-server-0.27.2.jar:na]
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:273) ~[mesh-server-0.27.2.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_60]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_60]

The text was updated successfully, but these errors were encountered:

Jotschi · 2018-11-15T18:32:43Z

As mentioned in the gitter channel. Please execute the following steps:

Stop the server
Copy the backup data of version 0.22.x into the data folder
Disable clustering in the config
Start Mesh 0.27.x
Send log to me

lilian-benoit · 2018-11-15T23:31:54Z

Migrate tasks appears in a log file. But i have a error

2018-11-16 00:23:13.673 [app1_cms0] INFO [main] - Changelog completed.
2018-11-16 00:23:13.673 [app1_cms0] INFO [main] - Updating stored database revision and mesh version.
2018-11-16 00:23:13.701 [app1_cms0] INFO [main] - Creating database indices. This may take a few seconds...
2018-11-16 00:23:15.682 [app1_cms0] ERROR [main] - Error while starting mesh
com.orientechnologies.orient.core.storage.ORecordDuplicatedException: Cannot index record #62:72: found duplicated key 'OCompositeKey{keys=[8f1ac513a0af4f8e9ac513a0afdf8e29, D, ]}' in index 'e.has_field_container_webrootpathinfoindex' previously assigned to the record #62:1
DB name="storage"
at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:47) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:37) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.doPut(OLocalHashTable.java:1539) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.put(OLocalHashTable.java:1459) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.validatedPut(OLocalHashTable.java:412) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.engine.OHashTableIndexEngine.validatedPut(OHashTableIndexEngine.java:156) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doValidatedPutIndexValue(OAbstractPaginatedStorage.java:2519) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.validatedPutIndexValue(OAbstractPaginatedStorage.java:2498) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:82) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:35) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.populateIndex(OIndexAbstract.java:940) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.indexCluster(OIndexAbstract.java:1040) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.fillIndex(OIndexAbstract.java:539) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexAbstract.create(OIndexAbstract.java:250) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:129) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:40) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexManagerShared.createIndex(OIndexManagerShared.java:171) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.index.OIndexManagerProxy.createIndex(OIndexManagerProxy.java:87) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1900) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1870) ~[mesh-server-0.27.2.jar:na]
at com.orientechnologies.orient.core.metadata.schema.OClassAbstractDelegate.createIndex(OClassAbstractDelegate.java:397) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType.access$701(OrientElementType.java:35) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType$8.call(OrientElementType.java:123) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType$8.call(OrientElementType.java:120) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.executeOutsideTx(OrientBaseGraph.java:1771) ~[mesh-server-0.27.2.jar:na]
at com.tinkerpop.blueprints.impls.orient.OrientElementType.createIndex(OrientElementType.java:120) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.graphdb.OrientDBDatabase.addCustomEdgeIndex(OrientDBDatabase.java:546) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.core.data.impl.GraphFieldContainerEdgeImpl.init(GraphFieldContainerEdgeImpl.java:52) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.core.data.impl.DatabaseHelper.init(DatabaseHelper.java:80) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.BootstrapInitializerImpl.initLocalData(BootstrapInitializerImpl.java:223) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.BootstrapInitializerImpl.init(BootstrapInitializerImpl.java:296) ~[mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.MeshImpl.run(MeshImpl.java:134) [mesh-server-0.27.2.jar:na]
at com.gentics.mesh.cli.MeshImpl.run(MeshImpl.java:98) [mesh-server-0.27.2.jar:na]
at com.gentics.mesh.server.ServerRunner.main(ServerRunner.java:55) [mesh-server-0.27.2.jar:na]
2018-11-16 00:23:15.682 [app1_cms0] INFO [main] - Mesh shutting down...
2018-11-16 00:23:16.287 [app1_cms0] INFO [main] - Closing Elasticsearch REST client.
2018-11-16 00:23:16.288 [app1_cms0] INFO [main] - Stopping Elasticsearch server.
2018-11-16 00:23:17.288 [] INFO [main] - Shutdown completed...
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Mesh shutting down...
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Closing Elasticsearch REST client.
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Stopping Elasticsearch server.
2018-11-16 00:23:36.288 [] INFO [Thread-0] - Shutdown completed...

Mesh stopped after migration successfully

lilian-benoit · 2018-11-15T23:34:19Z

url for complete log

https://framabin.org/p/?7f1c2745093707de#Y21UDoWJSavQMYG4J9WpMg3A1i6HEYfpuMsuR7Q3JwU=

A+,
Lilian.

lilian-benoit · 2018-11-20T22:52:57Z

in gitter, it is three options

Restore data of 0.22.x and start the latest 0.22.x version and try to repair the inconsistency and re-run the upgrade to 0.24.x -
Import the dump into a dedicated OrientDB studio
Dump all the data of 0.22.x via an export script using the API and re-import it into 0.27.x

lilian-benoit · 2018-11-20T23:01:46Z

I try first option.

I restored data of 0.22.x but database is consistency (without repair)
I upgraded to 0.24.x. I have the same error

Jotschi · 2018-11-21T00:07:25Z

@lilian-benoit I was able to reproduce the issue. The problem happens if you have at least two nodes in your system which have an empty segment field. This can happen if you:

Create a schema with two field (name, name2) - (No segment field selected, name2 is the display field)
Create two nodes of this schema in which only name2 field set (name field is left empty)
Update the schema and set the segment field to "name". Nodes will be migrated but the field remains empty.

It may also be possible to produce this issue if the segment field is not set to required and just left empty.

If you upgrade this data state to 0.24.x you will see the error. It should however be possible to update the nodes (draft and published versions) and set the missing value in version 0.22.x - If you update the nodes the error should not happen when launching 0.24.x.

I however have not yet confirmed this in my setup. I'll also work on a fix to resolve this problem in 0.24.x.

lilian-benoit · 2018-11-21T09:08:36Z

if my understand is good, i must verify value of segment field of all my nodes

Jotschi · 2018-11-21T16:15:55Z

@lilian-benoit Right - in both versions: draft and published

lilian-benoit · 2018-11-24T20:09:52Z

@Jotschi Two document was on error with segment field null and all folder (same root folder). I updated all nodes but I had same error on migration

Jotschi · 2018-11-25T00:31:41Z

@lilian-benoit In that case the error might still be happening due to older version which will also be migrated and still have the "wrong" data in them. I'll add a dedicated test for this case and see whether that reveals the issue.

You could try to move your nodes around and delete the affected nodes and see whether that resolves the issue.

Jotschi · 2018-11-25T00:35:16Z

@philippguertler ~~I think the error happens because I changed the webroot index mechanism from a simple single key of the NodeImpl to a composite index on HAS_FIELD_CONTAINER edge.~~

mesh/core/src/main/java/com/gentics/mesh/core/data/impl/GraphFieldContainerEdgeImpl.java

Line 52 in a59482c

    
           db.addCustomEdgeIndex(HAS_FIELD_CONTAINER, WEBROOT_INDEX_POSTFIX_NAME, fields, true);

Now there are there are 2 components of the index non-null and the last one is null. Before there was only a single key which was null and the index was setup to ignore null values. For composite indices this is no longer possible. This change could potentially also cause problems on new webroot indices.

Jotschi · 2018-11-26T10:20:27Z

@lilian-benoit I fixed a bug in the changelog entry that was supposed to update the database. The fix will be included in the next release.

Jotschi self-assigned this Nov 15, 2018

Jotschi mentioned this issue Nov 21, 2018

Crash on Startup #554

Closed

Jotschi added the f/webroot label Nov 23, 2018

Jotschi closed this as completed in 5dedb60 Nov 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

migrate mesh 0.22.0 to 0.27.2 #546

migrate mesh 0.22.0 to 0.27.2 #546

lilian-benoit commented Nov 13, 2018

Jotschi commented Nov 15, 2018

lilian-benoit commented Nov 15, 2018

lilian-benoit commented Nov 15, 2018

lilian-benoit commented Nov 20, 2018

lilian-benoit commented Nov 20, 2018 •

edited

Loading

Jotschi commented Nov 21, 2018

lilian-benoit commented Nov 21, 2018

Jotschi commented Nov 21, 2018

lilian-benoit commented Nov 24, 2018

Jotschi commented Nov 25, 2018

Jotschi commented Nov 25, 2018 •

edited

Loading

Jotschi commented Nov 26, 2018

migrate mesh 0.22.0 to 0.27.2 #546

migrate mesh 0.22.0 to 0.27.2 #546

Comments

lilian-benoit commented Nov 13, 2018

Gentics Mesh Version, operating system, or hardware.

Operating System

JVM

Problem

Reproducer

Expected behaviour and actual behaviour

Jotschi commented Nov 15, 2018

lilian-benoit commented Nov 15, 2018

lilian-benoit commented Nov 15, 2018

lilian-benoit commented Nov 20, 2018

lilian-benoit commented Nov 20, 2018 • edited Loading

Jotschi commented Nov 21, 2018

lilian-benoit commented Nov 21, 2018

Jotschi commented Nov 21, 2018

lilian-benoit commented Nov 24, 2018

Jotschi commented Nov 25, 2018

Jotschi commented Nov 25, 2018 • edited Loading

Jotschi commented Nov 26, 2018

lilian-benoit commented Nov 20, 2018 •

edited

Loading

Jotschi commented Nov 25, 2018 •

edited

Loading