Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply optimizations to queries to make use of adjacent ids #1961

Conversation

rngcntr
Copy link
Contributor

@rngcntr rngcntr commented Feb 14, 2020

This feature was originally part of #1937 which was splitted to decrease per-PR-complexity.

This PR introduces two new OptimizerStrategies, which further improve the existing optimizations based on adjacent vertex ids. In the current state, these queries will be optimized to check the adjacency on the edge before iterating to the vertex itself:

  • g.V(left).outE('label').where(__.inV().is(right))
  • g.V(left).outE('label').filter(__.inV().is(right))

The optimization of these queries is within the scope of #1960:

  • g.V(left).outE('label').where(__.inV().hasId(right))
  • g.V(left).outE('label').filter(__.inV().hasId(right))

With the new optimizer, these queries will also profit of optimization:

  • g.V(left).outE('label').inV().is(right)
  • g.V(left).outE('label').inV().hasId(right.id())
  • g.V(left).out('label').is(right)
  • g.V(left).out('label').hasId(right.id())
  • g.V(left).out('label').filter(is(right))
  • g.V(left).out('label').where(is(right))
  • g.V(left).out('label').filter(hasId(right.id()))
  • g.V(left).out('label').where(hasId(right.id()))

All of this also applies to all other valid variations of in()/out()/both().

For all changes:

  • Is there an issue associated with this PR? Is it referenced in the commit message?
  • Does your PR body contain #xyz where xyz is the issue number you are trying to resolve?
  • Has your PR been rebased against the latest commit within the target branch (typically master)?
  • Is your initial contribution a single, squashed commit?

For code changes:

  • Have you written and/or updated unit tests to verify your changes?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE.txt file, including the main LICENSE.txt file in the root of this repository?
  • If applicable, have you updated the NOTICE.txt file, including the main NOTICE.txt file found in the root of this repository?

@janusgraph-bot janusgraph-bot added the cla: external Externally-managed CLA label Feb 14, 2020
@rngcntr rngcntr force-pushed the feature/optimize-queries-with-adjacent-ids branch from de39e9c to a9c91c5 Compare February 14, 2020 09:53
@JanusGraph JanusGraph deleted a comment from rngcntr Feb 14, 2020
@JanusGraph JanusGraph deleted a comment from rngcntr Feb 14, 2020
@rngcntr rngcntr force-pushed the feature/optimize-queries-with-adjacent-ids branch from a9c91c5 to fd0e777 Compare February 17, 2020 09:28
@JanusGraph JanusGraph deleted a comment from rngcntr Feb 17, 2020
@JanusGraph JanusGraph deleted a comment from rngcntr Feb 17, 2020
@porunov
Copy link
Member

porunov commented Feb 20, 2020

@rngcntr Thank you very much for new optimizations! Could you please rebase your PR against the latest master commit to resolve conflicts?

@rngcntr
Copy link
Contributor Author

rngcntr commented Feb 20, 2020

@porunov I will do so as soon as I'm finally able to make all tests pass

@rngcntr rngcntr force-pushed the feature/optimize-queries-with-adjacent-ids branch 2 times, most recently from c963c0f to aa674d8 Compare February 21, 2020 07:50
@rngcntr
Copy link
Contributor Author

rngcntr commented Feb 24, 2020

So I've been working on this and there still remains one case which results in a flaky test. As I don't think I'm able to solve this problem myself, I'm looking for help here.
There's a test case which creates the following traversal:

Traversal t2 = gts.V(vs[0], vs[1], vs[2])
.repeat(__.inE("knows")
.outV()
.hasId(sv[0].id())
.outE("knows")
.inV()
.sideEffect(e -> loop2[0] = e.loops())
.has("id", loop2[0]))
.times(numV);

Because this traversal matches one of the given patterns, when applied, my strategy should rewrite it to look like this:

Traversal t1 = gts.V(vs[0], vs[1], vs[2])
.repeat(__.inE("knows")
.has("~adjacent", sv[0].id())
.outV()
.outE("knows")
.inV()
.sideEffect(e -> loop1[0] = e.loops())
.has("id", loop1[0]))
.times(numV);

And as expected, it does so deterministically. But when applying all optimization strategies one after another, some of the built-in TinkerPop strategies cause both query plans to diverge:

Traversal Explanation [ t1.explain() ]
==========================================================================================================================================================================================================================================================================================================================================================
Original Traversal                          [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]

ConnectiveStrategy                    [D]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
MatchPredicateStrategy                [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
FilterRankingStrategy                 [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
InlineFilterStrategy                  [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
IncidentToAdjacentStrategy            [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentToIncidentStrategy            [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
EarlyLimitStrategy                    [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
RepeatUnrollStrategy                  [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
PathRetractionStrategy                [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
CountStrategy                         [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
LazyBarrierStrategy                   [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentVertexHasIdOptimizerStrategy  [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentVertexFilterOptimizerStrategy [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentVertexIsOptimizerStrategy     [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), VertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
JanusGraphLocalQueryOptimizerStrategy [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
JanusGraphStepStrategy                [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
JanusGraphIoRegistrationStrategy      [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
ProfileStrategy                       [F]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)])@[~gremlin.incidentToAdjacent], EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
StandardVerificationStrategy          [V]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]

Final Traversal                             [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],vertex), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]

Traversal Explanation [ t2.explain() ]
=====================================================================================================================================================================================================================================================================================================================================================================
Original Traversal                          [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]

ConnectiveStrategy                    [D]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
MatchPredicateStrategy                [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
FilterRankingStrategy                 [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
InlineFilterStrategy                  [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
IncidentToAdjacentStrategy            [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentToIncidentStrategy            [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
EarlyLimitStrategy                    [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
RepeatUnrollStrategy                  [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
PathRetractionStrategy                [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
CountStrategy                         [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
LazyBarrierStrategy                   [O]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), EdgeVertexStep(OUT), HasStep([~id.eq(110688)]), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentVertexHasIdOptimizerStrategy  [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentVertexFilterOptimizerStrategy [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
AdjacentVertexIsOptimizerStrategy     [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([VertexStep(IN,[knows],edge), HasStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), VertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
JanusGraphLocalQueryOptimizerStrategy [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
JanusGraphStepStrategy                [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
JanusGraphIoRegistrationStrategy      [P]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
ProfileStrategy                       [F]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]
StandardVerificationStrategy          [V]   [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]

Final Traversal                             [GraphStep(vertex,[v[4312], v[8408], v[4288]]), RepeatStep([JanusGraphVertexStep([~adjacent.eq(110688)]), EdgeVertexStep(OUT), JanusGraphVertexStep(OUT,[knows],edge), EdgeVertexStep(IN), NoOpBarrierStep(2500), LambdaSideEffectStep(lambda), HasStep([id.eq(0)]), RepeatEndStep],until(loops(100)),emit(false))]

The differences are introduced by two strategies.

  • IncidentToAdjacentStrategy (adding the label ~gremlin.incidentToAdjacent to the first HasStep of t1 which does not happen when evaluating t2 and assumably more important not replacing JanusGraphVertexStep(OUT,[knows],edge), EdgeVertexStep(IN) by JanusGraphVertexStep(OUT, [knows], vertex) in t2)
  • LazyBarrierStrategy (adding a NoOpBarrierStep(2500) to the second query plan while not doing so for the first one.

I found out the presence of NoOpBarrierSteps depends on the number of preceeding steps, which has to be at least 5 (some magic number introduced by TinkerPop). The reason for the different number of steps in both queries is the behaviour of the IncidentToAdjacentStrategy, which fails to replace the matching steps in the second query.

I'm currently evaluating why IncidentToAdjacentStrategy behaves differently on both queries.

@rngcntr rngcntr force-pushed the feature/optimize-queries-with-adjacent-ids branch 3 times, most recently from 0c63a36 to 8d5093e Compare February 24, 2020 14:19
@rngcntr
Copy link
Contributor Author

rngcntr commented Feb 24, 2020

Update: We tracked the inconsistencies down to a TinkerPop Bug

None of the failed tests were directly caused by the newly implemented optimization strategies but yet they have randomly led the way to discover this bug. I have rewritten the affected queries to avoid this bug in the tests and as far as I can tell, all tests are passing now.

Copy link
Member

@porunov porunov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @rngcntr ! Looks good to me. I've checked the tests, they are failing without optimizations as expected.
I have only several nitpick comments. Otherwise looks very good!

@porunov porunov added this to the Release v0.5.0 milestone Feb 25, 2020
@rngcntr rngcntr force-pushed the feature/optimize-queries-with-adjacent-ids branch from 8d5093e to 422a840 Compare February 25, 2020 06:14
@rngcntr
Copy link
Contributor Author

rngcntr commented Feb 25, 2020

I see my intensive search for bugs has left it's traces in the code :D
I hope these are all gone now as I fixed all of your nitpicks.

@rngcntr rngcntr force-pushed the feature/optimize-queries-with-adjacent-ids branch from 422a840 to 0f7af8d Compare February 25, 2020 13:38
@FlorianHockmann
Copy link
Member

Codacy Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- janusgraph-core/src/main/java/org/janusgraph/graphdb/tinkerpop/optimize/AdjacentVertexOptimizerStrategy.java  7
- janusgraph-core/src/main/java/org/janusgraph/graphdb/tinkerpop/optimize/AdjacentVertexHasIdOptimizerStrategy.java  3
- janusgraph-core/src/main/java/org/janusgraph/graphdb/tinkerpop/optimize/AdjacentVertexIsOptimizerStrategy.java  2
         

See the complete overview on Codacy

Copy link
Member

@porunov porunov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you @rngcntr !

Copy link
Contributor

@mad mad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All TP tests passed for BerkeleyJanusGraphProcessTest

Only one failed org.apache.tinkerpop.gremlin.process.traversal.step.map.ProfileTest#testProfileStrategyCallback but on master too

@porunov porunov merged commit 7297adb into JanusGraph:master Mar 3, 2020
@porunov
Copy link
Member

porunov commented Mar 3, 2020

@mad Last time I was executing TP tests during the update of TinkerPop to 3.4.6 version. During that time all TP tests passed. This means that we introduced a bug later after that update. We need to investigate where we introduced that problem to fix it.
Will you be able to open an issue to track this bug?

@porunov
Copy link
Member

porunov commented Mar 4, 2020

@mad All TP tests passed on my environment for the master commit 7297adb
I have used the next command to execute TP tests: mvn clean install -Dtest.skip.tp=false -DskipTests=true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: external Externally-managed CLA kind/performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants