Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On leader switchover preserve the vxlan id for existing networks #1773

Merged
merged 1 commit into from
Nov 29, 2016

Conversation

sanimej
Copy link
Contributor

@sanimej sanimej commented Nov 28, 2016

On a leader change, the new leader restores the state for the existing networks from the store. But the overlay network's vxlan id was getting allocated afresh instead of restoring the current vxlan id allocated for the network.

related to docker #28493

Signed-off-by: Santhosh Manohar [email protected]

@codecov-io
Copy link

codecov-io commented Nov 28, 2016

Current coverage is 55.18% (diff: 66.66%)

Merging #1773 into master will increase coverage by 0.09%

@@             master      #1773   diff @@
==========================================
  Files           102        102          
  Lines         16871      16875     +4   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           9294       9313    +19   
+ Misses         6419       6404    -15   
  Partials       1158       1158          

Sunburst

Powered by Codecov. Last update 2422b48...c803824

@mavenugo
Copy link
Contributor

LGTM

@dongluochen
Copy link
Contributor

moby/moby#28493 reports it on mixed Swarm versions. If this is the root cause, the problem should also occur with same version of docker Swarm. Can we reproduce it?

@aaronlehmann
Copy link
Collaborator

This code looks like it could modify n.Spec.DriverConfig.Options, which doesn't seem right. The manager should never make changes to anything inside Spec.

@sanimej
Copy link
Contributor Author

sanimej commented Nov 28, 2016

@dongluochen In 1.13 we changed the default range of vxlan ids to start from 4096 (it was 256 in 1.12) This exposes the problem right away. This problem exists in 1.12.X though. Just that its hard to hit because it depends on the order in which networks are read from the store.

@sanimej
Copy link
Contributor Author

sanimej commented Nov 28, 2016

@aaronlehmann Yes, with the current change options in n.DriverState.Options will supersede n.Spec.DriverConfig.Options. Alternative would be to not use the option values from n.DriverState if the same option already exists in n.Spec.DriverConfig. Is that better ?

@sanimej
Copy link
Contributor Author

sanimej commented Nov 28, 2016

@aaronlehmann Updated the change to use a local copy of the options to avoid modifying n.Spec.DriverConfig.Options. PTAL

}
if n.DriverState != nil {
for k, v := range n.DriverState.Options {
options[k] = v
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any possibility of overlapping keys in n.Spec.DriverConfig.Options and n.DriverState.Options? If yes what does that mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, its possible. If user explicitly specifies a driver option during network create it will be in n.Spec.DriverConfig.Options. After the resource allocation by the driver all the options including the one from the spec will be available in n.DriverState.Options which is the operational state.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the resource allocation by the driver all the options including the one from the spec will be available in n.DriverState.Options which is the operational state.

In that case, wouldn't n.DriverState.Options be enough? It should be a superset of n.Spec.DriverConfig.Options.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, wouldn't n.DriverState.Options be enough? It should be a superset of n.Spec.DriverConfig.Options.

Its not enough. A leader manager could lose the leader status right after a network create before the allocation and the subsequent store update.

@dongluochen
Copy link
Contributor

LGTM

@mavenugo
Copy link
Contributor

@sanimej @dongluochen @aaronlehmann i could reproduce this issue in 1.12.3 as well. it seems fairly basic issue and we need to cherry-pick this in the 1.12.x branch as well.

@aaronlehmann
Copy link
Collaborator

@mavenugo: Makes sense. When it gets merged, I'll make sure it gets cherry-picked to both branches.

@aaronlehmann
Copy link
Collaborator

LGTM

@aaronlehmann
Copy link
Collaborator

Cherry-picked to bump_v1.12.4 and bump_v1.13.0.

@mavenugo
Copy link
Contributor

Thanks @aaronlehmann

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants