Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: never construct id token claim templates in parallel #552

Conversation

AlbinoDrought
Copy link
Contributor

@AlbinoDrought AlbinoDrought commented Oct 7, 2020

Fixes #551

Related issue

#551

Proposed changes

Ensure we never call templates.New().Parse() across more than one thread. I have implemented this for mutator_id_token.go which solves my issue in #551.

Checklist

  • I have read the contributing guidelines.
  • I have read the security policy.
  • I confirm that this pull request does not address a security
    vulnerability. If this pull request addresses a security. vulnerability, I
    confirm that I got green light (please contact
    [email protected]) from the maintainers to push
    the changes.
  • I have added tests that prove my fix is effective or that my feature
    works. (I'm not sure how to test this kind of issue)
  • I have added or changed the documentation. (not sure if required for a non-feature change)

Further comments

It appears like the same issue may exist in these files:

  • pipeline/mutate/mutator_header.go
  • pipeline/mutate/mutator_cookie.go
  • pipeline/authz/remote.go
  • pipeline/authz/remote_json.go
  • pipeline/authz/keto_engine_acp_ory.go

Affected blocks may look something like this:

	t := a.t.Lookup(templateID)
	if t == nil {
		var err error
		// if two goroutines reach this at the same time, we panic
		t, err = a.t.New(templateID).Parse(templateString)
		if err != nil {
			return "", err
		}
	}

If this fix is acceptable, I can carry it over to those files as well.

@CLAassistant
Copy link

CLAassistant commented Oct 7, 2020

CLA assistant check
All committers have signed the CLA.

Copy link
Member

@aeneasr aeneasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you for your contribution! This looks pretty good but I have some ideas how to improve it further :)

@@ -152,7 +154,9 @@ func (a *MutatorIDToken) Mutate(r *http.Request, session *authn.AuthenticationSe
t := a.templates.Lookup(c.ClaimsTemplateID())
if t == nil {
var err error
a.templatesLock.Lock()
t, err = a.templates.New(c.ClaimsTemplateID()).Parse(c.Claims)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation says:

Because associated templates share underlying data, template construction cannot be done safely in parallel. Once the templates are constructed, they can be executed in parallel.

So I think it would be better to have a a.getTemplate(c.ClaimsTemplateID()) which looks up the template and if it doesn't exist creates it and adds it to the cache. The creation part would be locked while the read part would not have to be locked. This will increase throughput as we don't need to lock here for every request :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, I'm not sure I understand. Do you have an example of what you're thinking?

Here are my assumptions right now with the current code:

	var templateClaims []byte
	if len(c.Claims) > 0 {
		// find the already-constructed template:
		t := a.templates.Lookup(c.ClaimsTemplateID())
		if t == nil {
			// if we couldn't find it, lock and construct a new one (only one time)
			var err error
			a.templatesLock.Lock()
			t, err = a.templates.New(c.ClaimsTemplateID()).Parse(c.Claims)
			a.templatesLock.Unlock()
			if err != nil {
				return errors.Wrapf(err, `error parsing claims template in rule "%s"`, rl.GetID())
			}
		}

		// after a template already been constructed once, requests should reach this point without locking:
		var b bytes.Buffer
		if err := t.Execute(&b, session); err != nil {
			return errors.Wrapf(err, `error executing claims template in rule "%s"`, rl.GetID())
		}

		templateClaims = b.Bytes()
		if err := json.NewDecoder(&b).Decode(&claims); err != nil {
			return errors.WithStack(err)
		}
	}

Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, my bad, I just overlooked the preceding if statement and template lookup - you are correct, this is indeed the way it can be solved! :)

@aeneasr aeneasr merged commit 4f504d9 into ory:master Oct 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"fatal error: concurrent map writes" panic, unable to reproduce
3 participants