Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[data.search] Allow search response to follow new hits format #88115

Merged
merged 16 commits into from
Feb 2, 2021

Conversation

lukasolson
Copy link
Member

@lukasolson lukasolson commented Jan 13, 2021

Summary

Closes #55974.

When making a search request to Elasticsearch, the hits.total format comes back as follows:

{"hits": {"total": {"value": 8504, "relation": "eq"}} // Could be "gt" instead of "eq"

However, we are currently shimming the old response format, which looks like this:

{"hits": {"total": 8504}}

The shimming prior to this PR was happening in the route and bfetch handlers, which didn't make sense so I moved it directly inside the search strategies. This PR also adds a new option, legacyHitsTotal, to the options you can provide a search request (ISearchOptions). When set to true (the default), we will continue to shim the old response format. When set to false, we will return the response format as-is from Elasticsearch. This gives applications the ability to migrate to the new format and opt out of tracking the total number of hits (track_total_hits) for performance reasons.

Checklist

Delete any items that are not applicable to this PR.

For maintainers

Plugin API changes

When using the data plugin search service search method, you can now provide an additional argument, legacyHitsTotal, in the options. When set to true (the default), the hits.total will be returned as a number. When set to false, the hits.total format will be returned as-is from the Elasticsearch response.

Open questions

How do we want to expose this option to esaggs?

@lukasolson lukasolson added review Feature:Search Querying infrastructure in Kibana release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. v8.0.0 Team:AppServices v7.12.0 labels Jan 13, 2021
@lukasolson lukasolson requested a review from a team as a code owner January 13, 2021 00:01
@lukasolson lukasolson self-assigned this Jan 13, 2021
@lukasolson lukasolson requested review from a team as code owners January 13, 2021 00:01
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServices)

@lukasolson lukasolson requested a review from a team January 19, 2021 16:47
Copy link
Contributor

@rylnd rylnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security Solution changes LGTM right now, and I'm happy to make another pass if the current discussions lead to further changes.

@lukasolson
Copy link
Member Author

@elasticmachine merge upstream

1 similar comment
@lizozom
Copy link
Contributor

lizozom commented Jan 31, 2021

@elasticmachine merge upstream

Copy link
Contributor

@lizozom lizozom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Should we send an email following merging this PR letting teams know that during 7.13 we'll be switching the default to true and that they should migrate?

Copy link
Member

@tsullivan tsullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lukasolson
Copy link
Member Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@lukasolson lukasolson merged commit 29e3886 into elastic:master Feb 2, 2021
lukasolson added a commit to lukasolson/kibana that referenced this pull request Feb 2, 2021
…c#88115)

* [data.search] Allow search response to follow new hits format

* Update docs

* Fix types

* Fix types

* Doc updates

* Remove declare module

* Remove declare module

Co-authored-by: Kibana Machine <[email protected]>
lukasolson added a commit that referenced this pull request Feb 2, 2021
#89951)

* [data.search] Allow search response to follow new hits format

* Update docs

* Fix types

* Fix types

* Doc updates

* Remove declare module

* Remove declare module

Co-authored-by: Kibana Machine <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>
gmmorris added a commit to lizozom/kibana that referenced this pull request Feb 2, 2021
…om/kibana into pr/89570

* 'sessions/save-all-sessions' of https://github.com/lizozom/kibana: (44 commits)
  [ML] Functional tests - skip DFA clone tests
  [Uptime] Fix synthetics detail step count (elastic#89940)
  Fixes the permissions to require cluster.manage in order to create an index and in order to update an index (elastic#89947)
  [Security Solution] [Detections] adds log info level for logging in cloud (elastic#89941)
  [Time to Visualize] Dashboard By Value Testing Lens (elastic#89581)
  [Uptime] Expand synthetic journey step thumbnail on hover (elastic#89179)
  TS project refs: Migrates snapshot_restore to a TS Project (elastic#89653)
  docs: APM 7.11 updates (elastic#89789)
  move skip to higher level (elastic#86952)
  Revert "Migrations v2: don't auto-create indices + FTR/esArchiver support (elastic#85778)"
  Revert "Revert "Enable v2 so migrations, disable in FTR tests (elastic#89297)""
  Revert "Enable v2 so migrations, disable in FTR tests (elastic#89297)"
  [data.search] Allow search response to follow new hits format (elastic#88115)
  [Maps] Change 'create multi-layer map' title to be use-case focused (elastic#89520)
  skip flaky suite (elastic#86952)
  [Security Solution] Remove focustrap (elastic#89905)
  [Workplace Search] Add remaining i18n support for the Content Sources tree (elastic#89910)
  [esArchiver] log when migrations complete and we're done loading data (elastic#89938)
  Add --ssl flag to make resolver generator use ssl with kbn and elasticsearch clients (elastic#89873)
  TS project refs: Migrates grokdebugger (elastic#89652)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Search Querying infrastructure in Kibana release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. review v7.12.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Search] Consistent support for track_total_hits
6 participants