Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip materialized view checking if source table does not exist #47975

Merged

Conversation

MikhailBurdukov
Copy link
Contributor

@MikhailBurdukov MikhailBurdukov commented Mar 24, 2023

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fixed UNKNOWN_TABLE exception when attaching to a materialized view that has dependent tables that are not available.
This might be useful when trying to restore state from a backup.

Steps to reproduce:

CREATE DATABASE test_db;

CREATE TABLE test_db.table (n Int32, s String) ENGINE MergeTree PARTITION BY n ORDER BY n;

CREATE TABLE test_db.mview_backend (n Int32, n2 Int64) ENGINE MergeTree PARTITION BY n ORDER BY n;

CREATE MATERIALIZED VIEW test_db.mview TO test_db.mview_backend AS SELECT n, n * n AS "n2" FROM test_db.table;

DROP TABLE test_db.table;

DETACH TABLE test_db.mview;

ATTACH TABLE test_db.mview;
Received exception from server (version 22.11.5):
Code: 60. DB::Exception: Received from localhost:9001. DB::Exception: Table test_db.table doesn't exist. (UNKNOWN_TABLE)

Expected behavior: attaching materialized view succeeds regardless of whether it's valid and references existing table, or not. It's important for backup-restore scenarios.

With ordinary view works as expected.

CREATE DATABASE test_db;

CREATE TABLE test_db.table (n Int32, s String) ENGINE MergeTree PARTITION BY n ORDER BY n;

CREATE VIEW test_db.view AS SELECT n, n * n AS "n2" FROM test_db.table;

DROP TABLE test_db.table;

DETACH TABLE test_db.view;

ATTACH TABLE test_db.view;
Ok.

@CLAassistant
Copy link

CLAassistant commented Mar 24, 2023

CLA assistant check
All committers have signed the CLA.

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Mar 24, 2023
@robot-ch-test-poll4 robot-ch-test-poll4 added pr-improvement Pull request with some product improvements and removed pr-bugfix Pull request with bugfix, not backported by default labels Mar 24, 2023
@MikhailBurdukov MikhailBurdukov changed the title Skip unavailable tables for attaching to a materialzed view. Skip materialized view checking if source table does not exist Mar 24, 2023
@kssenii kssenii added the can be tested Allows running workflows for external contributors label Mar 24, 2023
@@ -467,6 +467,7 @@ class IColumn;
M(Int64, max_partitions_to_read, -1, "Limit the max number of partitions that can be accessed in one query. <= 0 means unlimited.", 0) \
M(Bool, check_query_single_value_result, true, "Return check query result as single 1/0 value", 0) \
M(Bool, allow_drop_detached, false, "Allow ALTER TABLE ... DROP DETACHED PART[ITION] ... queries", 0) \
M(Bool, skip_materialized_view_checking_if_source_table_not_exist, false, "Allow attaching to a materialized view even if dependent tables are inaccessible.", 0) \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that an additional setting is required. For me this PR a bug fix that makes behaviour of ATTACH query consistent for the both ordinary and materialized views.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that setting is probably redundant, ok to remove. (Though if not removing it - in my opinion its name should end with _on_attach )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should always skip the check on ATTACH query, the check makes sense for CREATE only

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should always skip the check on ATTACH query, the check makes sense for CREATE only

If I understood you correctly, we should check type compatibility only in cases of CREATE. I have reverted changes and added a simple check that the query is not attached.

@kssenii kssenii self-assigned this Mar 27, 2023
src/Interpreters/InterpreterCreateQuery.h Outdated Show resolved Hide resolved
@@ -467,6 +467,7 @@ class IColumn;
M(Int64, max_partitions_to_read, -1, "Limit the max number of partitions that can be accessed in one query. <= 0 means unlimited.", 0) \
M(Bool, check_query_single_value_result, true, "Return check query result as single 1/0 value", 0) \
M(Bool, allow_drop_detached, false, "Allow ALTER TABLE ... DROP DETACHED PART[ITION] ... queries", 0) \
M(Bool, skip_materialized_view_checking_if_source_table_not_exist, false, "Allow attaching to a materialized view even if dependent tables are inaccessible.", 0) \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that setting is probably redundant, ok to remove. (Though if not removing it - in my opinion its name should end with _on_attach )

Comment on lines 1082 to 1086
if (StoragePtr to_table = DatabaseCatalog::instance().tryGetTable(
{create.to_table_id.database_name, create.to_table_id.table_name, create.to_table_id.uuid},
getContext()
))
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A minor suggestion: consider using "early return": https://dev.to/jpswade/return-early-12o5

@kssenii
Copy link
Member

kssenii commented Mar 28, 2023

AST fuzzer (ubsan) — Received signal 11

#48094

robot-clickhouse added a commit that referenced this pull request Mar 29, 2023
robot-clickhouse added a commit that referenced this pull request Mar 29, 2023
robot-clickhouse added a commit that referenced this pull request Mar 29, 2023
kssenii added a commit that referenced this pull request Mar 30, 2023
Backport #47975 to 23.2: Skip materialized view checking if source table does not exist
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Mar 31, 2023
kssenii added a commit that referenced this pull request Apr 5, 2023
Backport #47975 to 22.8: Skip materialized view checking if source table does not exist
kssenii added a commit that referenced this pull request Apr 11, 2023
Backport #47975 to 23.1: Skip materialized view checking if source table does not exist
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
can be tested Allows running workflows for external contributors pr-backport Changes, backported to release branch. Do not use manually - automated use only! pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-improvement Pull request with some product improvements pr-must-backport Pull request should be backported intentionally. Use this label with great care!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants