Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hbase uppercase table #2019

Merged
merged 11 commits into from
Jun 26, 2024
Merged

Hbase uppercase table #2019

merged 11 commits into from
Jun 26, 2024

Conversation

aimethed
Copy link
Contributor

Description of changes:
Previously, hbase would fail when trying to access tables with uppercase characters because athena does not support uppercase tables. This change allows querying of hbase uppercase tables by handling uppercase names the same way the dynamodb connector handles this: by getting all tables in the hbase namespace and mapping the normal case name to its lowercase name. This ensures that if multiple hbase tables would have the same lowercase name (ex: Table and tAble would both be table), an error is thrown.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@aimethed aimethed self-assigned this Jun 13, 2024
@AbdulR3hman AbdulR3hman self-requested a review June 14, 2024 17:45
@AbdulR3hman
Copy link
Contributor

I believe we should preserve the original behavior; and put this change behind a feature flag as it will change what existing customers are expecting the default connector behavior.

@aimethed
Copy link
Contributor Author

Summary of changes:

  • Now requires the following environment variable to be added to enable case insensitive search: enable_case_insensitive_match = true
  • Updated doGetTable to return the name from the case insensitive search instead of the one from the request
    • If it's disabled, just returns the name
  • Removed the check for if the table exists. Since this feature is now opt-in, the check causes unwanted behavior.
    • If there are two tables, table and Table, querying for Table before would succeed but match with table. Now errors due to multiple possible matches.
  • Added a check for if the given name isn't lowercase. Addresses an error where doListTables was called with correctly-cased table name.

@aimethed aimethed merged commit bb177bd into master Jun 26, 2024
8 checks passed
@aimethed aimethed deleted the hbase-uppercase-table branch June 26, 2024 15:18
github-actions bot pushed a commit that referenced this pull request Jun 26, 2024
  - Fix google bigquery dependency version
  - ignore google big query dependabot
  - Hbase uppercase table (#2019)
  - improve encryption key length check (#2044)
  - initial Athena Clickhouse connector commit related to issue 1754 (#1770)
  - Add Support for querying Protobuf Data in Kafka Connector (#2020)
  - Improved Neptune connector docs, CFN deploy for example (#1998)
  - Updated before installing Dependencies (#2042)
  - build(deps): bump software.amazon.awssdk:bom from 2.26.3 to 2.26.7 (#2041)
  - build(deps): bump software.amazon.awssdk:bom from 2.26.3 to 2.26.7
  - build(deps): bump com.github.spotbugs:spotbugs-annotations from 4.8.5 to 4.8.6 (#2040)
  - build(deps): bump com.github.spotbugs:spotbugs-annotations
  - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin from 3.7.0 to 3.7.1 (#2038)
  - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin
  - build(deps): bump aws-sdk.version from 1.12.744 to 1.12.748 (#2037)
  - build(deps): bump aws-sdk.version from 1.12.744 to 1.12.748
  - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.20.17 to 2.21.10 (#2036)
  - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.20.17 to 2.21.10
  - build(deps): bump org.apache.maven.plugins:maven-jar-plugin from 3.4.1 to 3.4.2 (#2035)
  - build(deps): bump org.apache.maven.plugins:maven-jar-plugin
  - build(deps): bump surefire.failsafe.version from 3.2.5 to 3.3.0 (#2032)
  - build(deps): bump surefire.failsafe.version from 3.2.5 to 3.3.0
  - build(deps): bump software.amazon.msk:aws-msk-iam-auth from 2.1.0 to 2.1.1 (#2029)
  - build(deps): bump software.amazon.msk:aws-msk-iam-auth
  - build(deps): bump software.amazon.jsii:jsii-runtime from 1.99.0 to 1.100.0 (#2030)
  - build(deps): bump software.amazon.jsii:jsii-runtime
  - build(deps): bump io.confluent:kafka-avro-serializer from 7.6.0 to 7.6.1 (#2026)
  - build(deps): bump io.confluent:kafka-avro-serializer from 7.6.0 to 7.6.1
  - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1 from 3.5.2 to 3.6.0 (#2033)
  - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1
  - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin from 3.6.1 to 3.7.0 (#2028)
  - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin
  - build(deps): bump software.amazon.awssdk:bom from 2.25.69 to 2.26.3 (#2031)
  - build(deps): bump software.amazon.awssdk:bom from 2.25.69 to 2.26.3
  - build(deps): bump org.apache.commons:commons-configuration2 from 2.10.1 to 2.11.0 (#2027)
  - build(deps): bump org.apache.commons:commons-configuration2
  - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.14.0 to 8.14.1 (#2025)
  - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client
  - build(deps): bump aws-sdk.version from 1.12.739 to 1.12.744 (#2024)
  - build(deps): bump aws-sdk.version from 1.12.739 to 1.12.744
  - Added changes to support without kerberos authentication (#2023)
  - Enhance exception handling and logging error message for Neptune QPT (#2018)
  - [FEATURE] Add Support for querying Avro Data in Kafka Connector (#1958)
  - build(deps): bump com.google.cloud:google-cloud-resourcemanager from 1.46.0 to 1.47.0 (#2017)
  - build(deps): bump com.google.cloud:google-cloud-resourcemanager
  - build(deps): bump software.amazon.awssdk:bom from 2.25.64 to 2.25.69 (#2012)
  - build(deps): bump software.amazon.awssdk:bom from 2.25.64 to 2.25.69
  - build(deps): bump com.amazon.redshift:redshift-jdbc42 from 2.1.0.28 to 2.1.0.29 (#2013)
  - build(deps): bump com.amazon.redshift:redshift-jdbc42
  - build(deps): bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 (#2010)
  - build(deps): bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1
  - build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin from 3.3.1 to 3.4.0 (#2008)
  - build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin
  - build(deps): bump com.google.cloud:google-cloud-storage from 2.39.0 to 2.40.0 (#2014)
  - build(deps): bump com.google.cloud:google-cloud-storage
  - build(deps): bump software.amazon.jsii:jsii-runtime from 1.98.0 to 1.99.0 (#2016)
  - build(deps): bump software.amazon.jsii:jsii-runtime
  - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.13.4 to 8.14.0 (#2009)
  - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client
  - build(deps): bump io.grpc:grpc-api from 1.63.0 to 1.64.0 (#2011)
  - build(deps): bump io.grpc:grpc-api from 1.63.0 to 1.64.0
  - build(deps): bump aws-sdk.version from 1.12.734 to 1.12.739 (#2007)
  - build(deps): bump aws-sdk.version from 1.12.734 to 1.12.739
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants