initial Athena Clickhouse connector commit related to issue 1754 #1770
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ejeffrli
reviewed
Mar 5, 2024
athena-clickhouse/src/main/java/com/amazonaws/athena/connectors/clickhouse/ClickHouseUtil.java
Outdated
Show resolved
Hide resolved
chngpe
reviewed
Mar 20, 2024
chngpe
reviewed
Mar 20, 2024
chngpe
requested changes
Mar 20, 2024
There was a problem hiding this comment.
Majority of code are basically copy over from MySQL connectors. Can we just extends from MySQL connector instead of JDBC?
For example, ClickHouseMetadataHandler.java extends from MySqlMetadataHandler so we can avoid duplicate code and leave the part that is necessary.
For example, https://github.com/awslabs/aws-athena-query-federation/blob/master/athena-redshift/src/main/java/com/amazonaws/athena/connectors/redshift/RedshiftMetadataHandler.java which extends from PostGreSqlMetadataHandler just override jdbc driver class and default port in constructor. (along with some necessary method for list paginated tables query)
Incorporate latest changes from parent aws-athena-query-federation repo into athena-clickhouse fork
aimethed
reviewed
Jun 5, 2024
...ouse/src/main/java/com/amazonaws/athena/connectors/clickhouse/ClickHouseMetadataHandler.java
Outdated
Show resolved
Hide resolved
chngpe
previously approved these changes
Jun 5, 2024
chngpe
reviewed
Jun 5, 2024
...khouse/src/main/java/com/amazonaws/athena/connectors/clickhouse/ClickHouseRecordHandler.java
Show resolved
Hide resolved
chngpe
reviewed
Jun 18, 2024
| @@ -71,7 +77,9 @@ Resources: | |||
| default: !Ref DefaultConnectionString | |||
| FunctionName: !Ref LambdaFunctionName | |||
| Handler: "com.amazonaws.athena.connectors.clickhouse.ClickHouseMuxCompositeHandler" | |||
| CodeUri: "./target/athena-clickhouse-2022.47.1.jar" | |||
| CodeUri: | |||
There was a problem hiding this comment.
We don't need to do that as SAR can handle this.
There was a problem hiding this comment.
Customer can actually invoke sam deploy which will take care of that part
aimethed
approved these changes
Jun 25, 2024
github-actions bot
pushed a commit
that referenced
this pull request
Jun 26, 2024
- Fix google bigquery dependency version - ignore google big query dependabot - Hbase uppercase table (#2019) - improve encryption key length check (#2044) - initial Athena Clickhouse connector commit related to issue 1754 (#1770) - Add Support for querying Protobuf Data in Kafka Connector (#2020) - Improved Neptune connector docs, CFN deploy for example (#1998) - Updated before installing Dependencies (#2042) - build(deps): bump software.amazon.awssdk:bom from 2.26.3 to 2.26.7 (#2041) - build(deps): bump software.amazon.awssdk:bom from 2.26.3 to 2.26.7 - build(deps): bump com.github.spotbugs:spotbugs-annotations from 4.8.5 to 4.8.6 (#2040) - build(deps): bump com.github.spotbugs:spotbugs-annotations - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin from 3.7.0 to 3.7.1 (#2038) - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin - build(deps): bump aws-sdk.version from 1.12.744 to 1.12.748 (#2037) - build(deps): bump aws-sdk.version from 1.12.744 to 1.12.748 - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.20.17 to 2.21.10 (#2036) - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.20.17 to 2.21.10 - build(deps): bump org.apache.maven.plugins:maven-jar-plugin from 3.4.1 to 3.4.2 (#2035) - build(deps): bump org.apache.maven.plugins:maven-jar-plugin - build(deps): bump surefire.failsafe.version from 3.2.5 to 3.3.0 (#2032) - build(deps): bump surefire.failsafe.version from 3.2.5 to 3.3.0 - build(deps): bump software.amazon.msk:aws-msk-iam-auth from 2.1.0 to 2.1.1 (#2029) - build(deps): bump software.amazon.msk:aws-msk-iam-auth - build(deps): bump software.amazon.jsii:jsii-runtime from 1.99.0 to 1.100.0 (#2030) - build(deps): bump software.amazon.jsii:jsii-runtime - build(deps): bump io.confluent:kafka-avro-serializer from 7.6.0 to 7.6.1 (#2026) - build(deps): bump io.confluent:kafka-avro-serializer from 7.6.0 to 7.6.1 - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1 from 3.5.2 to 3.6.0 (#2033) - build(deps): bump com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1 - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin from 3.6.1 to 3.7.0 (#2028) - build(deps): bump org.apache.maven.plugins:maven-dependency-plugin - build(deps): bump software.amazon.awssdk:bom from 2.25.69 to 2.26.3 (#2031) - build(deps): bump software.amazon.awssdk:bom from 2.25.69 to 2.26.3 - build(deps): bump org.apache.commons:commons-configuration2 from 2.10.1 to 2.11.0 (#2027) - build(deps): bump org.apache.commons:commons-configuration2 - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.14.0 to 8.14.1 (#2025) - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client - build(deps): bump aws-sdk.version from 1.12.739 to 1.12.744 (#2024) - build(deps): bump aws-sdk.version from 1.12.739 to 1.12.744 - Added changes to support without kerberos authentication (#2023) - Enhance exception handling and logging error message for Neptune QPT (#2018) - [FEATURE] Add Support for querying Avro Data in Kafka Connector (#1958) - build(deps): bump com.google.cloud:google-cloud-resourcemanager from 1.46.0 to 1.47.0 (#2017) - build(deps): bump com.google.cloud:google-cloud-resourcemanager - build(deps): bump software.amazon.awssdk:bom from 2.25.64 to 2.25.69 (#2012) - build(deps): bump software.amazon.awssdk:bom from 2.25.64 to 2.25.69 - build(deps): bump com.amazon.redshift:redshift-jdbc42 from 2.1.0.28 to 2.1.0.29 (#2013) - build(deps): bump com.amazon.redshift:redshift-jdbc42 - build(deps): bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 (#2010) - build(deps): bump com.microsoft.azure:msal4j from 1.15.0 to 1.15.1 - build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin from 3.3.1 to 3.4.0 (#2008) - build(deps): bump org.apache.maven.plugins:maven-checkstyle-plugin - build(deps): bump com.google.cloud:google-cloud-storage from 2.39.0 to 2.40.0 (#2014) - build(deps): bump com.google.cloud:google-cloud-storage - build(deps): bump software.amazon.jsii:jsii-runtime from 1.98.0 to 1.99.0 (#2016) - build(deps): bump software.amazon.jsii:jsii-runtime - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client from 8.13.4 to 8.14.0 (#2009) - build(deps): bump org.elasticsearch.client:elasticsearch-rest-client - build(deps): bump io.grpc:grpc-api from 1.63.0 to 1.64.0 (#2011) - build(deps): bump io.grpc:grpc-api from 1.63.0 to 1.64.0 - build(deps): bump aws-sdk.version from 1.12.734 to 1.12.739 (#2007) - build(deps): bump aws-sdk.version from 1.12.734 to 1.12.739
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
closes #1754
Description of changes:
Athena Clickhouse connector reusing Athena MySQL code and Clickhouse driver
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.