Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSV headers not getting added while writing to external location using CTAS #22541

Closed
singhalkavmware opened this issue Jun 28, 2024 · 2 comments

Comments

@singhalkavmware
Copy link

Hi team,

I am exporting data into .gz files using the below query on Trino from AWS Glue tables.

CREATE TABLE cht_mdr_temp.temp_table WITH (external_location='s3://cht-mdr/mdr_temp/temp_table_kanika',format = 'CSV') AS SELECT pre_installed_software AS pre_installed_software, region_code AS region_code, CAST(convertible_1yr_all_upfront_upfront_cost AS varchar) AS convertible_1yr_all_upfront_upfront_cost, FROM cht_mdr_temp.a0aapqkfo_1719536524875

The .gz files that are getting created does not contain the headers of the columns. We have tried using the property 'skip_header_line_count' but it does not help.

Is it possible to get CSV headers as per the aliases provided in the query, in the file exported via CTAS?

@findinpath
Copy link
Contributor

testing/bin/ptl env up --environment multinode --config config-default
trino> create table hive.default.ht1 (col1 varchar, col2 varchar) with (format = 'csv', skip_header_line_count =1);
CREATE TABLE
trino> insert into hive.default.ht1 values ('a', 'A'), ('b', 'BB'), ('c', 'CCC');
INSERT: 3 rows
[root@hadoop-master /]# hdfs dfs -copyToLocal /user/hive/warehouse/ht1/20240705_203744_00013_c3tem_230893da-0298-4e21-8eb0-3604aed0c3c9.gz .
[root@hadoop-master /]# zcat 20240705_203744_00013_c3tem_230893da-0298-4e21-8eb0-3604aed0c3c9.gz
"col1","col2"
"a","A"
"b","BB"
"c","CCC"

Seems to be working for me

@singhalkavmware
Copy link
Author

Thanks, we are able to get the headers in the csv file now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants