taeef_najib
11/05/2023, 5:58 PMtap-csv and put the data as parquet file into an S3 bucket which is connected to our Iceberg datalake. Should we take target-s3 as a reference and customize it to follow this: https://iceberg.apache.org/spec/#parquet or should we use pyspark to write files to our data lake? Not sure how to approach this. I'm really looking for a direction from the community.