Hello team, I am using `tap-google-analytics` and ...
# troubleshooting
h
Hello team, I am using
tap-google-analytics
and
target-athena.
I have created required data-stream with desired fields and metrics. However, when I am running the pipeline my data columns are coming up in jumbled up manner. Does anyone know why and how I can solve this issue?
p
Hey @hitesh can you share your meltano.yml config (with any sensitive credentials removed)? I used to use Athena before switching to Snowflake so I'm trying to remember how I got around this. I also saw something similar
Are you using CSV
output_format
? If I remember my issue correctly there was a bug where the ordering of the CSV columns were changing so the data got all jumbled like you mentioned, using JSONL looks like it fixed it. Also I see that I had
flatten_records: false
for a few data sources so I wonder if flattening was causing issue 🤔
h
Yeah it is csv
and
flatten records
are true
p
thats worth a shot then, try using jsonl instead and see if it helps
h
Ohh let me try it
p
I was also pinning to this branch
bump-sdk-to-0_4x
that looks like its still an open PR. It only bumps the SDK version so I'm not sure why I did that but maybe there was a bug fix or something, so we can try that next if needed
h
Thanks it solves my issue.