Hello, I'm trying to use the target-snowflake extractor and am just a little confused about the file_format variable that's required. I'm pulling in multiple tables and want to give them each their own file_name within the schema, but am not sure about how to distinguish the different tables. Additionally, when just trying to pull in the extractor with a given file format, I get a

Broken Pipe Error

stating that my file format is not found. Would love some additional insight on the

file_format

parameter and how to configure it properly for my use case

Hey @aditya_penmesta - have you created a file format object in your snowflake instance? Also check out the hub dependencies section. Behind the scenes the loader is writing to files to a Snowflake table stage then copying it into the table, the file format is defining what format those table stage files should use. These arent 1-1 with a table, its defined once for the loader and used across all tables/streams. The docs for the loader just recommend CSV (thats what I use too).

Hi Pat, thanks for the quick response. Didn't know about the setup in Snowflake so I'll start working through that now

Cool post back if you run into any issues!

I'm extracting data from a Postgres DB and dumping into Snowflake. I initially thought file_format was a 1:1 matching between the postgres db table and snowflake table name, but seems not. I'll let you know if I have problems

It could be a variety of things - try debug mode and see if the logs clarify anything https://docs.meltano.com/guide/troubleshooting#log-level-debug

Awesome thank you that helped clarify what was going wrong

Good to hear! Were you able to get it working?

Yes