I am trying to get a table from tap-snowflake to load it into a target-snowflake. The tap-snowflake has a tables field to specify what to get. However, it is unclear to me how I can rename the table when loading it to the target-snowflake db. I want to name the table to make sure the user knows in snowflake that the table was loaded by meltano and wasn't something they loaded themselves. Does anyone have a recommendation on how to do so? what I'm trying to run:

meltano run tap-snowflake target-snowflake

If they’re in the same dwh why not use dbt?

we are doing some transformations on the data before reuploading and i would like to make it clear to the user that this is post-transformation by the table name. that way they have the original table as well as the transformed table

I’m under the impression that meltano is great for getting data from source A to source B. When they’re in the same dwh I would imagine you can easily do this with dbt. Say your raw data comes from Segment and you want to put the table from the schema

segment

into the schema `analytics`… Then why can’t you use dbt to do this?

{{ config(materialized='table', schema='analytics') }}

select
  timestamp,
  event,
  ...
from
  {{ source('segment', <my_table>) }}

could clarify what you mean by DBT and DWH?

• dwh = data warehouse (e.g. snowflake) • dbt: Data-Build-Tool. Very powerful data transformation tool that lets you use jinja and sql to build complex data transformations. https://getdbt.com

I'm trying to see if there is a config for target-snowflake that will allow us to specify a table name for us. Data transformations are done in-house. We have a unique use-case. Are there no target-snowflake meltano configs that allow us to specify how we are saving our data? We would rather not introduce tool unless absolutely necessary.

I see. If you are already using meltano and not dbt then I guess it makes sense. If you use SQL at all i would highly recommend looking at dbt. I didn’t know how useful it was until I started using it (my last company built everything in python / R / mysql). I know we’re talking on a meltano thread but curious how are you doing transformations if it’s not with dbt? Pure sql / python -> snowflake? If you decide to take a look into dbt, it’s super simple to do what you’re asking. You would just create a filename with the table name you want, and then inside the file put the below code block

# once you cd inside /my_dbt_folder/models/sub_directory/transformed_segment_data.sql

{{ config(materialized='incremental', schema='analytics') }}

select * from {{ source('segment', <my_table>) }} 
{% if is_incremental() %}
  where timestamp > (select max(timestamp) from {{this}})
{% endif %}

^^ This will create a table called

transformed_segment_data

inside the schema

analytics

on snowflake and build it incrementally without fully loading the entire table each time. I don’t find it necessary to build a meltano tap to create a table within the same data warehouse. If it’s a big table, that’s even more reason to use dbt since extracting and loading that amount of data from snowflake isn’t cheap. Not using meltano or dbt you can do this with a few lines of python as well

class SnowflakeConnect():
    def connect():
        <connect to db>
        return self
    def run_sql():
        <sqlalchemy code to execute sql code>

db = SnowflakeConnect()
db.run_sql("create table if not exists <my table> as select * from <table>")

I agree with Matt that this sounds like something could be easily solved with DBT (which also can be integrated directly into Meltano!) but otherwise I believe you can use stream-maps on the tap-snowflake side to rename the tables/streams: https://sdk.meltano.com/en/latest/stream_maps.html