Hello I am trying to add a new column load_date to target stream using meltano-map- transformer, `ho...

choudary kukkapalli

04/19/2024, 3:11 AM

Hello I am trying to add a new column load_date to target stream using meltano-map- transformer,

how can i make this column created with timestamp datatype

in target table, with the below yml its getting created as string in this format YYYY-MM-DDTHH24;MI:SS.ssss

Copy code

mappers:
  - name: meltano-map-transformer
    variant: meltano
    pip_url: git+<https://github.com/MeltanoLabs/meltano-map-transform.git>
    mappings:
    - name: table_mapping
      config:
        stream_maps:
          dbo-cust:
            __alias__: cust_v1
            load_date: datetime.datetime.now()

dylan_just

04/19/2024, 4:04 AM

I don't know how to do this in a mapper, but you could parse the date in a dbt model.

dylan_just

04/19/2024, 4:05 AM

https://sdk.meltano.com/en/latest/stream_maps.html

dylan_just

04/19/2024, 4:05 AM

Copy code

Schema detection capabilities are limited
Schema detection currently relies on somewhat naive static text parsing. The workaround for the user is fairly trivial - which is to send hints by wrapping the entire expression in str(), float(), int(), etc. While this is perhaps not optimal, it meets our core requirement for static type evaluation with minimal config complexity.

dylan_just

04/19/2024, 4:05 AM

Maybe there's something in there?

choudary kukkapalli

04/22/2024, 7:28 PM

thank you @dylan_just for the response, i didnt find anything in that docs page, i think we will leave it string and cast it later

👍 1

37 Views

Open in Slack

Previous Next

Meltano

Meltano community