https://linen.dev logo
Join Slack
Powered by
Hello, guys! I'm having some problems trying to us...
# singer-taps
i
Hello, guys! I'm having some problems trying to use tap-postgres and I don't know exactly what's going on.
r
Can't quite see from the screenshot, but looks like it could be a schema type issue for the 
ordinal_position
property (something like expected string, got number).
d
Try to run it with 
--log-level=debug
option
i
@Denis I. @Reuben (Matatika)
r
Have to say I've never found those logs very helpful. I would do https://meltano.slack.com/archives/C069CQNHDNF/p1713474297844809?thread_ts=1713458172.389959&cid=C069CQNHDNF so you can see the error at the target more clearly.
i
Using meltano invoke tap-postgres > tap.out and then at tap.out | meltano invoke target-csv I had this issue:
Maybe is exctly what you told me in your first response, @Reuben (Matatika)
r
Yeah, so the tap is discovering that property as a string but the extracted record value is an integer. Which variant of the tap are you using? Do you know the data type of that column?
For the record, you can override the tap schema: https://docs.meltano.com/concepts/plugins/#how-to-use-3
i
I'm using postgres
r
The default variant (
meltanolabs
), I assume then?
i
Yep, the default
The datatype I don't know
@Reuben (Matatika) bpchar, I think
Yep
r
Looked at the code - I don't think there's a registered type mapping for that data type, so it is just falling back to a string.
Probably worth opening an issue for if you have the time. šŸ™‚
In the meantime, you can workaround it by overriding the schema for that property: https://meltano.slack.com/archives/C06A1MD6A6L/p1717614147353549?thread_ts=1717605228.789809&cid=C06A1MD6A6L You also might find it works with other targets that are less strict about record-schema validation.
i
I'll see, yesteday I started working with overriding the schemas
But it was getting huge
"information_schema-parameters: numeric_scale: type: ["integer", "null"] ordinal_position: type: ["integer", "null"] numeric_precision_radix: type: ["integer", "null"] information_schema-key_column_usage: ordinal_position: type: ["integer", "null"] information_schema-attributes: ordinal_position: type: ["integer", "null"] numeric_scale: type: ["integer", "null"] numeric_precision: type: ["integer", "null"] numeric_precision_radix: type: ["integer", "null"] character_maximum_length: # ModificaĆ§Ć£o para aceitar integer type: ["integer", "null"] information_schema-columns: numeric_scale: type: ["integer", "null"] ordinal_position: type: ["integer", "null"] numeric_precision: type: ["integer", "null"] numeric_precision_radix: type: ["integer", "null"] character_maximum_length: # ModificaĆ§Ć£o para aceitar integer type: ["integer", "null"] information_schema-element_types: numeric_precision: type: ["integer", "null"] numeric_scale: type: ["integer", "null"] numeric_precision_radix: type: ["integer", "null"] character_maximum_length: # ModificaĆ§Ć£o para aceitar integer type: ["integer", "null"] information_schema-routines: numeric_scale: type: ["integer", "null"] numeric_precision: type: ["integer", "null"] numeric_precision_radix: type: ["integer", "null"] character_maximum_length: # ModificaĆ§Ć£o para aceitar integer type: ["integer", "null"] information_schema-sequences: numeric_scale: type: ["integer", "null"] numeric_precision: type: ["integer", "null"] numeric_precision_radix: type: ["integer", "null"] character_maximum_length: # ModificaĆ§Ć£o para aceitar integer type: ["integer", "null"] information_schema-user_defined_types: numeric_precision: type: ["integer", "null"] numeric_precision_radix: type: ["integer", "null"] character_maximum_length: # ModificaĆ§Ć£o para aceitar integer type: ["integer", "null"]"
Then I stopped
I don't know if there's a better way
r
So you are finding this issue with a lot of fields, not just 
ordinal_position
?
i
Yep, I solved the ordinal_position
But the problem is just moving column to column
r
Hmm... Maybe just deselect the 
information_schema
streams then? You probably don't care about that data anyway.
Copy code
meltano select tap-postgres --exclude 'information_schema-*'
meltano select tap-postgres --all
šŸ‘ 1
5 Views
Open in Slack
PreviousNext
Powered by