I have an extractor that guesses the wrong data ty...
# troubleshooting
b
I have an extractor that guesses the wrong data type, is there a way to override the data type of a column? Basically harrd-code?
b
Yeah, just saw this: `
Copy code
export TAP_POSTGRES__SCHEMA_SOME_STREAM_ID_CREATED_AT_FORMAT=date
`
That seems to be the thing I need!
Hmm, doesn't to do what I expect. When I set this to string, it complains about finding an integer. What I really want is to force string type in the tap output, regardless of the input.
v
hmmm, I'd hope the catalog override would work. If not you could look at mapping the data with a small mapper with meltano. Probably the easiest is to work on the tap side and find out why it can't convert that to a string. I dont' know I guess it depends on what you're after if its quick and dirty a mapper would work
b
Just looking at this mapper thing (https://gitlab.com/meltano/meltano/-/issues/2299), so that's all implemented right? Using latest Meltano, 2.10.0.
Hmm, so far the last one isn't working:
Copy code
mappers:
   - name: meltano-map-transformer
     pip_url: "git+<https://github.com/MeltanoLabs/meltano-map-transform.git>"
     executable: meltano-map-transform
     mappings:
       - name: force_string
         config:
          stream_maps:
            NSP-PLAN_TABLE_ALL:
              BYTES: str(BYTES)
Is there an easy way to see what the output schema looks like after a map?
Ah it's actually working it seems, it's just another column now with the same error.
And that worked perfectly, great tip!