daniel
10/25/2023, 3:54 PMmeltano
to emit the SCHEMA messages coming from a tap for debugging purposes? i tried meltano --log-level=debug ...
but it didn't seem to workvisch
10/25/2023, 3:55 PMmeltano invoke tap-name > outfile
daniel
10/25/2023, 4:01 PMinvoke
works 👍daniel
10/25/2023, 4:39 PMtarget-redshift
which uses the schema records to construct a table if one doesn't exist already
these schema records need to include all available fields and not just the ones that are being selected so that it can construct a table capable of fitting fields that might be added later
however, i've created a custom tap using the Meltano SDK and noticed that it's only emitting a schema that includes records that happen to be selected on a specific run, which is not good for the reason above
is there some way that i can force Meltano to emit a full schema in a tap generated using the SDK?