Hi! I’m just getting started with Meltano. Got my first project up and running. It’s using inthe

tap-mysal

to

target-postgres

. I’m using the pipelinewise mysql tap so I can use the

LOG_BASED

replication. The first EL works great (when it’s just doing a

FULL_TABLE

replication). The second time I run it I get a postgres error below because of what appears to be an operation timestamp format error. Does anyone have an idea as to what’s going on here?

target_postgres.exceptions.PostgresError: ('Exception writing records', ParserError("Could not match input '2020-12-02T23:27:34+00:00+00:00' to any of the following formats: YYYY-MM-DDTHH:mm:ssZZ...

I realize that this is a meltano Slack channel and not piplinewise but I thought someone here might have some insight. Thanks!

@matt_cooley The timestamp is invalid because of the

+00:00

suffix repeated twice. Do you know where that value may be coming from? If you run

meltano elt

in debug mode (https://meltano.com/docs/command-line-interface.html#debugging,

meltano --log-level=debug elt

), you should see all messages output by the tap printed in your terminal identified by

tap-mysql (out)

, so we can determine which

RECORD

message is the culprit!

It appears to be an operational column (

_sdc_deleted_at

)recording a delete in the replication. I’m using the singer version of the Postgres Target. I’m going to try the piplinewise version and see if they’ve added some sugar to help with this

Did you find out where the timestamp is being generated with the double TZ suffix?

Yeah, it’s from the piplinewise tap-mysql. That’s the

_sdc_deleted_at

operational column I was referring to.

@matt_cooley Interesting, I don’t know why it would be generating invalid timestamps! You may want to look at the tap source code to see where this is happening, because I expect that other targets will also have trouble with this, since the value is simply invalid

ohhhhhh. I gotcha. That makes sense. I’m not 100% familiar with all the valid timestamp formats. Now that I look at it…yeah,

2020-12-02T23:27:34+00:00+00:00

doesn’t seem to make much sense at all

@matt_cooley All right, good to hear! That one may be more accepting of invalid formats 🙂

Ha! Totally fair