Lets start a thread on this :point_down:
# getting-starteda
adam_rudd
10/04/2021, 4:24 AMLets start a thread on this 👇
adam_rudd
10/04/2021, 4:24 AMhere
adam_rudd
10/04/2021, 4:24 AMSanthosh  [3:23 PM]
https://gitlab.com/vepalisantosh/tap1/-/tree/master
GitLab
Files · master · vempali santhosh / tap1
GitLab.com
adam_rudd
10/04/2021, 4:26 AMDid you try
meltano uitap-weatheradam_rudd
10/04/2021, 4:27 AMmaybe start with a
jsonls
santhosh
10/04/2021, 4:28 AMYes
santhosh
10/04/2021, 4:28 AMEverthing is fine
santhosh
10/04/2021, 4:28 AMBut showing str assignment error with postgres
a
adam_rudd
10/04/2021, 4:28 AM# Test invocation:
meltano invoke tap-genesyscloud --versioneltmeltano elt tap-genesyscloud target-jsonladam_rudd
10/04/2021, 4:29 AMoh str-> varchar
s
santhosh
10/04/2021, 4:29 AMsanthosh
10/04/2021, 4:29 AMThis rep of singer tap ive made
santhosh
10/04/2021, 4:29 AMJust look at init.py
a
adam_rudd
10/04/2021, 4:29 AMcant clone this right now man sorry using work machine
adam_rudd
10/04/2021, 4:29 AMbut yeah lets look at the init
s
santhosh
10/04/2021, 4:29 AMIn tap weather folder
a
adam_rudd
10/04/2021, 4:32 AMis this a forked singer tap?
adam_rudd
10/04/2021, 4:32 AMI
s
santhosh
10/04/2021, 4:32 AMNo
santhosh
10/04/2021, 4:32 AMIve built it locally
a
adam_rudd
10/04/2021, 4:32 AMso you wrote this tap?
s
santhosh
10/04/2021, 4:32 AMYes
a
adam_rudd
10/04/2021, 4:33 AMis it outputting with something like
target-jsonladam_rudd
10/04/2021, 4:33 AMwhere is the root issue in your pipeline?
adam_rudd
10/04/2021, 4:34 AMis it the target part or the tap part?
adam_rudd
10/04/2021, 4:35 AMbtw theres a meltano rest steam class if oyu use the meltano sdk
s
santhosh
10/04/2021, 4:37 AMYes
santhosh
10/04/2021, 4:37 AMSee
santhosh
10/04/2021, 4:38 AMmessage has been deleted
santhosh
10/04/2021, 4:38 AMJson file is like this
santhosh
10/04/2021, 4:43 AMFor target json
santhosh
10/04/2021, 4:43 AMExtracting complete
santhosh
10/04/2021, 4:43 AMBut it is outputting as above
a
adam_rudd
10/04/2021, 5:12 AMcool,! Okay, are you outputting each character intentionally?
adam_rudd
10/04/2021, 5:13 AMThat is not json-like. Each entry in your json should by a json object.
eg.
{"itemname":"itemvalue", "anothername":"anothervalue"}adam_rudd
10/04/2021, 5:14 AMgiven the postgres target will be expecting dictionaries, this might be your root cause
adam_rudd
10/04/2021, 5:17 AMtry ``singer.write_records('employees', records=[DATA])``
adam_rudd
10/04/2021, 5:17 AMwrapping DATA in an array
s
santhosh
10/04/2021, 5:20 AMNo
santhosh
10/04/2021, 5:20 AMIf u can see init file in tap-wetaher
santhosh
10/04/2021, 5:20 AMI was just passing
santhosh
10/04/2021, 5:20 AMDictionary
santhosh
10/04/2021, 5:20 AMTo records
santhosh
10/04/2021, 5:29 AMmessage has been deleted
santhosh
10/04/2021, 5:29 AMAfter trying above
santhosh
10/04/2021, 5:29 AMMethod
santhosh
10/04/2021, 5:30 AMRecords=[data]
a
adam_rudd
10/04/2021, 5:37 AMokay, but those records are still not valid json
adam_rudd
10/04/2021, 5:37 AMthey’re escaped strings
adam_rudd
10/04/2021, 5:37 AMso it’ll likely fail to process because it’s still not in a form that the target will be expecting
adam_rudd
10/04/2021, 5:38 AMtry removing the json.dumps()
s
santhosh
10/04/2021, 5:56 AMKk ill try
santhosh
10/04/2021, 6:15 AMThanks man @adam_rudd its working fine
a
adam_rudd
10/04/2021, 6:17 AMwoo!