aaronsteers
06/25/2021, 3:04 AMv0.1.3
SDK release, including Targets and Stream Maps. 🚀
Special ty to @visch, @andrew_stewart, and everyone else who helped pilot the prerelease and provided much-valued feedback during development.brandon_isom
06/25/2021, 9:22 PMtarget
not tap here: https://gitlab.com/meltano/sdk/-/blob/main/cookiecutter/target-template/%7B%7Bcookiecutter.library_name%7D%7D/pyproject.toml#L22
• And should have a cli = Target{{ cookiecutter.destination_name }}.cli
at the end of this file. https://gitlab.com/meltano/sdk/-/blob/main/cookiecutter/target-template/%7B%7Bcook[…]rary_name%7D%7D/%7B%7Bcookiecutter.library_name%7D%7D/target.py
Let me know if you'd like me to whip up an issue for itaaronsteers
06/25/2021, 9:40 PMaaronsteers
06/25/2021, 10:05 PMpyproject.toml
?
{{cookiecutter.target_id}} = '{{cookiecutter.library_name}}.target:Target{{ cookiecutter.destination_name }}.cli'
brandon_isom
06/25/2021, 10:40 PMaaronsteers
06/25/2021, 11:16 PMbrandon_isom
06/26/2021, 12:56 AMtap-carbon-intensity | INFO Loaded region Yorkshire
target-test | time=2021-06-25 17:47:29 name=target-test level=INFO message=Initializing 'target-test' target sink...
target-test | time=2021-06-25 17:47:29 name=target-test level=INFO message=Initializing target sink for stream 'generationmix'...
target-test | Traceback (most recent call last):
target-test | File "<string>", line 1, in <module>
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 829, in __call__
target-test | return self.main(*args, **kwargs)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 782, in main
target-test | rv = self.invoke(ctx)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
target-test | return ctx.invoke(self.callback, **ctx.params)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 610, in invoke
target-test | return callback(*args, **kwargs)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 343, in cli
target-test | target.listen()
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 129, in listen
target-test | self._process_lines(sys.stdin)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 188, in _process_lines
target-test | self._process_record_message(line_dict)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 223, in _process_record_message
target-test | sink._validate_record(transformed_record)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/sinks.py", line 180, in _validate_record
target-test | self._validator.validate(record)
target-test | File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/jsonschema/validators.py", line 353, in validate
target-test | raise error
target-test | jsonschema.exceptions.ValidationError: datetime.datetime(2018, 5, 15, 11, 30, tzinfo=tzutc()) is not of type 'string'
target-test |
target-test | Failed validating 'type' in schema['properties']['from']:
target-test | {'format': 'date-time', 'type': 'string'}
target-test |
target-test | On instance['from']:
target-test | datetime.datetime(2018, 5, 15, 11, 30, tzinfo=tzutc())
meltano | Loading failed (1): datetime.datetime(2018, 5, 15, 11, 30, tzinfo=tzutc())
meltano | ELT could not be completed: Loader failed
ELT could not be completed: Loader failed
Looks like validate_timestamps_in_record()
will convert to a python datetime before the jsonschema validator hits it.
• https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/sinks.py#L177-180
• https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/sinks.py#L197
I just looked up and it's 6pm tho, so I'm walking away, hahaaaronsteers
06/26/2021, 6:38 AM