<!channel> - I'm thrilled to announce this latest ...
# singer-tap-development
a
<!channel> - I'm thrilled to announce this latest
v0.1.3
SDK release, including Targets and Stream Maps. 🚀 Special ty to @visch, @andrew_stewart, and everyone else who helped pilot the prerelease and provided much-valued feedback during development.
b
Not sure if it really needs a full-blown issue, or if you've got one open to collect the minor stuff here and there, but there's a couple of minor issues with the template • Should be
target
not tap here: https://gitlab.com/meltano/sdk/-/blob/main/cookiecutter/target-template/%7B%7Bcookiecutter.library_name%7D%7D/pyproject.toml#L22 • And should have a
cli = Target{{ cookiecutter.destination_name }}.cli
at the end of this file. https://gitlab.com/meltano/sdk/-/blob/main/cookiecutter/target-template/%7B%7Bcook[…]rary_name%7D%7D/%7B%7Bcookiecutter.library_name%7D%7D/target.py Let me know if you'd like me to whip up an issue for it
a
This is perfect -thanks @brandon_isom! We'll get both of these in later today! Since the cookie cutter templates aren't tied to the Python / PyPi release directly, those fixes will take effect immediately for future use.
@brandon_isom - Rather than adding the line to target.py, can you test if this direct reference works for you in
pyproject.toml
?
Copy code
{{cookiecutter.target_id}} = '{{cookiecutter.library_name}}.target:Target{{ cookiecutter.destination_name }}.cli'
b
Sure does!
a
Merged! Thanks, @brandon_isom!
b
Was kicking the tires a bit more. I think the validation ordering might be a bit off. I'll whip up an issue on Monday. Error output
Copy code
tap-carbon-intensity | INFO Loaded region Yorkshire
target-test          | time=2021-06-25 17:47:29 name=target-test level=INFO message=Initializing 'target-test' target sink...
target-test          | time=2021-06-25 17:47:29 name=target-test level=INFO message=Initializing target sink for stream 'generationmix'...
target-test          | Traceback (most recent call last):
target-test          |   File "<string>", line 1, in <module>
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 829, in __call__
target-test          |     return self.main(*args, **kwargs)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 782, in main
target-test          |     rv = self.invoke(ctx)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
target-test          |     return ctx.invoke(self.callback, **ctx.params)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/click/core.py", line 610, in invoke
target-test          |     return callback(*args, **kwargs)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 343, in cli
target-test          |     target.listen()
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 129, in listen
target-test          |     self._process_lines(sys.stdin)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 188, in _process_lines
target-test          |     self._process_record_message(line_dict)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/target_base.py", line 223, in _process_record_message
target-test          |     sink._validate_record(transformed_record)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/singer_sdk/sinks.py", line 180, in _validate_record
target-test          |     self._validator.validate(record)
target-test          |   File "/snip/Library/Caches/pypoetry/virtualenvs/target-test-ACqxhjHu-py3.7/lib/python3.7/site-packages/jsonschema/validators.py", line 353, in validate
target-test          |     raise error
target-test          | jsonschema.exceptions.ValidationError: datetime.datetime(2018, 5, 15, 11, 30, tzinfo=tzutc()) is not of type 'string'
target-test          | 
target-test          | Failed validating 'type' in schema['properties']['from']:
target-test          |     {'format': 'date-time', 'type': 'string'}
target-test          | 
target-test          | On instance['from']:
target-test          |     datetime.datetime(2018, 5, 15, 11, 30, tzinfo=tzutc())
meltano              | Loading failed (1):     datetime.datetime(2018, 5, 15, 11, 30, tzinfo=tzutc())
meltano              | ELT could not be completed: Loader failed
ELT could not be completed: Loader failed
Looks like
validate_timestamps_in_record()
will convert to a python datetime before the jsonschema validator hits it. • https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/sinks.py#L177-180 • https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/sinks.py#L197 I just looked up and it's 6pm tho, so I'm walking away, haha
a
I think your assessment is correct, and I've started an update based on this feedback. Logged issue with draft MR.