Not sure the best channel to ask this in, so I’ll ...
# announcements
j
Not sure the best channel to ask this in, so I’ll ask it here 🙂. I’m trying to add the Github tap (https://github.com/singer-io/tap-github) (yes I realize the irony in asking GitLab for help 😜). I’ve run the following commands successfully:
Copy code
meltano select tap-github --list --all
meltano select tap-github 'issues'.'*'
meltano elt tap-github target-sqlite
When running the GitHub tap alone, I see all the issues from the defined repositories. I’m trying to understand how Meltano constructs the properties JSON. When I look at the
node
argument in
SelectExecutor#update_node_selection
, I only see the
'selected': True
attribute on the metadata id attribute:
Copy code
...	
'metadata': [{
		'breadcrumb': [],
		'metadata': {
			'table-key-properties': ['id'],
			'selected': True
		}
...
m
That is the stream's metadata, denoted by the empty breadcrumb. So this marks the stream to be selected.
Your problem lies in the fact that
tap-github
seems to use the legacy format (i.e. metadata on the
schema
directly).
You can see at https://gitlab.com/meltano/meltano/blob/master/src/meltano/core/plugin/singer/catalog.py#L154-173 that we try to support legacy catalog by setting the
selected: True
on the
stream
node directly.
I would say that this tap doesn't follow the Singer spec, as per https://github.com/singer-io/getting-started/blob/master/docs/DISCOVERY_MODE.md#the-catalog
If you want you can add the needed key in the
stream_node
function.