Hey, I’m testing meltano with Airflow and I can’t seem to run a tap on schedule for a specific entity. It looks like I can schedule a tap and a target like

meltano schedule zendesk-csv tap-zendesk target-csv "30 3 * * *"

but I can’t just run zendesk_tickets or zendesk_groups at a specific schedule, is that right? My goal is to be able to have different schedules for different entities, so something like

meltano schedule zendesk-tickets-csv tap-zendesk target-csv --select=tickets --job_id=zendesk_tickets "30 3 * * *"
meltano schedule zendesk-groups-csv tap-zendesk target-csv --select=groups --job_id=zendesk_groups "30 6 * * *"

@narrow-analyst-73746 This sounds like a good use case for plugin inheritance, so that you can manage different configurations for the same underlying tap as differently named plugins: http://meltano.com/docs/plugins.html#plugin-inheritance

meltano add extractor tap-zendesk--tickets --inherit-from tap-zendesk
meltano config tap-zendesk--tickets set _select "['tickets']"

meltano add extractor tap-zendesk--groups --inherit-from tap-zendesk
meltano config tap-zendesk--groups set _select "['groups']"

meltano schedule zendesk-tickets-csv tap-zendesk--tickets target-csv "30 3 * * *"
meltano schedule zendesk-groups-csv tap-zendesk--groups target-csv "30 6 * * *"

Thanks @ripe-musician-59933! Plugin inheritance works like a charm 🙂 Just FYI,

meltano config tap-zendesk--tickets set _select "['tickets']"

throws an error:

bash-4.2# meltano --log-level=debug config tap-zendesk--tickets set _select "['tickets']"
[2021-01-14 11:59:55,244] [128|MainThread|root] [DEBUG] Creating engine <meltano.core.project.Project object at 0x7feb78b8f550>@sqlite:////artemis/.meltano/meltano.db
[2021-01-14 11:59:55,404] [128|MainThread|urllib3.connectionpool] [DEBUG] Starting new HTTPS connection (1): <http://www.meltano.com:443|www.meltano.com:443>
[2021-01-14 11:59:56,213] [128|MainThread|urllib3.connectionpool] [DEBUG] <https://www.meltano.com:443> "GET /discovery.yml HTTP/1.1" 200 83079
[2021-01-14 11:59:56,988] [128|MainThread|meltano.cli.utils] [DEBUG] Expecting value: line 1 column 2 (char 1)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/meltano/cli/__init__.py", line 43, in main
    cli(obj={"project": None})
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/meltano/cli/config.py", line 104, in set
    value, metadata = settings.set_with_metadata(
  File "/usr/local/lib/python3.8/site-packages/meltano/core/settings_service.py", line 289, in set_with_metadata
    cast_value = setting_def.cast_value(value)
  File "/usr/local/lib/python3.8/site-packages/meltano/core/setting_definition.py", line 171, in cast_value
    value = json.loads(value)
  File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/meltano/cli/__init__.py", line 51, in main
    raise CliError(str(err)) from err
meltano.cli.utils.CliError: Expecting value: line 1 column 2 (char 1)
Expecting value: line 1 column 2 (char 1)

But

meltano config tap-zendesk--tickets set _select '["tickets"]'

works

@narrow-analyst-73746 Ah, right, I messed up the quotes 😄