https://linen.dev logo
#plugins-general
Title
# plugins-general
d

david_elston

05/02/2022, 11:19 PM
Hi, I’m writing a custom tap to explore Meltano a little bit. I pretty much have everything working for the moment, but I’m a little confused on config specification. I define the spec of my config in the
tap
definition something like this
Copy code
config_jsonschema = th.PropertiesList(
        th.Property(
            "auth_token",
            th.StringType,
            required=True,
            description="The token to authenticate against Okta"
        ),
        th.Property(
            "api_url",
            th.StringType,
            required=True,
            description="The url for the API service"
        ),
        th.Property(
            "start_date",
            th.DateTimeType,
            description="When using the log stream, the earliest date to pull a record"
        ),
        th.Property(
            "log_page_limit",
            th.IntegerType,
            default=1000,
            description="The single page limit for system log stream"
        ),
        th.Property(
            "user_page_limit",
            th.IntegerType,
            default=200,
            description="The single page limit for user log stream"
        ),
    ).to_dict()
But if I run
meltano config tap-mytap list
this pulls the spec from the
meltano.yml
file and not from the tap’s config spec? And in particular the
env
variables are pulled from the yaml spec and not the plugin config spec. I suppose my question would be why do I have to define my config in two separate places?