still alittle lost on how to pass stuff from the

meltano.yml

to the actual project code, for example i have this in the code:

@property
    def url_base(self) -> str:
        """Return the API URL root, configurable via tap settings."""
        # TODO: hardcode a value here, or retrieve it from self.config
        return "<https://pi.demo.pardot.com/api/v5/objects>"

instead of hardcoding, i want to get it from here:

version: 1
send_anonymous_usage_stats: false
project_id: tap-pardot
default_environment: dev
environments:
  - name: dev
    config:
      plugins:
        extractors:
          - name: tap-pardot
            load_schema: ${SCHEMA_NAME}
            credentials:
              - sfdc:
                client_key: ${SFDC_CLIENT_KEY}     
                client_secret: ${SFDC_CLIENT_SECRET}
                REDIRECT_URI: ${SFDC_REDIRECT_URI}
              - pardot:
        loaders:
          - name: target-redshift
            default_target_schema: ${SCHEMA_NAME}
    env:
      PARDOT_API_URL: <https://pi.demo.pardot.com>

do i just do

self.config.PARDOT_API_URL

?

In your meltano.yml

credentials

should be

config

. Usually you'd create a setting in your tap called something like

api_url

then in your meltano.yml you'd do:

plugins:
        extractors:
          - name: tap-pardot
            config:
              api_url: "<https://pi.demo.pardot.com>"

within your tap code you can then access it like

self.config["api_url"]

. Also you could use the

env

key in your meltano.yml if you wanted to, but its more unusual for this case, by setting it to

PARDOT_API_URL

(like you already have) then you can directly access it in your tap environment using

os.environ["PARDOT_API_URL"]

hmm so couple questions: 1. can i not have a custom key like

credentials

? 2. what if i want to nest it under

pardot/api_url

? 3. is there a way to log/print stuff, this way i can debug easier?

- name: dev
    config:
      plugins:
        extractors:
          - name: tap-pardot
            load_schema: ${SCHEMA_NAME}
            config:
              - sfdc:
                client_key: ${SFDC_CLIENT_KEY}     
                client_secret: ${SFDC_CLIENT_SECRET}
                redirect_uri: ${SFDC_REDIRECT_URI}
              - pardot:
                api_url: ${PARDOT_API_URL}

i tried something like

self.config['pardot']['api_url']

and got:

File "/projects/taps/tap-pardot/tap_pardot/client.py", line 30, in url_base
    return self.config['pardot']['api_url']
KeyError: 'pardot'

1. can i not have a custom key like

credentials

?

You can but they wont really do anything. Meltano interacts with taps by converting your relevant meltano.yml configs to a tap config.json file thats passed in using

--config config.json

like Singer taps expect. See the singer docs for more details

2. what if i want to nest it under

pardot/api_url

?

You could do that but with some limitations as of right now. Right now templating like

${PARDOT_API_URL}

within a nested array or object isnt supported, they dont get populated and will send your tap the string literal "${PARDOT_API_URL}" instead. There are ways around it and theres a feature issue to support it.

3. is there a way to log/print stuff, this way i can debug easier?

Yep

meltano config tap-pardot

see the docs

i tried something like

self.config['pardot']['api_url']

and got:

The config you shared is actually the yaml syntax for an array rather than a dict like youre expecting. You probably wont want to actually do this given the limitations I shared above but in the current state you'd probably need

self.config[0]['pardot']['api_url']

instead.

I'd also recommend getting your debugger set up so you can step through the tap code and see what the

self.config

actually looks like. You iteration cycles will be much faster. https://sdk.meltano.com/en/latest/dev_guide.html#vscode-debugging

oh i mean something like

print(self.config['pardot'])

How are you running the tap? It sort of depends I think

Maybe not as helpful but sometimes I just raise an exception to very loudly print it 😅

raise Exception('where can i see this print out?')

that works!