hi all, I have been working to get a survey monkey tap stood up and am getting turned around on the config I was able to get the tap installed, and performed a test and was returned this error:

CRITICAL Config is missing required keys: ['start_date', 'access_token']

I tried troubleshooting a couple of ways, not sure either correct. 1. I modified the plugins>extractors>tap-surveymonkey--singer.io.lock to include the above (access token is aliased) 2. The other is I added the above as config items in our environments config file for the extractor environments>environment.yml a.

environments:

b.

- name: env

c.

config:

d.

plugins:

e.

extractors:

f.

- name: tap-surveymonkey

g.

config:

h.

start_date: '2023-01-01T00:00:00Z'

i.

access_token: $(alias)

I am now met with:

Plugin configuration is invalid
Exception: key_properties must be a string or list of strings

any thoughts are appreciated

You probably don't want to edit the lock file you described in step 1 which I believe is leading to the plugin configuration error. You should be able to re-install the plugin to undo those changes. As for the original errors about missing configuration, try using "meltano config tap-surveymonkey list" to verify that your configuration options are being picked up correctly from the appropiate source (yml, database, environment, etc)

thank you for the follow up! I did revert and uninstall, then installed the variant version for the survey monkey tap (danladd) appeared to make a bit more progress, but when running in target env, I am now stumbling through a: "Plugin configuration is invalid Executable 'tap-surveymonkey' could not be found. Extractor 'tap-surveymonkey' may not have been installed yet using

meltano install extractor tap-surveymonkey

, or the executable name may be incorrect." error

Might be worth installing fresh to clear up any virtual env issues:

meltano install extractor tap-surveymonkey --clean

thank you again for helping. I ended up reverting back from the danladd version and went with a clean install of the default surveymonkey tap. I manually set the start_date and access_token, but when executing the test I am returned this error again (full circle): Plugin configuration is invalid Exception: key_properties must be a string or list of strings

I think the tap is producing an invalid catalog. You can override the key properties using metadata:

plugins:
  extractors:
  - name: tap-surveymonkey
    metadata:
      "*":
        key-properties: ["id"]

Are you trying to pass in the access token as an environment variable? If so, you don't have to put it anywhere in your YML configs, simply make sure it's set as

TAP_SURVEYMONKEY_ACCESS_TOKEN

and the tap will pick it up directly

thank you both. the key-properties override got the test complete I did have in the .env the protected token, and then was trying to point the config to that env var

You're welcome! I realize now that talking about "environments" can be a little confusing between Meltano's environments and actual environment variables. 😅