https://linen.dev logo
Join Slack
Powered by
:question: I am trying to see output of tap-openwe...
# getting-started
j
❓ I am trying to see output of tap-openweathermap in target json-l. I have installed the extractor and went through meltano config tap-openweathermap set --interactive providing all of the entries. The api key was added to my .env and the other entries to meltano.yml. At this point i would expect meltano config tap-openweathermap test would succeed. (I have validated my key is valid using thunderclient in vscode). I am a little confused by this part after trying to test the config The default environment ‘dev’ will be ignored for 
meltano config
. To configure a specific environment, please use the option 
--environment=<environment name>
.
t
We have a single line in our environments documentation about this: “Default environments are intended for execution related work and not configuration.” Meaning that invoke, elt, run, and test will use the default environment since those are typically doing work for the user. config is different and we felt it best not to use the default environment in that case and make it an explicit option to add config for a specific environment. So the message you’re seeing basically says, “to see the config for a given enviornment, run 
meltano --environment=<yourenv> config <plugin> test
”
j
Thanks, this is helpful. Should it even allow you to declare the config?
Also once going through and setting the config for dev i am still getting plugin configuration is invalid. Is there anything special I need to do to load the .env file for the dev environment?
t
Should it even allow you to declare the config?
I’m not sure I understand what you’re asking sorry. To clarify my answer more, the default for config is to set the values globally for your extractor. You can choose to set it specifically for an environment, but it’s not necessary. When you run invoke or test (or other pipelline-related command) it will read from the different configuration layers (https://docs.meltano.com/guide/configuration#configuration-layers), starting with environment variables (via .env as well). So you shouldn’t have to do anything special to get it to use the .env. Can you share your meltano.yml and the exact commands you were running?/
j
Copy code
version: 1
default_environment: dev
project_id: 3c178417-5a07-4152-a4ff-0f0507f0a327
environments:
- name: dev
  config:
    plugins:
      extractors:
      - name: tap-openweathermap
        config:
          current_weather_city_name: Louisville
          forecast_weather_longitude: -85.7634
          forecast_weather_lattitude: 38.2438
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-openweathermap
    variant: matatika
    pip_url: git+<https://github.com/Matatika/tap-openweathermap.git>
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
Copy code
.env TAP_OPENWEATHERMAP_API_KEY='<key>'
Copy code
meltano --environment=dev config tap-openweathermap test
2022-11-22T02:00:42.269404Z [info     ] Environment 'dev' is active
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
JSONSchema was: {'type': 'object', 'properties': {'api_key': {'type': ['string'], 'description': 'API Key is a required parameter to query the api endpoints'}, 'current_weather_city_name': {'type': ['string'], 'description': 'City name that you want to get weather for'}, 'forecast_weather_longitude': {'type': ['string'], 'description': 'Longitude of city to get forecast for'}, 'forecast_weather_lattitude': {'type': ['string'], 'description': 'Lattitude of city to get forecast for'}}, 'required': ['api_key', 'current_weather_city_name', 'forecast_weather_longitude', 'forecast_weather_lattitude']}
t
If you do 
meltano --environment=dev config tap-openweathermap
does it list all of the config properly?
oh - I wonder if the lat/long needs to be quoted
j
Ah good point let me try quoting them
Yes, that was it
Thanks for the explanation of envs and figuring out it was the quotes
t
oof - better error messages continue to be a great need for us
yeah no problem! happy to help 😄
Looks like we’ll have a PR out to fix this! https://github.com/meltano/meltano/pull/7017/files 😄
2 Views
Open in Slack
PreviousNext
Powered by