Hey, Folks! I'm trying to run in k8s a tap that i made, but i keep getting this message. Couldn't find much about it, any idea?

INFO - usage: tap-clickfrotas [-h] -c CONFIG [-s STATE] [-p PROPERTIES]
INFO -                        [--catalog CATALOG] [-d] [--dev]
INFO - tap-clickfrotas: error: argument -c/--config: expected one argument

Config:

version: 1
send_anonymous_usage_stats: true
project_id: tap-clickfrotas
default_environment: test
environments:
- name: test
plugins:
  extractors:
  - name: tap-clickfrotas
    namespace: tap_clickfrotas
    pip_url: -e .
    capabilities:
    - state
    - catalog
    - discover
    - about
    - stream-maps
    settings:
    - name: key
      sensitive: true
      value: $(TAP_CLICK_FROTAS_KEY)
    - name: days
      value: $(TAP_CLICK_FROTAS_DAYS)
    - name: filial
      value: $(TAP_CLICK_FROTAS_FILIAL)
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
  - name: target-s3
    variant: crowemi
    pip_url: git+<https://github.com/crowemi/target-s3.git>
    config:
      prefix: click_frotas/raw
      format:
        format_type: parquet
      append_date_to_prefix: false

Seems like meltano isn't sending the

config

option to your tap for some reason

Hey, thanks for the reply. So, locally and on a Docker container, it seems to run just fine. That is the result of the Meltano invoke tap-clickfrotas:

2024-04-03T19:08:54.883651Z [info     ] Environment 'test' is active
{"type":"STATE","value":{}}
2024-04-03 16:08:55,792 | INFO     | tap-clickfrotas.tickets | Beginning full_table sync of 'tickets'...
2024-04-03 16:08:55,792 | INFO     | tap-clickfrotas.tickets | Tap has custom mapper. Using 1 provided map(s).
{"type":"SCHEMA","stream":"tickets","schema":{"properties":{"dt":{"description":"Data de entrada na fabrica. Pode nao ser a data/hora de pesagem, mas sim a data que foi computadade a entrada do frota na fabrica ou fila de descarga.","type":["string","null"]},"dt_s":{"description":"Data de saida da fabrica","type":["string","null"]},"frota":{"type":["string","null"]},"doc":{"description":"Numero do documento de referencia (NF/Ticket/Guia CEM,...)","type":["integer","null"]},"prj":{"description":"Projeto","type":["string","null"]},"distancia":{"description":"Distancia do projeto","type":["integer","null"]},"peso_bruto":{"description":"Peso bruto da viagem. Lembrar de verificar com o setor responsavel na unidade sobre os cadados atualizados da tara/conjunto do frota.","type":["number","null"]},"peso":{"description":"Peso liquido","type":["number","null"]},"m3":{"type":["number","null"]},"vlrunit":{"description":"Valor unitario da tabela (TON).","type":["number","null"]},"vlrhrparado":{"type":["number","null"]},"gp_valor":{"type":["number","null"]},"total":{"description":"Valor total na NF/Ticket/Guia CEM (ou outro documento). Lembrando que esse valor e composto de outros valores menores como (se aplicavel): carregamento, tempo de espera, gestao de peso, entre outros. Consulte o suporte para obter mais informacoes.","type":["number","null"]},"madeira":{"description":"Tipo de madeira","type":["string","null"]},"altura_programada":{"type":["number","null"]},"altura_carregada":{"type":["number","null"]},"altura_resultado":{"type":["string","null"]}},"type":"object"},"key_properties":["doc"]}
2024-04-03 16:08:57,100 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "http_request_duration", "value": 1.184917, "tags": {"stream": "tickets", "endpoint": "Key=12345&dias=2&filial=2", "http_status_code": 200, "status": "succeeded"}}
2024-04-03 16:08:57,155 | INFO     | singer_sdk.helpers.jsonpath | JSONPath $.data[*] match count: 70
{"type":"RECORD","stream":"tickets","record":{"dt":"01/04/2024 16:11:00","dt_s":"01/04/2024 16:50:00","frota":"22390","doc":2100117,"prj":"S\u00c3O JOS\u00c9 III","distancia":215,"peso_bruto":59.25,"peso":38.55,"m3":43.54,"vlrunit":115.14,"vlrhrparado":-33.83,"gp_valor":0,"total":4666.49,"madeira":"COM CASCA","altura_programada":2.7,"altura_carregada":2.48,"altura_resultado":"Abaixo"},"time_extracted":"2024-04-03T19:08:57.155651+00:00"}
{"type":"RECORD","stream":"tickets","record":{"dt":"02/04/2024 03:48:00","dt_s":"02/04/2024 04:26:00","frota":"22053","doc":2100121,"prj":"S\u00c3O JOS\u00c9 III","distancia":215,"peso_bruto":57.53,"peso":36.71,"m3":42.96,"vlrunit":115.14,"vlrhrparado":-35.44,"gp_valor":0,"total":4449.54,"madeira":"COM CASCA","altura_programada":2.6,"altura_carregada":2.51,"altura_resultado":"Abaixo"},"time_extracted":"2024-04-03T19:08:57.155821+00:00"}
{"type":"RECORD","stream":"tickets","record":{"dt":"02/04/2024 03:52:00","dt_s":"02/04/2024 04:32:00","frota":"2159","doc":2100067,"prj":"S\u00c3O JOS\u00c9 III","distancia":215,"peso_bruto":56.98,"peso":36.06,"m3":42.21,"vlrunit":115.14,"vlrhrparado":-32.22,"gp_valor":0,"total":4373.41,"madeira":"COM CASCA","altura_programada":2.6,"altura_carregada":2.41,"altura_resultado":"Abaixo"},"time_extracted":"2024-04-03T19:08:57.155933+00:00"}

(...)

{"type":"RECORD","stream":"tickets","record":{"dt":"02/04/2024 22:55:00","dt_s":"02/04/2024 23:48:00","frota":"22387","doc":2100227,"prj":"S\u00c3O JOS\u00c9 III","distancia":215,"peso_bruto":52.53,"peso":31.94,"m3":43.85,"vlrunit":114.59,"vlrhrparado":-11.28,"gp_valor":512.22,"total":4424.48,"madeira":"COM CASCA","altura_programada":2.4,"altura_carregada":2.5,"altura_resultado":"Acima"},"time_extracted":"2024-04-03T19:08:57.161813+00:00"}
{"type":"RECORD","stream":"tickets","record":{"dt":"02/04/2024 23:12:00","dt_s":"02/04/2024 23:57:00","frota":"22380","doc":2100231,"prj":"S\u00c3O JOS\u00c9 III","distancia":215,"peso_bruto":53.03,"peso":32.23,"m3":43.33,"vlrunit":114.59,"vlrhrparado":-24.16,"gp_valor":0,"total":3929.49,"madeira":"COM CASCA","altura_programada":2.7,"altura_carregada":2.45,"altura_resultado":"Abaixo"},"time_extracted":"2024-04-03T19:08:57.161893+00:00"}
{"type":"RECORD","stream":"tickets","record":{"dt":"03/04/2024 00:48:00","dt_s":"03/04/2024 01:47:00","frota":"22004","doc":2100233,"prj":"S\u00c3O JOS\u00c9 III","distancia":215,"peso_bruto":53.69,"peso":33.6,"m3":44.72,"vlrunit":114.59,"vlrhrparado":-1.61,"gp_valor":379.29,"total":4496.67,"madeira":"COM CASCA","altura_programada":2.6,"altura_carregada":2.51,"altura_resultado":"Abaixo"},"time_extracted":"2024-04-03T19:08:57.161972+00:00"}
2024-04-03 16:08:57,178 | INFO     | singer_sdk.helpers.jsonpath | JSONPath $.next_page match count: 0
2024-04-03 16:08:57,178 | INFO     | singer_sdk.metrics   | METRIC: {"type": "counter", "metric": "http_request_count", "value": 1, "tags": {"stream": "tickets", "endpoint": "Key=12345&dias=2&filial=2"}}
2024-04-03 16:08:57,178 | INFO     | singer_sdk.metrics   | METRIC: {"type": "timer", "metric": "sync_duration", "value": 1.3859155178070068, "tags": {"stream": "tickets", "context": {}, "status": "succeeded"}}
2024-04-03 16:08:57,178 | INFO     | singer_sdk.metrics   | METRIC: {"type": "counter", "metric": "record_count", "value": 70, "tags": {"stream": "tickets", "context": {}}}
{"type":"STATE","value":{"bookmarks":{"tickets":{}}}}

cool so the tap works, now you just have to share how you're running in k8s and what's different :)_

Oh, of course, sorry about that. Here is my Airflow task:

how's the image created 🙂

That is the Dockerfile:

Something doesn't line up to me, the initial logs don't match that docker file to me. I'd dive into that myself. Unfortunately this is a k8s / airflow configuration issue as the meltano side looks good to me

Ok ok, no problem. Thanks for your time!

The error at the top of the thread makes me think the tap was invoked outside of Meltano (?)