Hi! I'm trying to configure tap-kafka (airbyte's v...
# getting-started
k
Hi! I'm trying to configure tap-kafka (airbyte's version) and experiencing error with connection to Kafka. After passing all the validation issues (some parameters were missed) I've to this error:
Copy code
tap_airbyte.tap.AirbyteException: java.lang.NullPointerException: Cannot invoke "com.fasterxml.jackson.databind.JsonNode.asText()" because the return value of "com.fasterxml.jackson.databind.JsonNode.get(String)" is null
But if I change user/pass to wrong values I still get this error, so I guess this is not an issue with deserializing data from kafka. Here's config:
Copy code
plugins:
  extractors:
    - name: tap-kafka
      variant: airbyte
      pip_url: git+<https://github.com/MeltanoLabs/tap-airbyte-wrapper.git>
      config:
        airbyte_config:
          bootstrap_servers: <http://rc1a-xxxx.yandexcloud.net:9091|rc1a-xxxx.yandexcloud.net:9091>
          MessageFormat: 
            deserialization_type: "JSON"
          subscription:
            subscription_type: "subscribe"  # Using subscribe method
            topic_pattern: "topic_name_here"
          protocol:
            security_protocol: "SASL_SSL"
            sasl_mechanism: "SCRAM-SHA-512"
            sasl_jaas_config: "org.apache.kafka.common.security.scram.ScramLoginModule required username='user' password='password';"
          client_id: "meltano-consumer"
          group_id: "meltano_consumer_groupv1"
          enable_auto_commit: true
          auto_commit_interval_ms: 5000
          auto_offset_reset: "earliest"
          max_poll_records: 500
          polling_time: 100
Any ideas or recommendations on how to connect to Kafka via SASL_SSL?
v
I havne't messed with the airbyte taps myself, but if I was having this error, I'd try to get this running directly with airbyte or I"d dive into the full output maybe the wrapper is cutting is off? We're missing logs somewhere, and I have no idea if that's the airbyte wrapper, or if it's the airbyte tap itself just giving you a poor error message in which case you'll have to dive into the tap