Hello, I'm experiencing a weird behavior where a job just freezes sometimes and nothing happens for upwards of 10 hours. I don't know exactly what goes wrong, if it is the source database ( I suspect it is, but the DBA is very sure the database works just as expected) or if something in meltano freezes, or if there is some weird behavior in cloud run that is causing it. I thought about setting log level to debugging to figure it out, but is there a way of doing that without writing out every record? Has anyone experienced something similar or have any other idea on how I can approach debugging this?

I thought about setting log level to debugging to figure it out, but is there a way of doing that without writing out every record?

https://github.com/meltano/meltano/issues/8760 might be useful (do 👍 it if it is)

Has anyone experienced something similar or have any other idea on how I can approach debugging this?

I've seen something similar, where a container randomly freezes after Meltano finishes installing plugins, but I haven't been able to identify a root cause.

It didn't seem to change anything in the output. So I tried changing to this ( see full meltano.yml file bellow, I change to debug on both ) and it still doesn't change anything in the output. Am I doing it incorrectly? I'm running on docker using this this image

meltano/meltano:v3.5.4-python3.9

version: 1

disable_existing_loggers: no
loggers:
  # Disable logging of tap and target stdout
  meltano.core.block.extract_load:
    level: DEBUG
root:
  level: DEBUG
  handlers: [console]

state_backend:
  uri: $MELTANO_STATE_BUCKET
send_anonymous_usage_stats: false
default_environment: prod
project_id: b3513b63-920c-47c0-a8a1-9bb7c3bef677
environments:
- name: dev
- name: staging
- name: prod

include_paths:
- ./load_configs/*.yaml

plugins:
  extractors:
  - name: tap-oracle
    variant: s7clarke10
    pip_url:
      git+<https://github.com/s7clarke10/pipelinewise-tap-oracle.git@53bb75ed27d7796d2f492e74cec87f10f1bce4d4>
    config:
      use_ora_rowscn: false
      ora_python_driver_type: thick
      use_singer_decimal: true

  - name: tap-testsource
    namespace: tap-testsource
    pip_url: -e extract/tap-testsource
    executable: tap-testsource
    config:
      test_val: foo

  - name: tap-jira
    namespace: tap-jira
    pip_url: -e extract/tap-jira
    executable: tap-jira
    capabilities:
    - state
    - catalog
    - discover
    - about
    - stream-maps
    config:
      auth:
        flow: password
        username: <radacted>
      domain: <radacted>

  - name: tap-mysql
    variant: transferwise
    pip_url:
      git+<https://github.com/transferwise/pipelinewise-tap-mysql.git@15a5be06273f85852d95b94f4e551aada61be05e>
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
  - name: target-bigquery
    variant: z3z1ma
    pip_url:
      git+<https://github.com/z3z1ma/target-bigquery.git@2f770e9c2f1735f3313cf08e3cd6793ceafa1183>
    config:
      location: $DATA_LOCATION
      project: $GOOGLE_CLOUD_PROJECT
      batch_size: 100000
      method: batch_job
      denormalized: true

Oh you need to put the logging config in a

logging.yaml

at the root of your repo. We do have an issue to allow users to embed it in `meltano.yml`: https://github.com/meltano/meltano/issues/7263.