Hey Everyone I m trying to debug my target in Pycharm but I Meltano #singer-target-development

Hey Everyone, I'm trying to debug my target in Pyc...

tim_frazer

12/07/2022, 5:12 PM

Hey Everyone, I'm trying to debug my target in Pycharm but I'm having a hell of a time using the debugger to step into the code - has anyone else done Target development in Pycharm?

tim_frazer

12/07/2022, 5:13 PM

doing a target-hubspot with a tap-bigquery - At the moment I'm cheating and using this select statement to generate some fake data - but alas I can't figure out how to use a debugger while using meltano to execute this stuff

Copy code

version: 1
project_id: 242e2eb5-8ffa-4e48-86f2-6f838822efbe
plugins:
  extractors:
  - name: tap-bigquery
    variant: anelendata
    config:
      streams:
      - name: hubspot-contact
        table: '( select ''<mailto:thrive.null_id@testmail.com|thrive.null_id@testmail.com>'' as email, ''tim_'' as first_name,
          ''frazer'' as last_name, ''1234'' as user_id, ''thrive'' as company, ''123''
          as company_id, ''1001'' as hubspot_id, date_sub(current_timestamp(), interval
          1 day) as updated_at, date_sub(current_timestamp(), interval 1 day) as created_at) '
        columns:
        - email
        - first_name
        - last_name
        - user_id
        - company
        - company_id
        - created_at
        - updated_at
        - hubspot_id
        datetime_key: updated_at
      start_datetime: '2022-12-01T00:00:00Z'

tim_frazer

12/07/2022, 5:14 PM

been a few months since I've touched this - so if anyone knows of an article or a place to get a this kind of dev ENV setup I would be super thankful

visch

12/07/2022, 5:33 PM

I'm not certain with PyCharm. The hard part with using PDB is PDB by default uses stdin which works terribly with targets because they use stdin to get data from stdout. https://github.com/meltano/sdk/issues/911 I wrote steps for how I do it with pdb. I pull out telnet 😮

01/12/2023, 9:32 AM

@visch wanted to let you know how much I appreciated these thorough steps! It is a serious velocity increase doing this vs trying to print everything to the console

visch

01/12/2023, 1:18 PM

@hg Awesome! I bet we could figure out a nicer way to hook into the debugger, but for something quick and to save a lot of time printing I agree! 😄

visch

01/12/2023, 1:18 PM

Really glad it helped, I've needed it a few times now and then and have came back to the steps

Open in Slack

Previous Next