Hello everyone, I’m having problems with the hubspot-tap. The problem that I have is that --properties is not deprecated (since feb 2019) And Meltano doesn’t support that parameter calling

meltano etl

although it works for

meltano invoke

What should I do ? 1. Fork the repository hubspot-tap 2. Is there a way to make it work with meltano Thanks for your help

I’m not sure how we’d want to handle a deprecated flag like that. @douwe_maan can you chime in?

@taylor technically, it’s just adding the --properties flag with the same file than for catalog. That’s it. Hubspot being a very populate tap, the ROI of implementing just a flag kind of worth it

I made https://gitlab.com/meltano/meltano/-/issues/2642 to address - feel free to chime in there too!

@juan_sebastian_suarez_valencia If you include

properties

in the plugin's

capabilities

instead of

catalog

, Meltano will know to use

--properties

instead of

--catalog

Is https://meltano.com/docs/contributor-guide.html#taps-targets-development the only documentation for this?

It's mentioned in the

meltano add --custom

prompt: https://gitlab.com/meltano/meltano/-/blob/e3dd1b3232dd081467685ea8a0f7a4829da21468/src/meltano/cli/utils.py#L169

I fixed the same issue with tap-hubspot by following this.

@pankaj_saini If you've already got it working, please consider contributing it to https://meltano.com/docs/contributor-guide.html#discoverable-plugins 😉

@douwe_maan Thanks for your input What I mentioned in my initial comment is that the tap works fine with the properties argument in invokation

@juan_sebastian_suarez_valencia Ah, if you've configured the tap to take

--properties

using the capability, you'll still want to use

--catalog

on

meltano elt

. Behind the scenes, it'll use

--properties

on the tap

Thank you very much @douwe_maan The ETL worked but now I have a new problem

😬

But that’s because of the tap. I saw that there’s an opened issue for that tap

Thanks!

I have seen that HubSpot API's output is massive and for each attribute(e.g. dealname) there are almost 5 sub-attributes. Writing this works okay on Redshift for me(I used transferwise-target-redshift), because the target-redshift doesn't do JSON normalisation and writes all sub attributes as a single JSON object in a column. So number of columns doesn't become so massive. But looks like JSON normalisation happens in big query target, that's why the number of columns are almost 5 times the number of attributes in a HubSpot object. There seems to be many other issues with tap-hubspot and I guess it needs more work before it is added to meltano's discoverable plugins. I am trying to make few changes to get this working well for me. If it looks good I will contribute this to meltano.

Thank you @pankaj_saini