https://linen.dev logo
Join Slack
Powered by
Can anyone help with the configuration of tap-exac...
# troubleshooting
h
Can anyone help with the configuration of tap-exacttarget? I am able to connect to my source and list the catalog objects. I do not know how to configure the selection of the attributes. How do I configure this in the 
meltano.yml
file? current 
meltano.yml
Copy code
- name: tap-exacttarget
    variant: singer-io
    pip_url: tap-exacttarget
    capabilities:
    - catalog
    config:
      client_id: <>
      client_secret: <>
      tenant_subdomain: <>
      start_date: '2014-01-01T00:00:00Z'
      request_timeout: 300
      batch_size: 2500
    catalog: catalog.json
Message I get that objects are not selected: ```Found catalog in /Users/<>/GitHub/meltano-<>/catalog.json Legend: selected excluded automatic Enabled patterns: . Selected attributes: [excluded ] campaign.campaignCode [excluded ] campaign.color [excluded ] campaign.createdDate [excluded ] campaign.description [automatic] campaign.id [excluded ] campaign.modifiedDate [excluded ] campaign.name [excluded ] content_area.BackgroundColor [excluded ] content_area.BorderColor [excluded ] content_area.BorderWidth [excluded ] content_area.CategoryID [excluded ] content_area.Cellpadding [excluded ] content_area.Cellspacing [excluded ] content_area.Content [excluded ] content_area.CreatedDate [excluded ] content_area.CustomerKey [excluded ] content_area.FontFamily [excluded ] content_area.HasFontSize [automatic] content_area.ID [excluded ] content_area.IsBlank [excluded ] content_area.IsDynamicContent [excluded ] content_area.IsLocked [excluded ] content_area.IsSurvey [excluded ] content_area.Key [automatic] content_area.ModifiedDate [excluded ] content_area.Name [excluded ] content_area.ObjectID [excluded ] content_area.PartnerProperties [excluded ] content_area.Width [excluded ] email.CategoryID [excluded ] email.CharacterSet [excluded ] email.ClonedFromID [excluded ] email.ContentAreaIDs [excluded ] email.ContentCheckStatus [excluded ] email.CreatedDate [excluded ] email.CustomerKey [excluded ] email.EmailType [excluded ] email.HTMLBody [excluded ] email.HasDynamicSubjectLine [automatic] email.ID [excluded ] email.IsActive [excluded ] email.IsHTMLPaste [automatic] email.ModifiedDate [excluded ] email.Name [excluded ] email.ObjectID [excluded ] email.PartnerProperties [excluded ] email.PreHeader [excluded ] email.Status [excluded ] email.Subject [excluded ] email.SyncTextWithHTML [excluded ] email.TextBody [excluded ] email.__AdditionalEmailAttribute1 [excluded ] email.__AdditionalEmailAttribute2 [excluded ] email.__AdditionalEmailAttribute3 [excluded ] email.__AdditionalEmailAttribute4 [excluded ] email.__AdditionalEmailAttribute5 [excluded ] event.BatchID [excluded ] event.CorrelationID [automatic] event.EventDate [automatic] event.EventType [automatic] event.SendID [automatic] event.SubscriberKey [excluded ] event.URL [excluded ] folder.AllowChildren [excluded ] folder.ContentType [excluded ] folder.CreatedDate [excluded ] folder.CustomerKey [excluded ] folder.Description [automatic] folder.ID [automatic] folder.ModifiedDate [excluded ] folder.Name [excluded ] folder.ObjectID [excluded ] folder.ParentFolder [excluded ] folder.PartnerProperties [excluded ] folder.Type [excluded ] list.Category [excluded ] list.CreatedDate [excluded ] list.Description [automatic] list.ID [exc…
example of the 
catalog.json
file ```{ "streams": [ { "tap_stream_id": "campaign", "stream": "campaign", "key_properties": [ "id" ], "schema": { "type": "object", "properties": { "id": { "type": [ "null", "string" ] }, "createdDate": { "type": [ "null", "string" ] }, "modifiedDate": { "type": [ "null", "string" ] }, "name": { "type": [ "null", "string" ] }, "description": { "type": [ "null", "string" ] }, "campaignCode": { "type": [ "null", "string" ] }, "color": { "type": [ "null", "string" ] } } }, "metadata": [ { "breadcrumb": [], "metadata": { "table-key-properties": [ "id" ], "selected": true, "forced-replication-method": "FULL_TABLE", "inclusion": "available" } }, { "breadcrumb": [ "properties", "id" ], "metadata": { "inclusion": "automatic" } }, { "breadcrumb": [ "properties", "createdDate" ], "metadata": { "inclusion": "available" } }, { "breadcrumb": [ "properties", "modifiedDate" ], "metadata": { "inclusion": "available" } }, { "breadcrumb": [ "properties", "name" ], "metadata": { "inclusion": "available" } }, { "breadcrumb": [ "properties", "description" ], "metadata": { "inclusion": "available" } }, { "breadcrumb": [ "properties", "campaignCode" ], "metadata": { "inclusion": "available" } }, { "breadcrumb": [ "properties", "color" ], "metadata": { "inclusion": "available" } …
t
checkout https://docs.meltano.com/getting-started/part3#select-more-source-data we have a command line helper for that or you can directly edit the 
select
key in your yaml file
m
yeah, what Taylor said
h
Thanks for the quick reply. i tried adding them with the 
select
CLI.
Copy code
- name: tap-exacttarget
    variant: singer-io
    pip_url: tap-exacttarget
    capabilities:
    - catalog
    config:
      client_id: <>
      client_secret: <>
      tenant_subdomain: <>
      start_date: '2014-01-01T00:00:00Z'
      request_timeout: 300
      batch_size: 2500
    catalog: catalog.json
    select:
    - campaign.*
    - content_area.*
    - email.*
    - event.*
    - folder.*
    - list.*
    - list_send.*
    - send.*
    - subscriber.*
    - list_subscriber.*
but am still getting 
no properties selected
Copy code
meltano invoke tap-exacttarget                   
2023-02-28T01:34:20.769919Z [info     ] Found catalog in /Users/<>/GitHub/meltano-<>/catalog.json
INFO No properties were selected
INFO Completed successfully, exiting.
t
You shouldn’t need the 
catalog
key if you have 
select
.
h
Thank you, still the same issue. But when I run 
meltano select --list --all
the properties are now marked as selected.
Copy code
- name: tap-exacttarget
    variant: singer-io
    pip_url: tap-exacttarget
    config:
      client_id: <>
      client_secret: <>
      tenant_subdomain: <>
      start_date: '2014-01-01T00:00:00Z'
      request_timeout: 300
      batch_size: 2500
    select:
    - campaign.*
    - content_area.*
    - email.*
    - event.*
    - folder.*
    - list.*
    - list_send.*
    - send.*
    - subscriber.*
    - list_subscriber.*
Copy code
meltano-elt-non-fedramp % meltano invoke tap-exacttarget
INFO No properties were selected
INFO Completed successfully, exiting.
Copy code
[selected ] subscriber.Addresses
        [selected ] subscriber.Attributes
        [selected ] subscriber.CreatedDate
        [selected ] subscriber.CustomerKey
        [selected ] subscriber.EmailAddress
        [selected ] subscriber.EmailTypePreference
        [automatic] subscriber.ID
        [selected ] subscriber.ListIDs
        [selected ] subscriber.Locale
        [automatic] subscriber.ModifiedDate
        [selected ] subscriber.ObjectID
        [selected ] subscriber.PartnerKey
        [selected ] subscriber.PartnerProperties
        [selected ] subscriber.PartnerType
        [selected ] subscriber.PrimaryEmailAddress
        [selected ] subscriber.PrimarySMSAddress
        [selected ] subscriber.PrimarySMSPublicationStatus
        [selected ] subscriber.Status
        [selected ] subscriber.SubscriberKey
        [selected ] subscriber.SubscriberTypeDefinition
        [selected ] subscriber.UnsubscribedDate
Copy code
meltano config tap-exacttarget test              

Plugin configuration is invalid
No RECORD message received
t
Do you still have the 
capabilities
key in your yaml file? It looks like this target takes the old 
properties
config which we’ve defined in the metadata on the Hub - but it’s possible adding that to your file overwrote it
h
No, I removed the capabilities key
Copy code
- name: tap-exacttarget
    variant: singer-io
    pip_url: tap-exacttarget
    config:
      client_id: <>
      client_secret: <>
      tenant_subdomain: <>
      start_date: '2014-01-01T00:00:00Z'
      request_timeout: 300
      batch_size: 2500
    select:
    - campaign.*
    - content_area.*
    - email.*
    - event.*
    - folder.*
    - list.*
    - list_send.*
    - send.*
    - subscriber.*
    - list_subscriber.*
thats the setup I have now
this is from the log file, if it helps at all
Copy code
"TAP_EXACTTARGET__LOAD_SCHEMA": "tap_exacttarget",
    "MELTANO_EXTRACT__LOAD_SCHEMA": "tap_exacttarget",
    "TAP_EXACTTARGET__SELECT": "["campaign.*", "content_area.*", "email.*", "event.*", "folder.*", "list.*", "list_send.*", "send.*", "subscriber.*", "list_subscriber.*"]",
    "MELTANO_EXTRACT__SELECT": "["campaign.*", "content_area.*", "email.*", "event.*", "folder.*", "list.*", "list_send.*", "send.*", "subscriber.*", "list_subscriber.*"]",
    "TAP_EXACTTARGET__METADATA": "{}",
    "MELTANO_EXTRACT__METADATA": "{}",
    "TAP_EXACTTARGET__SCHEMA": "{}",
    "MELTANO_EXTRACT__SCHEMA": "{}",
    "TAP_EXACTTARGET__SELECT_FILTER": "[]",
    "MELTANO_EXTRACT__SELECT_FILTER": "[]",
t
do you have a folder for /plugins/extractors/tap-exacttarget--singer-io.lock?
h
Yes, I do sorry
Copy code
{
  "plugin_type": "extractors",
  "name": "tap-exacttarget",
  "namespace": "tap_exacttarget",
  "variant": "singer-io",
  "label": "SalesForce Marketing Cloud",
  "docs": "<https://hub.meltano.com/extractors/tap-exacttarget>",
  "repo": "<https://github.com/singer-io/tap-exacttarget>",
  "pip_url": "tap-exacttarget",
  "capabilities": [
    "catalog",
    "discover"
  ]
}
t
can you run 
meltano lock tap-extracttarget --update
and see if that changes it? that 
catalog
in there is probably causing the problem.
you could also delete the file and just run 
meltano lock tap-extracttarget
that will pull the latest definition from MeltanoHub and should have the correct capabilities defined.
p
I think theres a typo in that command in case you were copy and pasting - it should be 
meltano lock tap-exacttarget
t
😬 my bad
p
haha I literally typed the same thing too - my fingers want to write extractor. I also struggle with date and always write data
@helder_dasilva any luck? I installed that tap locally but I dont have credentials and it looks like you need them to run a select
h
im running it now. but seems like its getting further than before
table have been created. waiting to see data flow through 🙏
Open in Slack
PreviousNext
Powered by