https://linen.dev logo
Join Slack
Powered by
Hello, I am using tap-facebook(meltano variant) a...
# troubleshooting
n
Hello, I am using tap-facebook(meltano variant) and trying to include a breakdown of "product_id". Unfortunately this also adds "product_id" to the field list which will cause a failure on a POST request(which is what is used in the tap. Any solutions? Maybe I am doing something wrong. I have tested CURL statements to confirm this behavior. GET works with "product_id" as breakdown and in field. POST works with breakdown = "product_id" and not in field list. Log below:
Copy code
2025-01-03T19:18:28.866480Z [info     ]         "message": "(#100) product_id is not valid for fields param. please check <https://developers.facebook.com/docs/marketing-api/reference/ads-insights/> for all valid values", cmd_type=extractor name=tap-facebook run_id=ef447137-2f40-4a42-ba37-c2cf4fddba9f state_id=2025-01-03T191052--tap-facebook--target-gcs stdio=stderr
2025-01-03T19:18:28.866603Z [info     ]         "type": "OAuthException", cmd_type=extractor name=tap-facebook run_id=ef447137-2f40-4a42-ba37-c2cf4fddba9f state_id=2025-01-03T191052--tap-facebook--target-gcs stdio=stderr
2025-01-03T19:18:28.866731Z [info     ]         "code": 100,           cmd_type=extractor name=tap-facebook run_id=ef447137-2f40-4a42-ba37-c2cf4fddba9f state_id=2025-01-03T191052--tap-facebook--target-gcs stdio=stderr
2025-01-03T19:18:28.866853Z [info     ]         "fbtrace_id": "AAVY2ctUcCgRvErNc479e_a" cmd_type=extractor name=tap-facebook run_id=ef447137-2f40-4a42-ba37-c2cf4fddba9f state_id=2025-01-03T191052--tap-facebook--target-gcs stdio=stderr
2025-01-03T19:18:28.866972Z [info     ]       }                        cmd_type=extractor name=tap-facebook run_id=ef447137-2f40-4a42-ba37-c2cf4fddba9f state_id=2025-01-03T191052--tap-facebook--target-gcs stdio=stderr
2025-01-03T19:18:28.867104Z [info     ]     }
.yml
Copy code
extractors:
- name: tap-facebook
  config:
    api_version: v20.0
    account_ids: "${ACCOUNT_IDS}"
    start_date: "${START_DATA_DATE}"
    end_date: "${END_DATA_DATE}"
    access_token: "${FACEBOOK_ACCESS_TOKEN}"
    insight_reports_list:
      - name: "defaultTest"
        level: "ad"
        breakdowns: ["product_id"]
        action_attribution_windows:
          - "1d_click"
          - "7d_click"
Any help would be great.
e
Is there anything in particular about 
product_id
that causes it to break the request when present in 
fields
, or is this some more general behavior? I'm trying to think of a fix of that here: https://github.com/MeltanoLabs/tap-facebook/blob/59a5d96465165aee181d654796918c6608e2f903/tap_facebook/streams/ad_insights.py#L220-L222.
n
Well nothing in particular besides that it happens. I am have been toying with this piece of code. product_id just isn't allowed on a POST. GET yes POST no
Copy code
def schema(self) -> dict:
    properties: list[th.Property] = []
    columns = list(AdsInsights.Field.__dict__)[1:]
    for field in columns:
        if field in EXCLUDED_FIELDS:
            continue
        if data_type := self._get_datatype(field):
            properties.append(th.Property(field, data_type))

    properties.extend(
        [
            th.Property(breakdown, th.StringType())
            for breakdown in self._report_definition["breakdowns"]
        ],
    )

    return th.PropertiesList(*properties).to_dict()
doesn't seem to be working atm
my CURL test GET: WORKS GET https://graph.facebook.com/v21.0/act_##/insights?fields=clicks,ad_id,adset_id,campaign_id,account_id,impressions,cpc,cost_per_inline_link_click,cost_per_inline_post_engagement,cpm,ctr,inline_link_click_ctr,account_currency,account_name,action_values,actions,ad_name,adset_name,website_purchase_roas&amp;limit=10&amp;breakdowns=product_id&amp;level=ad Authorization: Bearer Content-Type: application/x-www-form-urlencoded #level = ad & #action_breakdowns = [] & #action_report_time = mixed & #breakdowns = ["product_id"] & #fields = ["account_currency","account_id","account_name","action_values","actions","ad_click_actions","ad_id","ad_impression_actions","ad_name","adset_id","adset_name","attribution_setting","auction_bid","auction_competitiveness","auction_max_competitor_bid","average_purchases_conversion_value","buying_type","campaign_id","campaign_name","canvas_avg_view_percent","canvas_avg_view_time","catalog_segment_actions","catalog_segment_value","catalog_segment_value_mobile_purchase_roas","catalog_segment_value_omni_purchase_roas","catalog_segment_value_website_purchase_roas","clicks","conversion_rate_ranking","conversion_values","conversions","converted_product_quantity","converted_product_value","cost_per_15_sec_video_view","cost_per_2_sec_continuous_video_view","cost_per_action_type","cost_per_ad_click","cost_per_conversion","cost_per_estimated_ad_recallers","cost_per_inline_link_click","cost_per_inline_post_engagement","cost_per_outbound_click","cost_per_thruplay","cost_per_unique_action_type","cost_per_unique_click","cost_per_unique_inline_link_click","cost_per_unique_outbound_click","cpc","cpm","cpp","created_time","ctr","date_start","date_stop","engagement_rate_ranking","estimated_ad_recall_rate","estimated_ad_recallers","frequency","full_view_impressions","full_view_reach","impressions","inline_link_click_ctr","inline_link_clicks","inline_post_engagement","instant_experience_clicks_to_open","instant_experience_clicks_to_start","instant_experience_outbound_clicks","marketing_messages_delivery_rate","marketing_messages_link_btn_click_rate","marketing_messages_media_view_rate","marketing_messages_phone_call_btn_click_rate","marketing_messages_quick_reply_btn_click_rate","marketing_messages_read_rate","marketing_messages_website_purchase_values","mobile_app_purchase_roas","objective","onsite_conversion_messaging_detected_purchase_deduped","optimization_goal","outbound_clicks","outbound_clicks_ctr","purchase_roas","qualifying_question_qualify_answer_rate","quality_ranking","reach","shops_assisted_purchases","social_spend","spend","unique_actions","unique_clicks","unique_ctr","unique_inline_link_click_ctr","unique_inline_link_clicks","unique_link_clicks_ctr","unique_outbound_clicks","unique_outbound_clicks_ctr","updated_time","video_15_sec_watched_actions","video_30_sec_watched_actions","video_avg_time_watched_actions","video_continuous_2_sec_watched_actions","video_p100_watched_actions","video_p25_watched_actions","video_p50_watched_actions","video_p75_watched_actions","product_id"] & #limit = 100 & #action_attribution_windows = ["1d_view","7d_click"] & #time_range = {"since":"2025-01-01","until":"2025-01-01"} POST: No product_id in fields list WORKS POST https://graph.facebook.com/v21.0/act_##/insights Authorization: Bearer Content-Type: application/x-www-form-urlencoded level = ad & action_breakdowns = [] & action_report_time = mixed & breakdowns = ["product_id"] & fields = ["account_currency","account_id","account_name","action_values","actions","ad_click_actions","ad_id","ad_impression_actions","ad_name","adset_id","adset_name","attribution_setting","auction_bid","auction_competitiveness","auction_max_competitor_bid","average_purchases_conversion_value","buying_type","campaign_id","campaign_name","canvas_avg_view_percent","canvas_avg_view_time","catalog_segment_actions","catalog_segment_value","catalog_segment_value_mobile_purchase_roas","catalog_segment_value_omni_purchase_roas","catalog_segment_value_website_purchase_roas","clicks","conversion_rate_ranking","conversion_values","conversions","converted_product_quantity","converted_product_value","cost_per_15_sec_video_view","cost_per_2_sec_continuous_video_view","cost_per_action_type","cost_per_ad_click","cost_per_conversion","cost_per_estimated_ad_recallers","cost_per_inline_link_click","cost_per_inline_post_engagement","cost_per_outbound_click","cost_per_thruplay","cost_per_unique_action_type","cost_per_unique_click","cost_per_unique_inline_link_click","cost_per_unique_outbound_click","cpc","cpm","cpp","created_time","ctr","date_start","date_stop","engagement_rate_ranking","estimated_ad_recall_rate","estimated_ad_recallers","frequency","full_view_impressions","full_view_reach","impressions","inline_link_click_ctr","inline_link_clicks","inline_post_engagement","instant_experience_clicks_to_open","instant_experience_clicks_to_start","instant_experience_outbound_clicks","marketing_messages_delivery_rate","marketing_messages_link_btn_click_rate","marketing_messages_media_view_rate","marketing_messages_phone_call_btn_click_rate","marketing_messages_quick_reply_btn_click_rate","marketing_messages_read_rate","marketing_messages_website_purchase_values","mobile_app_purchase_roas","objective","onsite_conversion_messaging_detected_purchase_deduped","optimization_goal","outbound_clicks","outbound_clicks_ctr","purchase_roas","qualifying_question_qualify_answer_rate","quality_ranking","reach","shops_assisted_purchases","social_spend","spend","unique_actions","unique_clicks","unique_ctr","unique_inline_link_click_ctr","unique_inline_link_clicks","unique_link_clicks_ctr","unique_outbound_clicks","unique_outbound_clicks_ctr","updated_time","video_15_sec_watched_actions","video_30_sec_watched_actions","video_avg_time_watched_actions","video_continuous_2_sec_watched_actions","video_p100_watched_actions","video_p25_watched_actions","video_p50_watched_actions","video_p75_watched_actions"] & limit = 100 & action_attribution_windows = ["1d_view","7d_click"] & time_range = {"since":"2025-01-01","until":"2025-01-01"} Retrieve report when ready: GET https://graph.facebook.com/v21.0/run_report_id/insights Authorization: Bearer
e
Can you create an issue in the project repo? The tap uses the official client library so I wonder if it has a way of change the HTTP method.
n
https://github.com/MeltanoLabs/tap-facebook/issues/285 I think a POST needs to be made due to the nature of how the report is delivered and processed. I will make a PR but two things: 1. Fields in general are not being respected but that's kind of a separate issue in general I think. 2. This piece of code seems to work.
Copy code
columns = self._get_selected_columns()
breakdowns = self._report_definition.get("breakdowns", [])
fields = [col for col in columns if col not in set(breakdowns)]

while report_start <= sync_end_date:
    params = {
        "level": self._report_definition["level"],
        "action_breakdowns": self._report_definition["action_breakdowns"],
        "action_report_time": self._report_definition["action_report_time"],
        "breakdowns": breakdowns,
        "fields": fields,
        "time_increment": time_increment,
        "limit": 100,
        "action_attribution_windows": [
            self._report_definition["action_attribution_windows_view"],
            self._report_definition["action_attribution_windows_click"],
        ],
        "time_range": {
            "since": report_start.to_date_string(),
            "until": report_end.to_date_string(),
        },
    }
3 Views
Open in Slack
PreviousNext
Powered by