https://linen.dev logo
Join Slack
Powered by
Hi everyone, our tap-google-analytics to redshift-...
# troubleshooting
j
Hi everyone, our tap-google-analytics to redshift-pipelinewise elt had been running perfectly. But during some manual backfill, we had to create new tables (same columns, variable types, and names). Now I run the elt, I see this:
Copy code
Google-Analytics     | INFO Mapping: vegalash_traffic_sources to None
Google-Analytics     | INFO Mapping: transaction_revenue to ['transaction_revenue']
Google-Analytics     | INFO Mapping: user_sessions to ['user_sessions']
Google-Analytics     | INFO Mapping: device_metrics to None
Google-Analytics     | INFO Mapping: country_metrics to None
Google-Analytics     | INFO Mapping: landing_page_transaction_id_to_source_medium to None
Google-Analytics     | INFO Mapping: page_metrics to None
Google-Analytics     | INFO Mapping: source_medium_transactions_sessions to None
Google-Analytics     | INFO Mapping: revenue_report to None
Google-Analytics     | INFO Mapping: shopping_stage to None
Google-Analytics     | INFO Mapping: transaction_id_to_source_medium to ['transaction_id_to_source_medium']
Google-Analytics     | ERROR Exception writing records
Followed by this error:
Copy code
Google-Analytics   | target_postgres.exceptions.PostgresError: ('Exception writing records', TypeError("'NoneType' object is not subscriptable"))
Any idea why it can't map some of the reports?
Here's the full error: ```Google-Analytics | INFO write_batch: Schema after nullability: {'type': ['object', 'null'], 'properties': {'ga_date': {'type': ['string', 'null']}, 'ga_medium': {'type': ['string', 'null']}, 'ga_source': {'type': ['string', 'null']}, 'ga_bounceRate': {'type': ['number', 'null']}, 'ga_bounces': {'type': ['integer', 'null']}, 'ga_newUsers': {'type': ['integer', 'null']}, 'ga_pageviews': {'type': ['integer', 'null']}, 'ga_percentNewSessions': {'type': ['number', 'null']}, 'ga_sessionDuration': {'type': ['number', 'null']}, 'ga_sessions': {'type': ['integer', 'null']}, 'ga_transactions': {'type': ['integer', 'null']}, 'ga_users': {'type': ['integer', 'null']}, 'report_start_date': {'type': ['string', 'null']}, 'report_end_date': {'type': ['string', 'null']}, '_sdc_received_at': {'type': ['null', 'string'], 'format': 'date-time'}, '_sdc_sequence': {'type': ['null', 'integer']}, '_sdc_table_version': {'type': ['null', 'integer']}, '_sdc_batched_at': {'type': ['null', 'string'], 'format': 'date-time'}}} Google-Analytics | INFO Mapping: vegalash_traffic_sources to None Google-Analytics | INFO Mapping: transaction_revenue to ['transaction_revenue'] Google-Analytics | INFO Mapping: user_sessions to ['user_sessions'] Google-Analytics | INFO Mapping: device_metrics to None Google-Analytics | INFO Mapping: country_metrics to None Google-Analytics | INFO Mapping: landing_page_transaction_id_to_source_medium to None Google-Analytics | INFO Mapping: page_metrics to None Google-Analytics | INFO Mapping: source_medium_transactions_sessions to None Google-Analytics | INFO Mapping: revenue_report to None Google-Analytics | INFO Mapping: shopping_stage to None Google-Analytics | INFO Mapping: transaction_id_to_source_medium to ['transaction_id_to_source_medium'] Google-Analytics | ERROR Exception writing records Google-Analytics | Traceback (most recent call last): Google-Analytics | File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/loaders/Google-Analytics/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 239, in write_batch Google-Analytics | current_table_schema = self.get_table_schema(cur, root_table_name) Google-Analytics | File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/loaders/Google-Analytics/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 775, in get_table_schema Google-Analytics | return self.__get_table_schema(cur, name) Google-Analytics | File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/loaders/Google-Analytics/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 789, in __get_table_schema Google-Analytics | metadata = self._get_table_metadata(cur, name) Google-Analytics | File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/loaders/Google-Analytics/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 705, in _get_table_metadata Google-Analytics | comment = cur.fetchone()[0] Google-Analytics | TypeError: 'NoneType' object is not subscriptable Google-Analytics | CRITICAL ('Exception writing records', TypeError("'NoneType' object is not subscriptable")) Google-Analytics | Traceback (most recent call last): Google-Analytics | File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/loaders/Google-Analytics/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 239, in write_batch Google-Analytics | current_table_schema = self.get_table_schema(cur, root_table_name) Google-Analytics | File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/loaders/Google-Analytics/venv/lib/python3.8/site-packages/target_postgres/postgres.py", line 775, in get_table_schema Google-Analytics | return self.__get_table_schema(cur, name) Google-Analytics | File "/home/ubuntu/rm-projects/meltano-pro…
a
I'm not experienced with the google analytics side but the postgres target seems to be what is throwing the error.
Is your target plugin aliased "Google-Analytics" by chance? I'm confused why this path calls "Google Analytics" a loader and then has inside it the 'target_postgres' library: 
/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/loaders/Google-Analytics/venv/lib/python3.8/site-packages/target_postgres/postgres.py
j
Sorry for the confusion! I should've clarified that the target for GA is aliased as Google-Analytics. And I agree it’s something with the loader, but I can't figure out why it's mapping out several of the reports as None, when they've run before and the tables still exist in redshift
Open in Slack
PreviousNext
Powered by