jeff_katz
03/21/2022, 7:41 PMtap-mixpanel
with postgres...and while the elt logs says that the command is successful, no new schemas or tables are created and no data is ultimately loaded...
This is the log I get when I run meltano elt tap-mixpanel target-postgres
022-03-21T19:32:20.203082Z [info ] Environment 'dev' is active
2022-03-21T19:32:21.052805Z [info ] Running extract & load... job_id=2022-03-21T193220--tap-mixpanel--target-postgres name=meltano run_id=45dc01f1-b81e-4bd4-98a2-3156626487bb
2022-03-21T19:32:21.867689Z [info ] Extract & load complete! job_id=2022-03-21T193220--tap-mixpanel--target-postgres name=meltano run_id=45dc01f1-b81e-4bd4-98a2-3156626487bb
2022-03-21T19:32:21.868203Z [info ] Transformation skipped. job_id=2022-03-21T193220--tap-mixpanel--target-postgres name=meltano run_id=45dc01f1-b81e-4bd4-98a2-3156626487bb
And this is my project.yml
file
version: 1
project_id: ea1d85fb-6250-41b9-8fca-effccbdc5278
plugins:
extractors:
- name: tap-mixpanel
namespace: tap_mixpanel
pip_url: git+<https://github.com/hotgluexyz/tap-mixpanel.git>
config:
api_secret: 0de51f8cb611de93c70990e9f06e37e5
date_window_size: 30
attribution_window: 30
project_timezone: US/EST
select_properties_by_default: true
denest_properties: true
start_date: '2022-01-01T00:00:00Z'
user_agent: 'tap-mixpanel <mailto:jeff@jigsawlabs.io|jeff@jigsawlabs.io>'
select:
- export.*
- engage.*
- funnels.*
loaders:
- name: target-postgres
config:
postgres_username: jeffreykatz
postgres_database: mixpanel
user: jeffreykatz
password: password
dbname: mixpanel
environments:
- name: dev
- name: staging
- name: prod
thomas_briggs
03/21/2022, 8:09 PMmeltano select
to, well, select the streams you want to replicate?jeff_katz
03/21/2022, 8:45 PMthomas_briggs
03/21/2022, 9:03 PMmeltano select tap-mixpanel --list --all
and get the stream names from there? The format of those names has tripped me up more than once.jeff_katz
03/21/2022, 9:53 PMmeltano select tap-mixpanel --list --all
Cannot list the selected attributes: Could not find catalog. Verify that the tap supports discovery mode and advertises the `discover` capability as well as either `catalog` or `properties`
thomas_briggs
03/21/2022, 10:00 PMmeltano invoke tap-mixpanel --discover
, maybe?jeff_katz
03/22/2022, 3:38 AM{
"tap_stream_id": "export",
"key_properties": [],
"schema": {
"properties": {
"event": {
"type": [
"null",
"string"
]
},
"distinct_id": {
"type": [
"null",
"string"
]
},
"time": {
"format": "date-time",
"type": [
"null",
"string"
]
},
"labels": {
"anyOf": [
{
"type": "array",
"items": {
"type": "string"
}
},
{
"type": "null"
}
]
},
"sampling_factor": {
"type": [
"null",
"integer"
]
},
"dataset": {
"type": [
"null",
"string"
]
},
"mp_reserved_event_name": {
"type": [
"null",
"string"
]
},
"mp_reserved_browser_version": {
"type": [
"null",
"string"
]
},
"mp_reserved_current_url": {
"type": [
"null",
"string"
]
},
"mp_reserved_device_id": {
"type": [
"null",
"string"
]
},
jeff_katz
03/22/2022, 3:39 AM