Is there anyway to handle rate limits in `tap-face...
# plugins-general
j
Is there anyway to handle rate limits in
tap-facebook
? I am unable to run even part of an import because the rate limits are hit almost instantly.
Copy code
2022-01-24T10:24:29.189827Z [info     ]   Status:  400                 cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.190004Z [info     ]   Response:                    cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.190170Z [info     ]     {                          cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.190346Z [info     ]       "error": {               cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.190518Z [info     ]         "message": "(#17) User request limit reached", cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.190688Z [info     ]         "type": "OAuthException", cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.190887Z [info     ]         "is_transient": true,  cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.191074Z [info     ]         "code": 17,            cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.191284Z [info     ]         "error_subcode": 2446079, cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.191518Z [info     ]         "fbtrace_id": "AMK727wIfIfbMCRbro1UFbE" cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.191720Z [info     ]       }                        cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
2022-01-24T10:24:29.191924Z [info     ]     }                          cmd_type=extractor job_id=facebook_to_bigquery name=tap-facebook run_id=87059e1a-ed36-40cd-be25-f6a203c876ae stdio=stderr
Facebooks own SDK doesn't even try to look at rate-limits - let alone waiting if it is approaching the limit...