Hi everyone! I am setting up tap-salesforce. I hav...
# troubleshooting
f
Hi everyone! I am setting up tap-salesforce. I have configured its configurations for username/password based authentication. But when i test tab settings, getting this error messge:
Copy code
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
Exception: Error syncing Account: Python int too large to convert to C long
Here is meltano.yaml file:
Copy code
version: 1
default_environment: dev
project_id: b6ec1556-37eb-4f14-9dfb-6505ac1eacd5
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-salesforce
    variant: meltanolabs
    pip_url: git+<https://github.com/meltanolabs/tap-salesforce.git@v1.5.1>
    config:
      api_type: BULK
      select_fields_by_default: true
      username: <mailto:feroze@force.com|feroze@force.com>
      start_date: '2024-07-10T00:00:00Z'
e
That's probably coming from a library using C extensions, but it's hard to tell which from the error message
f
Python error fixed but now getting this format error:
Copy code
2024-07-15T07:14:59.303768Z [debug    ] INFO Used 496 of 15000 daily REST API quota
2024-07-15T07:14:59.304729Z [debug    ] WARNING Invalid format string, (2024-07-15T07:11:57.000+0000)
2024-07-15T07:14:59.305766Z [debug    ] WARNING Invalid format string, (2024-07-15T07:11:57.000+0000)
2024-07-15T07:14:59.305766Z [debug    ] WARNING Invalid format string, (2024-07-15T07:11:57.000+0000)
2024-07-15T07:14:59.306727Z [debug    ] WARNING Invalid format string, (2024-07-15T07:12:02.000+0000)
2024-07-15T07:14:59.306727Z [debug    ] WARNING Invalid format string, (2024-07-15T07:12:02.000+0000)
2024-07-15T07:14:59.307763Z [debug    ] INFO METRIC: {"type": "counter", "metric": "record_count", "value": 1, "tags": {"endpoint": "Account"}}
2024-07-15T07:14:59.307763Z [debug    ] CRITICAL Error syncing Account: Invalid format string
2024-07-15T07:14:59.307763Z [debug    ] Traceback (most recent call last):
2024-07-15T07:14:59.308763Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\sync.py", line 95, in sync_stream
2024-07-15T07:14:59.309729Z [debug    ] sync_records(sf, catalog_entry, state, counter, state_msg_threshold)
2024-07-15T07:14:59.309729Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\sync.py", line 125, in sync_records
2024-07-15T07:14:59.310728Z [debug    ] singer.write_message(
2024-07-15T07:14:59.310728Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\singer\messages.py", line 217, in write_message
2024-07-15T07:14:59.311732Z [debug    ] sys.stdout.write(format_message(message) + '\n')
2024-07-15T07:14:59.311732Z [debug    ] ^^^^^^^^^^^^^^^^^^^^^^^
2024-07-15T07:14:59.311732Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\singer\messages.py", line 213, in format_message
2024-07-15T07:14:59.312727Z [debug    ] return json.dumps(message.asdict(), use_decimal=True)
2024-07-15T07:14:59.312727Z [debug    ] ^^^^^^^^^^^^^^^^
2024-07-15T07:14:59.313732Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\singer\messages.py", line 63, in asdict
2024-07-15T07:14:59.313732Z [debug    ] result['time_extracted'] = u.strftime(as_utc)
2024-07-15T07:14:59.313732Z [debug    ] ^^^^^^^^^^^^^^^^^^
2024-07-15T07:14:59.314733Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\singer\utils.py", line 43, in strftime
2024-07-15T07:14:59.314733Z [debug    ] dt_str = dtime.strftime(format_str)
2024-07-15T07:14:59.314733Z [debug    ] ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-15T07:14:59.315728Z [debug    ] ValueError: Invalid format string
2024-07-15T07:14:59.315728Z [debug    ] The above exception was the direct cause of the following exception:
2024-07-15T07:14:59.315728Z [debug    ] Traceback (most recent call last):
2024-07-15T07:14:59.316728Z [debug    ] File "<frozen runpy>", line 198, in _run_module_as_main
2024-07-15T07:14:59.316728Z [debug    ] File "<frozen runpy>", line 88, in _run_code
2024-07-15T07:14:59.316728Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Scripts\tap-salesforce.exe\__main__.py", line 7, in <module>
2024-07-15T07:14:59.317728Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\__init__.py", line 557, in main
2024-07-15T07:14:59.323726Z [debug    ] raise e
2024-07-15T07:14:59.324724Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\__init__.py", line 548, in main
2024-07-15T07:14:59.324724Z [debug    ] main_impl()
2024-07-15T07:14:59.325726Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\__init__.py", line 531, in main_impl
2024-07-15T07:14:59.325726Z [debug    ] do_sync(sf, catalog, state)
2024-07-15T07:14:59.326729Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\__init__.py", line 501, in do_sync
2024-07-15T07:14:59.327727Z [debug    ] loop.run_until_complete(tasks)
2024-07-15T07:14:59.327727Z [debug    ] File "C:\Users\fahmad\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
2024-07-15T07:14:59.327727Z [debug    ] return future.result()
2024-07-15T07:14:59.328724Z [debug    ] ^^^^^^^^^^^^^^^
2024-07-15T07:14:59.328724Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\__init__.py", line 482, in sync_catalog_entry
2024-07-15T07:14:59.328724Z [debug    ] await loop.run_in_executor(None, sync_stream, sf, catalog_entry, state, state_msg_threshold)
2024-07-15T07:14:59.329725Z [debug    ] File "C:\Users\fahmad\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\thread.py", line 58, in run
2024-07-15T07:14:59.329725Z [debug    ] result = self.fn(*self.args, **self.kwargs)
2024-07-15T07:14:59.329725Z [debug    ] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-07-15T07:14:59.330726Z [debug    ] File "D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\tap_salesforce\sync.py", line 102, in sync_stream
2024-07-15T07:14:59.330726Z [debug    ] raise Exception("Error syncing {}: {}".format(
2024-07-15T07:14:59.330726Z [debug    ] Exception: Error syncing Account: Invalid format string
2024-07-15T07:14:59.408725Z [debug    ] Process return code: 1
2024-07-15T07:14:59.409762Z [debug    ] Deleted configuration at D:\meltano\salesforce\.meltano\run\tap-salesforce\tap.ca4f15f0-b376-409f-a819-d745420279d8.config.json
C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\structlog\stdlib.py:1098: UserWarning: Remove `format_exc_info` from your processor chain if you want pretty exceptions.
  ed = p(logger, meth_name, cast(EventDict, ed))
2024-07-15T07:14:59.411725Z [debug    ] Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
Exception: Error syncing Account: Invalid format string
Traceback (most recent call last):
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\meltano\cli\__init__.py", line 103, in _run_cli
    cli(obj={"project": None})
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\meltano\cli\cli.py", line 46, in main
    return super().main(*args, windows_expand_args=False, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\meltano\cli\utils.py", line 643, in invoke
    super().invoke(ctx)
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\meltano\cli\utils.py", line 643, in invoke
    super().invoke(ctx)
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\meltano\cli\utils.py", line 692, in invoke
    super().invoke(ctx)
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\meltano\cli\config.py", line 420, in test
    raise CliError("\n".join(("Plugin configuration is invalid", detail)))
meltano.cli.utils.CliError: Plugin configuration is invalid
Exception: Error syncing Account: Invalid format string

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\fahmad\pipx\venvs\meltano\Lib\site-packages\meltano\cli\__init__.py", line 113, in _run_cli
    raise CliError(f"{troubleshooting_message}\n{err}") from err  # noqa: EM102
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
meltano.cli.utils.CliError: Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
Exception: Error syncing Account: Invalid format string
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
Exception: Error syncing Account: Invalid format string
e
@Feroze Ahmad what version and variant of tap-salesforce are you using? I think it's using a somewhat outdate version of the singer-python library.
f
I am using this tap-salesforce
Copy code
<https://hub.meltano.com/extractors/tap-salesforce/?_gl=1%2a1ulgdc0%2a_gcl_au%2aODMyOTg4NDg4LjE3MTY3OTMwNzk>.
I am stuck here and not getting any clue how to get rid of this issue.
e
Can you update your
pip_url
to point to the v1.6.1 tag and try again?
f
I have already tried this but no success. I don't know if it is due to using meltano on windows. No idea. ☹️
e
I do see an error coming from
loop.run_until_complete
but that's probably the root cause. Rather this:
Copy code
>>> dt_str = dtime.strftime(format_str)
ValueError: Invalid format string
It'd be good to know the values of
dtime
and
format_str
at that point. Maybe you can edit the file in
D:\meltano\salesforce\.meltano\extractors\tap-salesforce\venv\Lib\site-packages\singer\utils.py
to log the values before the function call?