Hello 🙂 I'm trying to set up tap-clickup to extract ClickUp tasks with their custom fields. The custom fields come as an array of objects containing a

value

from the API. The issue I'm facing is that the tap outputs these

value

properties exclusively as booleans, when in reality they can be numbers, strings, objects etc. .

{
   "type": "RECORD",
   "stream": "task",
   "record": {
      "id": "86byf3p5d",
      [...]
      "custom_fields": [
         {
            "id": "01f2ec13-9830-4786-b117-a3750961cba5",
            "value": true
            [...]

Getting the URL that the tap calls according to the log

<https://api.clickup.com/api/v2/team/$TEAM_ID/task?page=70&archived=false&include_closed=true&subtasks=true&order_by=updated&reverse=true>

with curl returns the proper values:

{
   "tasks": [
      {
         "id": "86byf3p5d",
         [...]
         "custom_fields": [
            {
               "id": "01f2ec13-9830-4786-b117-a3750961cba5",
               "value": "7800",
               [...]

The

task.json

schema of the tap identifies the types properly:

"value": {"type": ["integer", "string", "boolean", "null", "array", "object", "number"]}

Does anyone have any tips on how to troubleshoot this and find out why and where these values are being converted to booleans?

Definitely a bug in the tap, I swear someone mentioned this though in an issue for the tap. Let me see... https://github.com/AutoIDM/tap-clickup/pull/144/files Yeah someone else hit this, we should go find where in the SDK there's an issue. Maybe it's been fixed now and we just need to bump the SDK? Could you try bumping the SDK to the latest version and seeing if that works for you?

Could be https://github.com/meltano/sdk/issues/2553

@visch Thank you 🙏 , indeed it seems to be the same problem. Removing

boolean

from the types gets me the right values with actual booleans as string (Interestingly "number" and "currency" fields also become strings, but "formula" fields come out as numbers). I'll try to find out how to update the SDK (this is the first time i'm digging into the code of a tap) and see what happens. I tried it with singer-sdk 0.40 and it doesn't change anything, have to wait until #2553 is resolved i guess.

@claude_keller We could just use the same patch

def patched_is_boolean_type(property_schema: dict) -> bool | None:
    """Return true if the JSON Schema type is a boolean or None if detection fails.

    Without this patch, is_boolean_type() will return true for schemas that contain
    non-boolean types, which causes values to be coerced to booleans.

    For example, without this patch, a field with a value of `"abc"` and a jsonschema
    type of `["boolean", "string"]` would cause this function to return `True`. Then the
    SDK's _conform_primitive_property() would coerce `"abc"` to boolean, resulting in
    that field's value being `True`.

    See: <https://github.com/MeltanoLabs/tap-universal-file/issues/59>
    """
    if "anyOf" not in property_schema and "type" not in property_schema:
        return None  # Could not detect data type
    for property_type in property_schema.get("anyOf", [property_schema.get("type")]):
        schema_type = (
            property_type.get("type", [])
            if isinstance(property_type, dict)
            else property_type
        )
        if schema_type == "boolean" or (
            "boolean" in schema_type
            and (
                len(schema_type) == 1
                or ("null" in schema_type and len(schema_type) == 2)  # noqa: PLR2004
            )
        ):
            return True
    return False


singer_sdk.helpers._typing.is_boolean_type = patched_is_boolean_type  # noqa: SLF001

@visch Yes i did just that 👍 Here a PR

ty https://github.com/AutoIDM/tap-clickup/releases/tag/v0.0.23