Meltano

How can I configure the `select` in meltano.yml to get nested properties?

I have a response body structured like
```{
  "id": 1234,
  "attributes": {
    "name": "name_1234",
    "unwanted": 999
  }
}```

If in the select list I specify `entity.id` and `entity.attributes`, everything is returned

However, if I specify `entity.attributes.name`, I get nothing

I found the `records_jsonpath` and digging in there

I ended up doing specific extraction logic in `parse_response`. Is that the best practice?

Messed around with this a bit with an sdk tap, and also couldn't get sub objects to select properly even though
`meltano select --list tap-name` showed them available

Implementing `parse_response` gets me the behavior I want... when I run `poetry run ...` but `meltano invoke ...` gives me different behavior

...yes, I had to delete the .meltano directory and reinstall the tap

Agreed <@U06CCB0EUBC> I saw the same behavior. The properties would show up in the list, but couldn't select them

Flattening using `parse_response` gave me exactly what I was after

Yeah it seems like the select syntax should handle this otherwise everyone will have to override parse_response

<https://gitlab.com/meltano/sdk/-/issues/285>

<@U06C8NEFG2Z> <@U06CCB0EUBC> I went digging. I don't think this is bug in the SDK, and you should definitely not need to override `parse_response` to deselect fields. Meltano's walk through the `select` may be improved so simplify things, but something like this  should work:

```# meltano.yml
plugins:
  extractors:
    - name: tap-example
      select:
      - entity.id
      - entity.attributes             # select parent property
      - entity.attributes.*           # select all parent property fields
      - entity.attributes.name        # select wanted nested property
      - "!entity.attributes.unwanted" # deselect unwanted nested property```
Also commented in the issue :simple_smile:

makes sense, the higher level object metadata wasn't set to selected

I searched the catalog quickly and found selected: true, didn't think the parent stream would need to be selected but that makes perfect sense. If you say selected:false on the parent stream you'd expect to not get the higher level property

Is confusing from the meltano side (says the stream is selected as of course it is, but it doesn't get displayed)

Ah interesting. I'll give that a try. <@U06BX9QQ4A3> thanks a ton!