Daniel Poot Uc
05/13/2024, 4:28 PMmappers:
- name: map-gpt-embeddings
variant: meltanolabs
pip_url: git+<https://github.com/meltanolabs/map-gpt-embeddings>
executable: map-gpt-embeddings
settings:
- name: openai_api_key
kind: string
sensitive: true
mappings:
- name: add-embeddings
config:
document_text_property: name
document_metadata_property: metadata
splitter_config:
chunk_size: 500
chunk_overlap: 200
I'm just having some trouble configuring the Mapper with document_metadata_property
. I'm still quite new to all of this; any help would be appreciated.Edgar Ramírez (Arch.dev)
05/13/2024, 4:33 PMmetadata
the right field name coming from tap-csv?Daniel Poot Uc
05/13/2024, 4:50 PMdocument_metadata_property
in my document? I am still not clear about this configuration.Edgar Ramírez (Arch.dev)
05/13/2024, 4:56 PMpage_content
and metadata
fields: https://github.com/MeltanoLabs/tap-beautifulsoup/blob/5c494852ea64cdcbdf82356e81efbff8b44bc9ce/tap_beautifulsoup/client.py#L91-L95
I guess the mapper then expects your tap to have a similar pair of fields, whose names you can configure with document_text_property
and document_metadata_property
respectively.