GitHub
03/15/2024, 4:08 PMaccess_token
• store
• admin_url
• start_date
• is_plus_account
Now, implicit SDK settings are merged with developer-defined settings:
• access_token
• admin_url
• batch_config.encoding.compression
• batch_config.encoding.format
• batch_config.storage.prefix
• batch_config.storage.root
• flattening_enabled
• flattening_max_depth
• is_plus_account
• start_date
• store
• stream_map_config
• stream_maps
The problem here is that store, for example, is a (conditionally) required setting and, due to alphabetical sorting, is hidden amongst settings that are not required and/or not defined by the developer originally. This makes it hard to see what a user needs to provide to get the tap working at a minimum.
The quickest fix here might be to stop sorting settings alphabetically in the plugin definition .yml files and display them in the order that they are defined, in an effort to group by importance and behaviour. It would also be good to group optional settings by whether they were explicitly defined by the developer or inherited from the SDK (given the plugin was built with the SDK) - maybe there could be an Advanced or SDK-inherited subsection? In the long-term, perhaps there needs to be a better implementation of settings_group_validation, in order to show the required (unconditionally and conditionally) settings first, followed by the unconditionally optional settings.
meltano/hubGitHub
03/15/2024, 4:08 PM