Okay, I'm having what I hope is a stupid problem (bc maybe it would be easy to solve??). I ran

meltano --log-level=debug run tap-csv target-duckdb

because I was having some whack a mole broken pipe errors and was fixing them one by one. Finally, I got that command to run with no errors! Hopefully, this log message below means the csv I wanted to load into duckdb actually worked.

time=2023-01-21 15:45:46 name=target_duckdb level=INFO message=Loading 23735 rows into 'main."game"' cmd_type=elb consumer=True name=target-duckdb producer=False stdio=stderr string_id=target-duckdb

But now, embarrassingly enough, I don't know how to go and check to try to query the game table that (it looks like) I successfully loaded into duckdb...when i open the duckdb.exe file and try to query main.game it doesn't know that it's there, so that makes me think I didn't actually load it correctly. This is what the loaders section of my .yml file looks like.

loaders:
  - name: target-duckdb
    variant: jwills
    pip_url: target-duckdb~=0.4
    config:
      filepath: ../../../duckdb/duckdb
      default_target_schema: main
      add_metadata_columns: true

I suspect the issue is with the filepath. My questions are: 1. Should my filepath be referencing some sort of .db file? I tried creating a hockey.db file within the same folder as the duckdb.exe , but that wasn't recognized as a valid duckdb file. 2. Did I not actually load the data correctly? Or do I need to do something else to be able to query it?

The file does not have to exist beforehand. The target will create the file.

filepath

is where the actual duckdb file will be written. It would probably be easiest to make the filepath local to your project. You can use an interpolated env var for this:

filepath: ${MELTANO_PROJECT_ROOT}/hockey.duckdb

The convention that seems to be common is to use

.duckdb

as the file ext. That would make it easier to differentiate. So thats what I put in the above config.

Yes use the duckDB cli, or you can use dbeaver if you want to use an IDE instead

Thanks guys, trying this out now.