How can I pass metadata configuration to an SDK-ba...
# singer-tap-development
m
How can I pass metadata configuration to an SDK-based Tap in a test? I’d like to test both incremental and log-based replication methods.
Copy code
import datetime
from singer_sdk.testing import get_tap_test_class
from tap_mongodb.tap import TapMongoDB

SAMPLE_CONFIG = {
    "start_date": datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d")
    # TODO: Initialize minimal tap config
}

# Run standard built-in tap tests from the SDK:
TestTapMongoDB = get_tap_test_class(tap_class=TapMongoDB, config=SAMPLE_CONFIG)

# TODO: Create additional tests as appropriate for your tap.
v
Could do something like this I think https://github.com/AutoIDM/tap-clickup/blob/main/tap_clickup/tests/test_state.py#L150-L153 No easy helper functions I know of off the bat but I'm sure there's some somewhere! I find this not to be too bad 🤷
m
hmm. that seems to depend on the meltano.yml file being present, though, as that’s where i’ve been setting the metadata. And the tests run outside the meltano “context”, just invoked via pytest+poetry.
v
It doesn't depend on Meltano!
m
welp, I guess I need to dig more into singer internals? How can I change the replication method in this context?
v
This is probably wrong but close to the idea
Copy code
for stream in catalog1["streams"]:
        if stream.get("stream") and stream["stream"] not in ("task", "team"):
            for metadata in stream["metadata"]:
                metadata["metadata"]["replication-method"] = "FULL_TABLE"
Not I"m hand waving a bit here but generally that's the idea
m
OH! I think I misread your first message. I see now what you mean - I could have the test get the streams and then override the replication method 💡
thank you!