https://linen.dev logo
Join Slack
Powered by
I'm testing out the new target-oracle (<https://hu...
# troubleshooting
m
I'm testing out the new target-oracle (https://hub.meltano.com/loaders/target-oracle/), and the output from the pipeline is resulting in only empty rows for each record. I know the tap is working fine, and I can swap out the target-oracle for target-csv in the pipeline and the output is fine. It could be that I didn't configure the stream_maps setting? Are there any recommendations for anything I can do to begin troubleshooting where this problem is occurring? I have logging set to debug, but even with the verbose output I don't see anything obviously wrong.
v
https://github.com/meltano/meltano/issues/6613#issuecomment-1215074973 This comment is about performance but some of the steps can apply here I'd can an output from your tap
Copy code
meltano invoke tap-name > tap.singer
And verify there isn't any records that are empty there. Then I'd use that 
tap.singer
file and trim away the excess (just leave a schema and a corresponding record line) then see if it still happens? If it does cool you have a reproducible example now and it's almost certainly the target.
To run the target with that new file you do something like 
cat tap.singer | meltano invoke target-oracle
m
Thanks @visch, that looks like a good way to eliminate the tap. Doing that now.
I followed your suggestion and created a safe output from the tap and tried to insert it to the target with similar results, two empty rows in the target oracle database. I'm attaching the output from the tap and the logs from the run here in case you're curious. I appreciate the help so far, if you have any other ideas I could do to troubleshoot I'm all ears.
tap.eirb
v
The target oracle you're using has multiple issues it looks like 😕 first one 
INSERT INTO i_r_b_i2b2_direct_view
is almost certainly this bug (underscores in name) https://github.com/meltano/sdk/issues/1205 Multiple records is definietly wrong too. From here I"d put an issue into the target dev, he's actually active here (I hesitate to ping him but @Henning Holgersen 😄 )
From a quick run through the code there isn't anything jumping out at me but if you're trying to fix the target I'd look here https://github.com/radbrt/target-oracle/blob/c6850f03cb4890708033e82ab3124daa720a9dbf/target_oracle/sinks.py#L376
m
I had noticed the underscores in name with camelcase problem, but figured one thing at a time 🙂 , glad to have that link. The target-oracle is so fresh, I expect there to bit plenty of issues, hopefully I can contribute back some to it.
v
😄 that would be great, focusing on the record issue is what I'd recommend. There's something going on there. Maybe it's sneaking in here https://github.com/radbrt/target-oracle/blob/c6850f03cb4890708033e82ab3124daa720a9dbf/target_oracle/sinks.py#L218
h
Hi @michael_horvath, glad to hear someone trying out the target. I had some trouble with empty tables while developing, it seemed the temporary tables were… a little too temporary. So I switched it out with non-temporary tables, which seemed to work. Are you able to try running it with a stream that has no key? If that works, it might be the same issue that keeps returning. If not, I will have to start searching again. Thank you for the logs, I will have a look at them tomorrow.
OK @michael_horvath, after once again loosing a few arguments with Oracle drivers, I have been able to reproduce your issue and confirmed that the problem is the casing of the column names. I even tried with all-caps with the same result as you, it was fixed once I changed your sample-stream to all lowercase. I have logged this as an issue here: https://github.com/radbrt/target-oracle/issues/2. I’ll keep you updated, and thank you for finding this. @visch: We should create some tests for table contents as well. I don’t remember if I have camelcase in my testsuite currently, but if I do it would pass without problem.
v
Yeah there's two spots conform gets called that have to be overridden I forget exactly where, but yes the tests should absolutely catch this
m
That's great, thanks for finding that @Henning Holgersen. I'm very excited to be able to use oracle as a target, thanks for working on this plugin
h
@michael_horvath I pushed a fix to the repo, it looks good on my simple testcases, let me know if you try it again. The current results can be a little… inelegant… we have a way to go on dealing with camel cased columns so be aware there might be breaking changes in the future. I’ll add a tag/release for you if it works for you and you want to peg the version, just let me know.
m
Great success, I was able to load some of my data with the changes you made. I do see the camelcase issue showing up in column names as well, but I'm happy with what I have.
h
Good to hear @michael_horvath. I will be improving the column naming in the days ahead, but I created an initial release so that you can avoid any breaking changes if you want: https://github.com/radbrt/target-oracle/releases/tag/v0
Open in Slack
PreviousNext
Powered by