hi, I'm getting an error of loading false data, can anyone help me with the error

Can you copy some more of the error above? And paste the text rather than an image? What command are you running?

2023-05-30T084345.839786Z [error ] Loader failed 2023-05-30T084345.842298Z [error ] Block run completed. block_type=ExtractLoadBlocks err=RunnerError('Loader failed') exit_codes={<PluginType.LOADERS: 'loaders'>: 1} set_number=0 success=False Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to join our friendly Slack community. Run invocation could not be completed as block failed: Loader failed

Can you copy more lines above? Maybe 50 more lines? There is probably a Python error somewhere above.

lof@lof-H110M-Gaming-3:~/DEV/Python/dagster-meltano/meltano_project$ meltano run tap-shopify target-jsonl 2023-05-30T084343.075147Z [info ] Environment 'dev' is active 2023-05-30T084343.472162Z [warning ] No state was found, complete import. 2023-05-30T084343.799509Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=tap-shopify v0.0.1, Meltano SDK v0.4.9) cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.799918Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=Skipping parse of env var settings... cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.800058Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=Config validation passed with 0 warnings. cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.804391Z [info ] time=2023-05-30 154343 name=root level=INFO message=Operator '__else__=None' was not found. Unmapped streams will be included in output. cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.812949Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=Added 'inventory_items' as child stream to 'inventory_levels' cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.813170Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=Added 'inventory_levels' as child stream to 'locations' cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.813322Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=Added 'transactions' as child stream to 'orders' cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.813759Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=Beginning incremental sync of 'abandoned_checkouts'... cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084343.813946Z [info ] time=2023-05-30 154343 name=tap-shopify level=INFO message=Tap has custom mapper. Using 1 provided map(s). cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084344.172858Z [info ] time=2023-05-30 154344 name=tap-shopify level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 0.355799, 'tags': {'endpoint': '/api/2022-01/checkouts.json', 'http_status_code': 200, 'status': 'succeeded'}} cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084344.224956Z [info ] time=2023-05-30 154344 name=tap-shopify level=INFO message=INFO METRIC: {'type': 'counter', 'metric': 'record_count', 'value': 0, 'tags': {'stream': 'abandoned_checkouts'}} cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084344.225390Z [info ] time=2023-05-30 154344 name=tap-shopify level=INFO message=Beginning incremental sync of 'collects'... cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084344.225607Z [info ] time=2023-05-30 154344 name=tap-shopify level=INFO message=Tap has custom mapper. Using 1 provided map(s). cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify 2023-05-30T084344.584962Z [info ] time=2023-05-30 154344 name=tap-shopify level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 0.356935, 'tags': {'endpoint': '/api/2022-01/collects.json', 'http_status_code': 200, 'status': 'succeeded'}} cmd_type=elb consumer=False name=tap-shopify producer=True stdio=std…

stdio=stderr string_id=target-jsonl
2023-05-30T08:43:45.814691Z [info     ] jsonschema.exceptions.ValidationError: None is not of type 'string' cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2023-05-30T08:43:45.815176Z [info     ]                                cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2023-05-30T08:43:45.815484Z [info     ] Failed validating 'type' in schema['properties']['first_name']: cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl

Not familiar with the tap but looks like some kind of validation issue on 'first_name'.

i run the command meltano invoke tap-shopify paid for my data

What about the output when running

invoke

? Do you get a similar error?

I run on the command, the system has obtained the data on shopify

It's the validation step which is failing, as you have a blank 'first_name'. The validation doesn't happen if you run 'invoke', it's only when you run a tap & target combination.

@daniel_walker it seems like a schema needs a quick patch 👆

Hey, I'm taking a look into this now. Should be able to quickly change this schema, but in testing it myself I've run into another schema expect

string

got

null

issue. @edgar_ramirez_mondragon I think when using the SDK to build a schema as a python class it makes values nullable by default, is there any way to make the

.json

schemas nullable on import as well? Might just be a case that I switch this tap over to SDK schemas.

is there any way to make the

.json

schemas nullable on import as well?

There’s not and we probably won’t prioritize that ourselves since we’ll encourage the use of

singer_sdk.typing

classes

We can fix the issue with

first_name

easily by updating the schema, but I think that might be just the tip of the iceberg here (related). I think that our best course of action is probably to retire the

.json

schemas in favour of the SDK class-based approach, but that will not be a quick fix. In the meantime, we have pushed a fix to make

first_name

and

last_name

nullable to hopefully resolve this issue with what information we have here. @dinh_d___c_d____ng You will need to

meltano install --clean extractor tap-shopify

. If you still have problems, it would be great if you could provide some sample record data that we can work with to narrow down the what constitutes a valid schema for each stream. 🙂

@edgar_ramirez_mondragon just to confirm from Reubens earlier message, if you use typing classes as opposed to a JSON file for schema, do you get 'nullable' by default?

@Andy Carter yes, all properties with

required=False

(the default) will be serialized with

"type": […, "null"]