Siddu Hussain
06/13/2024, 8:00 PMbatch_size_rows: 150000
batch_config:
batch_size: 150000
encoding:
format: jsonl
compression: gzip
storage:
root: ""
I don't see this happening for API calls, as the data I pull is below 10000 when I see target it is still loading data inbetween the batch size , So I was wondering if I am doing something wrong .