Hi all, I am new to Meltano and exploring the tool...
# getting-started
a
Hi all, I am new to Meltano and exploring the tool. While running "meltano run" getting below error as showing in snip. How to resolve this? Thanks in advance.
e
The
meltano run
command expects at least one argument. https://docs.meltano.com/reference/command-line-interface#run
a
Thanks @Edgar Ramírez (Arch.dev)
After running with arguments I am getting below error
here is my meltano.yml file
Any help here Please
a
You need entity, path and keys specificed under the
files:
Copy code
{	"entity" : "leads",
						"path" : "/path/to/leads.csv",
						"keys" : ["Id"],
						"delimiter": ";"
					},
You have
entity
but
file
is not a setting by the looks of things
a
Thanks @Andy Carter Will check by adding this
a
Sorry this is a better link if you want to define files in
meltano.yml
https://hub.meltano.com/extractors/tap-csv#files-setting
a
Thanks. but still facing issue
file is available in that path, but still showing error
Please any one help here
e
What's the absolute path of the file on your machine?
a
this is the path(/home/rsiva_sai/meltano-project/poc/)
e
Ok. What's the output of
head -n1 /home/rsiva_sai/meltano-project/poc/testing.csv
?
a
name,legacy_id,location
e
Ok. What does
meltano.yml
look like after the changes? I think something's still misconfigured.
a
e
Ok. Try instead
Copy code
config:
  files:
  - entity: testing
    path: /home/rsiva_sai/meltano-project/poc/testing.csv
    keys: [name]
a
Thanks @Edgar Ramírez (Arch.dev) The data is able to load. But, why the schema is changed?
e
changed from what?
a
it should be look like this
e
tl;dr z3z1ma/target-bigquery makes the decision of nesting the record in a
data
field of JSON type
other targets do not
a
Thanks let me check that link
also, some times while loading it getting struck only table is creating and data is not loading
also, if you dontmind can you please share some documents regarding how to start and end learning Meltano completely.
e
only table is creating and data is not loading
I haven't used z3z1ma/target-bigquery beyond simple testing, but it seems to be doing some work. You could also try setting the
LOGLEVEL=debug
environment variable and rerunning in case the target is logging things at the debug level.
also, if you dontmind can you please share some documents regarding how to start and end learning Meltano completely.
https://docs.meltano.com/getting-started/ is probably the best resource
a
Thanks @Edgar Ramírez (Arch.dev)
np 1
Hi, I am trying to extract the data from google sheet and load it to bigquery But facing the below issue. Any help here please
This is my meltano.yaml file
Hi @Edgar Ramírez (Arch.dev) Can you please help here
e
Does
meltano config tap-google-sheets test
work for you?
a
Hi
Showing this error
in first image there it showing plugin configuration invalid @Edgar Ramírez (Arch.dev)
e
Can you confirm your credentials have access to that instance of gsheet?
a
ya its working now
why these extra column are creating and how to avoid it?
@Edgar Ramírez (Arch.dev) can you please help
a
@amor apprendido they are the record metadata columns added by targets (not taps): https://sdk.meltano.com/en/latest/implementation/record_metadata.html#record-metadata Under your
target-bigquery
config in yaml, try adding
add_record_metadata: false
and then see if that removes the columns. You might need to delete your target tables and run a full refresh.
a
Hi @Andy Carter I tried the same but still metadata creating