https://linen.dev logo
Join Slack
Powered by
Hi! I am doing a test run with a .<csv file> I upl...
# plugins-general
m
Hi! I am doing a test run with a .csv file I uploaded to my github. Raw is here. Trying to make sure my meltano.yml is right. Looked at previous successful meltano run, docs and did a search on the slack for references. Could someone verify this? Thanks!
Copy code
version: 1
send_anonymous_usage_stats: true
project_id: 045333fd-3826-4b33-8f43-bfd8de29749f
plugins:
  extractors:
  - name: tap-investing
    namespace: tap_investing
    pip_url: git+<https://gitlab.com/DouweM/tap-investing.git>
    settings:
    - name: assets
      kind: object
  - name: tap-spreadsheets-anywhere
    pip_url: git+<https://github.com/ets/tap-spreadsheets-anywhere.git>
    config:
      tables:
      - path: <https://raw.githubusercontent.com/mindyng/2021-Projects/main/>
        name: amazon_stocks
        pattern: null
        start_date: '2019-01-02T00:00:00Z'
        key_properties: []
        format: csv
        delimiter: ','
  loaders:
  - name: target-postgres
    variant: meltano
    pip_url: git+<https://github.com/meltano/target-postgres.git>
    config:
      user: postgres
      host: 127.0.0.1
      port: 5432
      dbname: pricing
t
the meltano.yml looks right at first glance. the csv though seems like it’s missing a header for the very first column
m
@taylor thanks for the heads up on index! I made the column header and ran 
meltano elt tap-spreadsheets-anywhere target-postgres
though got an error msg:
something wrong with my meltano.yml? pattern val?
meawhile, upgraded meltano. its up to date atm
t
I see you’re on Python 3.9. We are seeing some problems with other users on that version of Python. Can you try 3.8 in a new virtualenvironment?
fyi @aaronsteers for more Python 3.9 problems
a
@mindy_ng - The python version could be a problem, as Taylor calls out. Also, I noticed your meltano.yml has ‘null’ for pattern, and the error message implies it is expecting a string there. Perhaps, have you tried removing the line 
pattern: null
or perhaps putting another text value in that space?
m
@aaronsteers i tried commenting out pattern and re ran same command and same error occurred
so i just replaced pattern: null with 
pattern: '.csv'
and now I got this:
not sure why its asking me to complete 
meltano upgrade
since i already did
but extract and load is done! right? not sure what this means/whats the significance of it (at end of my last output u/l)? 
type object '_asyncio.Task' has no attribute 'all_tasks'
and 
transformation skipped
<-- thats fine since im doing it in dbt, right?
a
Yes, This looks a lot better.
m
what about the type object?
a
This error (
type object '_asyncio.Task' has no attribute 'all_tasks'
) looks like it is indicative of the python version. Reverting to Python 3.8 if you are able should resolve that issue.
(@douwe_maan - FYI) @mindy_ng - we can certainly fix this in a future release of Meltano and I’ve logged the issue here along with a promising fix I was able to locate: Bug re: Python 3.9 and asyncio (#2662) That said, there are other areas where Python 3.9 is not yet compatible and it’s safest to move to 3.8 if you are able until such time as 3.9 is broadly supported by Meltano and all related tools and libraries.
Open in Slack
PreviousNext
Powered by