Hey I m trying to get UI running but getting the following e Meltano #announcements

Hey, I'm trying to get UI running but getting the ...

manuel_biermann

03/16/2021, 3:52 PM

Hey, I'm trying to get UI running but getting the following error thrown:

(.venv) manuelbiermann@MacBook-Air talmore-topjobs % meltano ui

`[2021-03-16 164735,278] [24309|MainThread|root] [WARNING] The

discovery.yml

manifest received from https://www.meltano.com/discovery.yml has version 17, while this version of Meltano requires version 16.` `[2021-03-16 164735,279] [24309|MainThread|root] [WARNING] Please install the latest compatible version of Meltano using

meltano upgrade

.` `[2021-03-16 164735,279] [24309|MainThread|root] [WARNING] Falling back on the cached

discovery.yml

manifest...`

[2021-03-16 16:47:35,403] [24309|MainThread|root] [INFO] Auto-compiling models in '/Users/manuelbiermann/Development/meltano-projects/talmore-topjobs/model'

[2021-03-16 16:47:35,408] [24309|MainThread|meltano.cli.ui] [INFO] All workers started.

[2021-03-16 16:47:35,771] [24310|MainThread|gunicorn.error] [INFO] Starting gunicorn 19.10.0

[2021-03-16 16:47:35,772] [24310|MainThread|gunicorn.error] [INFO] Listening at: <http://0.0.0.0:5000> (24310)

[2021-03-16 16:47:35,772] [24310|MainThread|gunicorn.error] [INFO] Using worker: sync

/opt/homebrew/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py:1023: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used

return io.open(fd, *args, **kwargs)

[2021-03-16 16:47:35,774] [24311|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24311

[2021-03-16 16:47:35,878] [24312|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24312

[2021-03-16 16:47:35,897] [24313|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24313

[2021-03-16 16:47:35,991] [24314|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24314

[2021-03-16 16:47:37,472] [24313|MainThread|<http://meltano.api.app|meltano.api.app>] [ERROR] Exception on / [GET]

Traceback (most recent call last):

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/sqlalchemy/util/_collections.py", line 1008, in __call__

return self.registry[key]

KeyError: <greenlet.greenlet object at 0x102aa3670 (otid=0x102144ec0) current active started main>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app

response = self.full_dispatch_request()

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router

return original_handler(e)

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router

return original_handler(e)

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception

reraise(exc_type, exc_value, tb)

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise

raise value

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 1948, in full_dispatch_request

rv = self.preprocess_request()

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 2242, in preprocess_request

rv = func()

File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_principal.py", line 479, in _on_before_request

self.set_identity(identity)

`File "/Users/manuelbiermann/De…

manuel_biermann

03/16/2021, 3:54 PM

I'm on macOs 11.2.2, Apple Chip, Python 3.9

visch

03/16/2021, 3:59 PM

Have you tried running

meltano upgrade

visch

03/16/2021, 4:01 PM

Looks like an issue that should be in, as I'd think even if your version of Meltano is behind you should be able to use Discovery. I don't know that I"m just spit balling!

visch

03/16/2021, 4:01 PM

But upgrade should fix it (i think)

douwe_maan

03/16/2021, 5:22 PM

@manuel_biermann Meltano does not currently support Python 3.9: https://gitlab.com/meltano/meltano/-/blob/master/pyproject.toml#L20, https://gitlab.com/meltano/meltano/-/issues/2545 Can you try again in 3.8?

visch

03/16/2021, 5:31 PM

🙂 I was wrong

manuel_biermann

03/16/2021, 5:47 PM

I'll give it a try, what's the best way to downgrade to 3.8?

douwe_maan

03/16/2021, 5:56 PM

Looks like you installed

python@3.9

using Homebrew. You can separately install

python@3.8

and I think Homebrew should give you some way of making one the default instead of the other. Then you can reinstall Meltano and try again

douwe_maan

03/16/2021, 5:56 PM

Note that i'd typically recommend using pyenv to manage different Python installations

manuel_biermann

03/16/2021, 6:14 PM

Should I brew remove python 3.9 first and then use pyenv?

manuel_biermann

03/16/2021, 6:14 PM

Going straight for pyenv gives me this:

manuel_biermann

03/16/2021, 6:14 PM

manuelbiermann@MacBook-Air ~ % pyenv install 3.8.7

python-build: use openssl@1.1 from homebrew

python-build: use readline from homebrew

Downloading Python-3.8.7.tar.xz...

-> <https://www.python.org/ftp/python/3.8.7/Python-3.8.7.tar.xz>

Installing Python-3.8.7...

python-build: use readline from homebrew

python-build: use zlib from xcode sdk

*BUILD FAILED* (OS X 11.2.2 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/l2/xxh1pz194278z_rwwyrsvwqm0000gn/T/python-build.20210316191048.26753

Results logged to /var/folders/l2/xxh1pz194278z_rwwyrsvwqm0000gn/T/python-build.20210316191048.26753.log

Last 10 log lines:

checking size of _Bool... 1

checking size of off_t... 8

checking whether to enable large file support... no

checking size of time_t... 8

checking for pthread_t... yes

checking size of pthread_t... 8

checking size of pthread_key_t... 8

checking whether pthread_key_t is compatible with int... no

configure: error: Unexpected output of 'arch' on OSX

make: *** No targets specified and no makefile found. Stop.

douwe_maan

03/16/2021, 6:16 PM

@manuel_biermann Looks like pyenv still has some issues on M1: https://github.com/pyenv/pyenv/issues/1768

manuel_biermann

03/16/2021, 6:45 PM

Awesome, this worked for me:

Copy code

pyenv install --patch 3.8.7 <<(curl -sSL <https://raw.githubusercontent.com/Homebrew/formula-patches/9811be33170a8f31a684fae6955542e31eb4e61e/python/3.8.7.patch>)

manuel_biermann

03/16/2021, 7:20 PM

Seems like it was still installing it under 3.9. Now with setting it globally to 3.8.7 I'm getting a lot of errors thrown when running pip3 install meltano:

manuel_biermann

03/16/2021, 7:23 PM

Untitled

douwe_maan

03/16/2021, 7:31 PM

@manuel_biermann Seems like the issue is with installing numpy...

douwe_maan

03/16/2021, 7:33 PM

https://github.com/numpy/numpy/issues/17807

manuel_biermann

03/16/2021, 7:41 PM

Before trying to install Meltano I ran now:

(.venv) manuelbiermann@MacBook-Air meltano-projects-3-8-7 % brew install openblas

OPENBLAS="$(brew --prefix openblas)" pip3 install numpy

==> *Auto-updated Homebrew!*

Updated 1 tap (homebrew/core).

==> *Updated Formulae*

Updated 1 formula.

Warning: openblas 0.3.13 is already installed and up-to-date.

To reinstall 0.3.13, run:

brew reinstall openblas

Processing /Users/manuelbiermann/Library/Caches/pip/wheels/d5/e2/15/f5249b342e83b9321f9081c1318f566d28307eb0434d6c1c20/numpy-1.20.1-cp38-cp38-macosx_11_2_arm64.whl

Installing collected packages: numpy

Successfully installed numpy-1.20.1

WARNING: You are using pip version 20.2.3; however, version 21.0.1 is available.

You should consider upgrading via the '/Users/manuelbiermann/Development/meltano-projects-3-8-7/.venv/bin/python3 -m pip install --upgrade pip' command.

manuel_biermann

03/16/2021, 7:42 PM

And it's still failing at the same spot

manuel_biermann

03/16/2021, 7:43 PM

Untitled

douwe_maan

03/16/2021, 9:58 PM

@manuel_biermann Annoying 😕 I'm not sure how I can be of more help though, since this doesn't look specific to Meltano

manuel_biermann

03/16/2021, 10:06 PM

Yeah, it seems to be a problem with poetry which doesn’t like BigSur and hates the Apple Chipset 😄 https://github.com/python-poetry/poetry/issues/3458

manuel_biermann

03/17/2021, 12:06 PM

So I got it running on my M1. Here are the steps needed: 1. Install miniforge/conda via homebrew (https://github.com/conda-forge/miniforge) 2. Install numpy via conda-forge (https://numpy.org/install/) [you might have to do the same for pyarrow] 3. update pip in your venv: pip3 install --upgrade pip 4. Go ahead with: pip3 install meltano

aaronsteers

03/25/2021, 6:06 PM

@manuel_biermann - thanks so much for sharing back your experience and your workaround. I’ve logged this M1 topic here in our issue tracker: Macbook M1 Support (#2675) · Issues · meltano / Meltano · GitLab Should I keep this open to track further improvements in compatibility, or close it since there’s a known workaround?

10 Views

Open in Slack

Previous Next