https://meltano.com/ logo
#announcements
Title
# announcements
f

future-lifeguard-59603

03/16/2021, 3:52 PM
Hey, I'm trying to get UI running but getting the following error thrown:
(.venv) manuelbiermann@MacBook-Air talmore-topjobs % meltano ui
`[2021-03-16 164735,278] [24309|MainThread|root] [WARNING] The
discovery.yml
manifest received from https://www.meltano.com/discovery.yml has version 17, while this version of Meltano requires version 16.` `[2021-03-16 164735,279] [24309|MainThread|root] [WARNING] Please install the latest compatible version of Meltano using
meltano upgrade
.` `[2021-03-16 164735,279] [24309|MainThread|root] [WARNING] Falling back on the cached
discovery.yml
manifest...`
[2021-03-16 16:47:35,403] [24309|MainThread|root] [INFO] Auto-compiling models in '/Users/manuelbiermann/Development/meltano-projects/talmore-topjobs/model'
[2021-03-16 16:47:35,408] [24309|MainThread|meltano.cli.ui] [INFO] All workers started.
[2021-03-16 16:47:35,771] [24310|MainThread|gunicorn.error] [INFO] Starting gunicorn 19.10.0
[2021-03-16 16:47:35,772] [24310|MainThread|gunicorn.error] [INFO] Listening at: <http://0.0.0.0:5000> (24310)
[2021-03-16 16:47:35,772] [24310|MainThread|gunicorn.error] [INFO] Using worker: sync
/opt/homebrew/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py:1023: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
 
return io.open(fd, *args, **kwargs)
[2021-03-16 16:47:35,774] [24311|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24311
[2021-03-16 16:47:35,878] [24312|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24312
[2021-03-16 16:47:35,897] [24313|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24313
[2021-03-16 16:47:35,991] [24314|MainThread|gunicorn.error] [INFO] Booting worker with pid: 24314
[2021-03-16 16:47:37,472] [24313|MainThread|<http://meltano.api.app|meltano.api.app>] [ERROR] Exception on / [GET]
Traceback (most recent call last):
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/sqlalchemy/util/_collections.py", line 1008, in __call__
  
return self.registry[key]
KeyError: <greenlet.greenlet object at 0x102aa3670 (otid=0x102144ec0) current active started main>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
  
response = self.full_dispatch_request()
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
  
rv = self.handle_user_exception(e)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router
  
return original_handler(e)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router
  
return original_handler(e)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
  
reraise(exc_type, exc_value, tb)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
  
raise value
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 1948, in full_dispatch_request
  
rv = self.preprocess_request()
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask/app.py", line 2242, in preprocess_request
  
rv = func()
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_principal.py", line 479, in _on_before_request
  
self.set_identity(identity)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_principal.py", line 418, in set_identity
  
self._set_thread_identity(identity)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_principal.py", line 462, in _set_thread_identity
  
identity_loaded.send(current_app._get_current_object(),
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/blinker/base.py", line 266, in send
  
return [(receiver, receiver(sender, **kwargs))
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/blinker/base.py", line 266, in <listcomp>
  
return [(receiver, receiver(sender, **kwargs))
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/meltano/api/security/auth.py", line 151, in _identity_loaded_hook
  
for role in current_user.roles:
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/werkzeug/local.py", line 348, in __getattr__
  
return getattr(self._get_current_object(), name)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/meltano/api/security/identity.py", line 65, in roles
  
return set(Role.query.all())
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 514, in __get__
  
return type.query_class(mapper, session=<http://self.sa|self.sa>.session())
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/sqlalchemy/orm/scoping.py", line 128, in __call__
  
return self.registry()
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/sqlalchemy/util/_collections.py", line 1010, in __call__
  
return self.registry.setdefault(key, self.createfunc())
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 4027, in __call__
  
return self.class_(**local_kw)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 138, in __init__
  
bind = options.pop('bind', None) or db.engine
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 943, in engine
  
return self.get_engine()
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 962, in get_engine
  
return connector.get_engine()
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 555, in get_engine
  
options = self.get_options(sa_url, echo)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 570, in get_options
  
self._sa.apply_driver_hacks(self._app, sa_url, options)
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/flask_sqlalchemy/__init__.py", line 914, in apply_driver_hacks
  
sa_url.database = os.path.join(app.root_path, sa_url.database)
AttributeError: can't set attribute
[2021-03-16 16:47:37,480] [24313|MainThread|meltano.api] [INFO] Error: 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
[2021-03-16 16:47:39,508] [24314|MainThread|<http://meltano.api.app|meltano.api.app>] [ERROR] Exception on / [GET]
Traceback (most recent call last):
 
File "/Users/manuelbiermann/Development/meltano-projects/.venv/lib/python3.9/site-packages/sqlalchemy/util/_collections.py", line 1008, in __call__
  
return self.registry[key]
KeyError: <greenlet.greenlet object at 0x102aa3670 (otid=0x102144ec0) current active started main>
1
I'm on macOs 11.2.2, Apple Chip, Python 3.9
f

flat-bear-81546

03/16/2021, 3:59 PM
Have you tried running
meltano upgrade
?
Looks like an issue that should be in, as I'd think even if your version of Meltano is behind you should be able to use Discovery. I don't know that I"m just spit balling!
But upgrade should fix it (i think)
r

ripe-musician-59933

03/16/2021, 5:22 PM
@future-lifeguard-59603 Meltano does not currently support Python 3.9: https://gitlab.com/meltano/meltano/-/blob/master/pyproject.toml#L20, https://gitlab.com/meltano/meltano/-/issues/2545 Can you try again in 3.8?
f

flat-bear-81546

03/16/2021, 5:31 PM
🙂 I was wrong
f

future-lifeguard-59603

03/16/2021, 5:47 PM
I'll give it a try, what's the best way to downgrade to 3.8?
r

ripe-musician-59933

03/16/2021, 5:56 PM
Looks like you installed
python@3.9
using Homebrew. You can separately install
python@3.8
and I think Homebrew should give you some way of making one the default instead of the other. Then you can reinstall Meltano and try again
Note that i'd typically recommend using pyenv to manage different Python installations
f

future-lifeguard-59603

03/16/2021, 6:14 PM
Should I brew remove python 3.9 first and then use pyenv?
Going straight for pyenv gives me this:
manuelbiermann@MacBook-Air ~ % pyenv install 3.8.7
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.8.7.tar.xz...
-> <https://www.python.org/ftp/python/3.8.7/Python-3.8.7.tar.xz>
Installing Python-3.8.7...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
*BUILD FAILED* (OS X 11.2.2 using python-build 20180424)
Inspect or clean up the working tree at /var/folders/l2/xxh1pz194278z_rwwyrsvwqm0000gn/T/python-build.20210316191048.26753
Results logged to /var/folders/l2/xxh1pz194278z_rwwyrsvwqm0000gn/T/python-build.20210316191048.26753.log
Last 10 log lines:
checking size of _Bool... 1
checking size of off_t... 8
checking whether to enable large file support... no
checking size of time_t... 8
checking for pthread_t... yes
checking size of pthread_t... 8
checking size of pthread_key_t... 8
checking whether pthread_key_t is compatible with int... no
configure: error: Unexpected output of 'arch' on OSX
make: *** No targets specified and no makefile found. Stop.
r

ripe-musician-59933

03/16/2021, 6:16 PM
@future-lifeguard-59603 Looks like pyenv still has some issues on M1: https://github.com/pyenv/pyenv/issues/1768
f

future-lifeguard-59603

03/16/2021, 6:45 PM
Awesome, this worked for me:
Copy code
pyenv install --patch 3.8.7 <<(curl -sSL <https://raw.githubusercontent.com/Homebrew/formula-patches/9811be33170a8f31a684fae6955542e31eb4e61e/python/3.8.7.patch>)
👍 1
Seems like it was still installing it under 3.9. Now with setting it globally to 3.8.7 I'm getting a lot of errors thrown when running pip3 install meltano:
Untitled
r

ripe-musician-59933

03/16/2021, 7:31 PM
@future-lifeguard-59603 Seems like the issue is with installing numpy...
f

future-lifeguard-59603

03/16/2021, 7:41 PM
Before trying to install Meltano I ran now:
(.venv) manuelbiermann@MacBook-Air meltano-projects-3-8-7 % brew install openblas
OPENBLAS="$(brew --prefix openblas)" pip3 install numpy
==> *Auto-updated Homebrew!*
Updated 1 tap (homebrew/core).
==> *Updated Formulae*
Updated 1 formula.
Warning: openblas 0.3.13 is already installed and up-to-date.
To reinstall 0.3.13, run:
 
brew reinstall openblas
Processing /Users/manuelbiermann/Library/Caches/pip/wheels/d5/e2/15/f5249b342e83b9321f9081c1318f566d28307eb0434d6c1c20/numpy-1.20.1-cp38-cp38-macosx_11_2_arm64.whl
Installing collected packages: numpy
Successfully installed numpy-1.20.1
WARNING: You are using pip version 20.2.3; however, version 21.0.1 is available.
You should consider upgrading via the '/Users/manuelbiermann/Development/meltano-projects-3-8-7/.venv/bin/python3 -m pip install --upgrade pip' command.
And it's still failing at the same spot
Untitled
r

ripe-musician-59933

03/16/2021, 9:58 PM
@future-lifeguard-59603 Annoying 😕 I'm not sure how I can be of more help though, since this doesn't look specific to Meltano
f

future-lifeguard-59603

03/16/2021, 10:06 PM
Yeah, it seems to be a problem with poetry which doesn’t like BigSur and hates the Apple Chipset 😄 https://github.com/python-poetry/poetry/issues/3458
👍 1
So I got it running on my M1. Here are the steps needed: 1. Install miniforge/conda via homebrew (https://github.com/conda-forge/miniforge) 2. Install numpy via conda-forge (https://numpy.org/install/) [you might have to do the same for pyarrow] 3. update pip in your venv: pip3 install --upgrade pip 4. Go ahead with: pip3 install meltano
👍 1
s

salmon-actor-23953

03/25/2021, 6:06 PM
@future-lifeguard-59603 - thanks so much for sharing back your experience and your workaround. I’ve logged this M1 topic here in our issue tracker: Macbook M1 Support (#2675) · Issues · meltano / Meltano · GitLab Should I keep this open to track further improvements in compatibility, or close it since there’s a known workaround?