I am getting this in the logs on Meltano version 1...
# announcements
m
I am getting this in the logs on Meltano version 1.34.1
Copy code
[2020-05-27 17:43:09,618|W|MainThread|root] You must update Meltano to get new plugins updates.
[2020-05-27 17:43:09,778|C|MainThread|root] This version of Meltano cannot parse the plugins manifest, please update Meltano.
[2020-05-27 17:43:09,778|E|MainThread|meltano.cli.ui] Initial compilation failed: Version 10 required, currently at 11
[2020-05-27 17:43:09,787|I|MainThread|meltano.cli.ui] All workers started.
Is there a way to update to the correct version?
a
meltano upgrade
I believe
m
I have tried that. I get this in the update:
Copy code
System database up-to-date.
[2020-05-27 17:53:44,363|W|MainThread|root] You must update Meltano to get new plugins updates.
[2020-05-27 17:53:44,527|C|MainThread|root] This version of Meltano cannot parse the plugins manifest, please update Meltano.
Traceback (most recent call last):
  File "/usr/local/bin/meltano", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/meltano/cli/__init__.py", line 27, in main
    cli(obj={"project": None})
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/meltano/cli/params.py", line 24, in decorate
    return func(engine_uri, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/meltano/cli/params.py", line 56, in decorate
    
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/meltano/cli/upgrade.py", line 34, in upgrade
    upgrade_service.upgrade(**kwargs)
  File "/usr/local/lib/python3.6/site-packages/meltano/core/upgrade_service.py", line 96, in upgrade
    self.compile_models()
  File "/usr/local/lib/python3.6/site-packages/meltano/core/upgrade_service.py", line 87, in compile_models
    ProjectCompiler(self.project).compile()
  File "/usr/local/lib/python3.6/site-packages/fasteners/lock.py", line 293, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/meltano/core/compiler/project_compiler.py", line 44, in compile
    self.parse()
  File "/usr/local/lib/python3.6/site-packages/meltano/core/compiler/project_compiler.py", line 35, in parse
    self.package_topics = self.parser.parse_packages()
  File "/usr/local/lib/python3.6/site-packages/meltano/core/m5o/m5o_file_parser.py", line 207, in parse_packages
    plugin for plugin in discovery.plugins() if plugin.type is PluginType.MODELS
  File "/usr/local/lib/python3.6/site-packages/meltano/core/m5o/m5o_file_parser.py", line 207, in <listcomp>
    plugin for plugin in discovery.plugins() if plugin.type is PluginType.MODELS
  File "/usr/local/lib/python3.6/site-packages/meltano/core/plugin_discovery_service.py", line 179, in plugins
    yield from discovery_plugins
  File "/usr/local/lib/python3.6/site-packages/meltano/core/plugin_discovery_service.py", line 175, in <genexpr>
    for plugin in self.discovery[plugin_type]
  File "/usr/local/lib/python3.6/site-packages/meltano/core/plugin_discovery_service.py", line 108, in discovery
    self.ensure_compatible()
  File "/usr/local/lib/python3.6/site-packages/meltano/core/behavior/versioned.py", line 37, in ensure_compatible
    version,
meltano.core.behavior.versioned.IncompatibleVersionError: Version 10 required, currently at 11
a
1.34.1 looks like the latest version. Sounds like something internal
d
@matthew_trinkle Hmm, that’s not right. The error message implies that in your meltano installation, this line: https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin_discovery_service.py#L39 still says
10
, rather than
11
, despite that having been changed in v1.34.0 (and v1.34.1). Can you check if that’s actually the case on your file system?
It’s possible that “meltano upgrade” specifically is broken when it comes to changes to this version identifier. (in which case we’ll want to file an issue for that) But “meltano upgrade” should still successfully update your meltano installation, and subsequent “meltano” command invocations should succeed without complaining about the version. Assuming that file actually got updated, that is.
m
where is meltano core located in the docker?
d
@matthew_trinkle Ah, you're using the Docker image from https://hub.docker.com/r/meltano/meltano?
m
I believe so
d
@matthew_trinkle Can you show me the output of
docker run meltano/meltano --version
?
m
1.32.1, but inside docker
meltano --version
says 1.34.1
d
@matthew_trinkle All right, I think the issue is that
meltano upgrade
is not actually supported when using the Docker installation method (which I didn't fully realize until now and isn't documented), because while you can make changes inside a Docker container, the Docker image itself is a snapshot and won't be updated until you actually pull a new version of the image. So if you're using Meltano with Docker, the way to update would be to run
docker pull meltano/meltano
and to then recreate your container. If you're using
docker-compose
, it'd be even easier:
docker-compose pull meltano && docker-compose up -d meltano
m
okay thanks!
d
@matthew_trinkle Let me know if that works!
m
I was just testing this out on my local, but I think I am switching to something else in prod
but ill keep docker-compose in mind
d
@matthew_trinkle "something else" as in not-Meltano, or as in, a different installation method? 🙂 In the former case, I'd love to know why Meltano turned out to ultimately not be the right fit for your use case right now!
m
sorry, installation method
d
@matthew_trinkle Cool, let me know if you run into any more trouble 🙂
m
My company uses Convox
d
@matthew_trinkle Ah, cool! I plan to look into making it easier to deploy Meltano onto K8s (https://gitlab.com/meltano/meltano/-/issues/2014), so any feedback based on your experience would be appreciated in that issue!
m
@douwe_maan Will do!
d