Anyone know which container / code actually gets r...
# troubleshooting
p
Anyone know which container / code actually gets run when Airflow executes a bash operator? I'm seeing a different output when I run
docker-compose exec airflow-webserver meltano invoke tap-leaflink
vs manually running it in Airflow and worried it's not using the latest version of the tap i have published
Appears to be the
airflow-scheduler
container, but unclear why meltano isn't grabbing the latest version from the pip link even when using the
--clean
flag
e
Are you mounting a volume to the container?
p
yeah
i think that docker just reads the cache and doesn't actually run the install command - unclear how to git docker to ignore it after reading a bunch of github issues. sounds like it's an open thing, esp when using docker compose
the main issue that this causes is that you have to clean out all your docker stuff and remove the images, then build everything again which is rather painful since you have to recreate the admin users for airflow and meltano again etc. would be nice to be able to handle updates to plugins a bit more gracefully after you update it and are hosting it in pip or a git repo
or you can ssh into your docker server / the airflow scheduler container and run that command manually, but also a pain