Hi I am new to meltano and am just testing it at t...
# getting-started
b
Hi I am new to meltano and am just testing it at the moment. I am trying to move data from MongoDB Atlas. I have having issues connecting to Atlas it seems. My Error is CRITICAL cluster0.kefxq.mongodb.net:27017: [Errno 8] nodename nor servname provided, or not known Traceback (most recent call last): File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/bin/tap-mongodb", line 8, in <module> sys.exit(main()) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/tap_mongodb/__init__.py", line 394, in main raise exc File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/tap_mongodb/__init__.py", line 391, in main main_impl() File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/tap_mongodb/__init__.py", line 378, in main_impl client.server_info().get('version', 'unknown')) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1656, in server_info return self.admin.command("buildinfo", File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/database.py", line 654, in command with self.__client._socket_for_reads( File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1135, in _socket_for_reads server = topology.select_server(read_preference) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/topology.py", line 224, in select_server return random.choice(self.select_servers(selector, File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/topology.py", line 183, in select_servers server_descriptions = self._select_servers_loop( File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/topology.py", line 199, in _select_servers_loop raise ServerSelectionTimeoutError( pymongo.errors.ServerSelectionTimeoutError: cluster0.kefxq.mongodb.net:27017: [Errno 8] nodename nor servname provided, or not known Any idea of what I am doing wrong?
e
Hi @birkir_bjornsson! That looks like some kind of permission/connection error. Are you able to connect with a raw pymongo client?
Copy code
>>> import pymongo
>>> client = pymongo.MongoClient(<YOUR_ATLAS_URI>)
>>> client.list_databases()
b
Hi @edgar_ramirez_mondragon sorry for a really late reply have off for the last few days :S But yes I can connect with raw pymongo and get a list of the databases. Any ideas of what would be the next natural step?
e
Hi @birkir_bjornsson. I dug around the tap's repo and found https://github.com/singer-io/tap-mongodb/issues/42#issuecomment-866379009. Do let me know if that solves your problem 😄
b
Hi @edgar_ramirez_mondragon again sorry for a late reply, I have been trying all sorts of things to try to get this going. I tried your suggestion and some progress or at least a new error 😄 I have no idea what would be next, this is the config from meltano.yml - name: tap-mongodb variant: singer-io pip_url: tap-mongodb config: database: admin host: cluster0-shard-00-00.kefxq.mongodb.net user: meltano_read verify_mode: false ssl: true CRITICAL Authentication failed. Traceback (most recent call last): File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/bin/tap-mongodb", line 8, in <module> sys.exit(main()) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/tap_mongodb/__init__.py", line 394, in main raise exc File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/tap_mongodb/__init__.py", line 391, in main main_impl() File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/tap_mongodb/__init__.py", line 378, in main_impl client.server_info().get('version', 'unknown')) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1656, in server_info return self.admin.command("buildinfo", File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/database.py", line 654, in command with self.__client._socket_for_reads( File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1137, in _socket_for_reads with self._get_socket(server) as sock_info: File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 1094, in _get_socket with server.get_socket(self.__all_credentials) as sock_info: File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/pool.py", line 1011, in get_socket sock_info.check_auth(all_credentials) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/pool.py", line 682, in check_auth auth.authenticate(credentials, self) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/auth.py", line 565, in authenticate auth_func(credentials, sock_info) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/auth.py", line 540, in _authenticate_default return _authenticate_scram(credentials, sock_info, 'SCRAM-SHA-1') File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/auth.py", line 302, in _authenticate_scram res = sock_info.command(source, cmd) File "/Users/birkirb/Repos/meltano-projects/demo-project/.meltano/extractors/tap-mongodb/venv/lib/python3.8/site-packages/pymongo/pool.py", line 574…
e
@birkir_bjornsson it seems the client is failing to authenticate in order to invoke the buildinfo command. I see you don't have a password set in
meltano.yml
(which you shouldn't anyway 😄) but are you still passing it through some other mean, like the
.env
file? One way to confirm the expected config values are picked up is to run
Copy code
meltano config tap-mongodb list
b
Hi @edgar_ramirez_mondragon yes I have the password set in the .env file and the command picks that up.
Hi @edgar_ramirez_mondragon just wanted to confirm that I managed to connect and the solution was this as you suggested. I was having issues with authentication as you suggested in your last comment so I created a test user and new password and that worked. Thank you for your help and if anyone is having a similliar problem this worked for me. https://github.com/singer-io/tap-mongodb/issues/42#issuecomment-866379009
e
@birkir_bjornsson I'm glad you solved it! And thanks for sharing the solution that worked for you!