Hey team! Were there any changes to how the edk `r...
# meltano-plugin-development
s
Hey team! Were there any changes to how the edk
run_and_log
function takes in cli arguments ? My current architecture sends a list of arguments for the CLI (see in thread), but I'm now getting an error saying that this is no longer accepted. Is this normal?
Current function:
Copy code
def monitor_send_report(self) -> None:
        """Generates a report through and send it to a specific slack channel.

        Args:
            profiles-dir: Path to dbt profiles directory
            slack-token: Slack token for channel
            slack-channel-name: Name of the slack channel

        """
        command_name = "monitor send report"

        <http://log.info|log.info>(
            f"elementary {command_name}",
            slack_token=self.slack_token,
            slack_channel_name=self.slack_channel_name,
            dbt_profiles_dir=self.dbt_profiles_dir,
        )

        try:
            self.elementary_invoker.run_and_log(
                None,
                [
                    "monitor",
                    "send-report",
                    f"--profiles-dir={self.dbt_profiles_dir}",
                    f"--slack-token={self.slack_token}",
                    f"--slack-channel-name={self.slack_channel_name}",
                ],
            )
        except subprocess.CalledProcessError as err:
            log_subprocess_error(
                f"elementary {command_name}", err, "elementary invocation failed"
            )
            sys.exit(err.returncode)

        <http://log.info|log.info>(
            f"elementary {command_name}",
            slack_token=self.slack_token,
            slack_channel_name=self.slack_channel_name,
            dbt_profiles_dir=self.dbt_profiles_dir,
        )
Focus on argument call:
Copy code
try:
            self.elementary_invoker.run_and_log(
                None,
                [
                    "monitor",
                    "send-report",
                    f"--profiles-dir={self.dbt_profiles_dir}",
                    f"--slack-token={self.slack_token}",
                    f"--slack-channel-name={self.slack_channel_name}",
                ],
            )
Returned error:
Copy code
elementary monitor send report dbt_profiles_dir=PosixPath('/project/transform/profiles/bigquery') slack_channel_name=team-data-engineering-notifications slack_token=xoxb-59056927297-4112037323750-jiGYEGggsXwli26v3dLpyZ1J
Traceback (most recent call last):
  File "/project/.meltano/utilities/elementary/venv/lib/python3.8/site-packages/elementary_ext/main.py", line 55, in monitor_send_report
    ext.monitor_send_report()
  File "/project/.meltano/utilities/elementary/venv/lib/python3.8/site-packages/elementary_ext/extension.py", line 195, in monitor_send_report
    self.elementary_invoker.run_and_log(
  File "/project/.meltano/utilities/elementary/venv/lib/python3.8/site-packages/meltano/edk/process.py", line 172, in run_and_log
    result = asyncio.run(self._exec(sub_command, *args))
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/project/.meltano/utilities/elementary/venv/lib/python3.8/site-packages/meltano/edk/process.py", line 126, in _exec
    p = await asyncio.create_subprocess_exec(
  File "/usr/local/lib/python3.8/asyncio/subprocess.py", line 236, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 1630, in subprocess_exec
    transport = await self._make_subprocess_transport(
  File "/usr/local/lib/python3.8/asyncio/unix_events.py", line 197, in _make_subprocess_transport
    transp = _UnixSubprocessTransport(self, protocol, args, shell,
  File "/usr/local/lib/python3.8/asyncio/base_subprocess.py", line 36, in __init__
    self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
  File "/usr/local/lib/python3.8/asyncio/unix_events.py", line 789, in _start
    self._proc = subprocess.Popen(
  File "/usr/local/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1639, in _execute_child
    self.pid = _posixsubprocess.fork_exec(
TypeError: expected str, bytes or os.PathLike object, not list
monitor send report failed with uncaught exception, please report to maintainer
w
Hi @Stéphane Burwash. There have been some recent changes to the EDK: https://github.com/meltano/edk/commits/main What version of the EDK are you depending on? I'm guessing like most EDK plugins the dependency is on the git repository's
main
branch. We've tagged
v0.1.0
, and will be creating more tagged releases going forward. If your plugin depends on
main
, I'd encourage you to a more stable reference.
s
w
Yes
As for
run_and_log
, I'm not yet sure why it's now failing for you (or why it ever worked in the first place). I'm looking into it
s
Basically my goal is to introduce CLI arguments with variables to include things like project dir. I'm happy to apply any suggested fixes 😆 Would you have any suggestions on how to apply the changes to the
pyproject.toml
to point towards a tagged version as opposed to main?
w
I'll open a PR for the version change
s
Awesome thank you!
w
It pins the EDK version to
v0.1.0
, and should fix the error you encountered. I have not tested these changes - please verify that they're working as expected before merging.
s
Your fix works, thank you very much! This is awesome 😄
w
Happy to hear it 🙂
s
Is it normal tht install is now significantly longer?
w
Maybe it's a one-time wait because the dependencies have been bumped?
And so new packages have to be downloaded and cached