https://linen.dev logo
Join Slack
Powered by
Hello Everyone, I have upgraded the meltano versio...
# troubleshooting
m
Hello Everyone, I have upgraded the meltano version ( Meltano version= 2.1.0 ) and also the airflow and when i am trying to run 
meltano schedule run <pipeline-name>
it's throwing this below error. 
Usage: bin elt [OPTIONS] EXTRACTOR LOADER
Try 'bin elt --help' for help.
Error: no such option: --state-id
I have found this meltano doc where we need to update job_id with state-id, can you please tell me where i need to make these changes? @douwe_maan @edgar_ramirez_mondragon
e
Hi @monika_rajput! How is your job defined?
m
Hello @edgar_ramirez_mondragon , Couldn’t get it but i am using this command to run my pipeline 
meltano elt <extractor-name> <loader-name> --state-id=<pipeline-name>
Currently the Meltano version = 2.1.0 and airflow version = 2.1.2 , I am not able to run the 
RUN Now
button on Meltano UI and also when i am trying to run 
meltano schedule run <pipeline-name>
it's throwing 
Error: no such option: --state-id
. I am using this command to run the pipelines 
meltano elt facebook-ads-insights Facebook --state-id=facebook-ads-insights
e
I am not able to run the 
RUN Now
button on Meltano UI
Are you seeing any error messages in the UI? Note that we’re not focusing on porting features to the UI at the moment: https://github.com/meltano/meltano/issues/7075
when i am trying to run 
meltano schedule run <pipeline-name>
it’s throwing 
Error: no such option: --state-id
How is 
<pipeline-name>
defined in your project?
m
Ex: 
<pipeline-name> == facebook-ads
And i am not seeing any error in the UI.
Have a qusetion if the meltano team is planning to depreciate the UI then how we can schedule or run the multiple pipeline at once?
I am trying to schedule my pipeline and in airflow i am getting this error 
File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/orchestrate/dags/meltano.py", line 192, in create_dags
if schedule_export.get("schedules"):
AttributeError: 'list' object has no attribute 'get'
And when i run the scheduler 
meltano invoke airflow scheduler
It's throwing below error for every pipeline.
Copy code
ERROR - DAG 'meltano_recharge-subscriptions_redshift-datamill' not found in serialized_dag table
Traceback (most recent call last):
 File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/jobs/scheduler_job.py", line 1668, in _update_dag_next_dagruns
  dag = self.dagbag.get_dag(dag_model.dag_id, session=session)
 File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/utils/session.py", line 67, in wrapper
  return func(*args, **kwargs)
 File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/models/dagbag.py", line 186, in get_dag
  self._add_dag_from_db(dag_id=dag_id, session=session)
 File "/home/ubuntu/rm-projects/meltano-projects/dev-vegamour/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/models/dagbag.py", line 258, in _add_dag_from_db
  raise SerializedDagNotFound(f"DAG '{dag_id}' not found in serialized_dag table")
airflow.exceptions.SerializedDagNotFound: DAG 'meltano_recharge-subscriptions_redshift-datamill' not found in serialized_dag table
Added my meltano.py file.
meltano.py
d
@monika_rajput Can you share the 
facebook-ads-insights
schedule definition from 
meltano.yml
?
m
Hello @douwe_maan Here's the schedule define like below
Copy code
- name: facebook-ads-insights
  interval: '@hourly'
  extractor: facebook-ads-insights
  loader: Facebook
  transform: skip
  start_date: 2020-01-01 00:00:00
d
@monika_rajput Thanks! That looks OK. You mentioned you’re on Meltano 2.1.0, an you start by upgrading to the latest version (2.12.0) and trying again? This may be caused by a bug that’s already been fixed
m
Sure i will try to upgrade it and will let you know. Thank you so much.
Open in Slack
PreviousNext
Powered by