Hi! I wanna make a tap-mssql to target-postgres etl but is not working and logs says taht every it's ok, but finally y the target postgres database mssql tables not appear, can someone hel0?

Hi, @hugo_vargas. You may need to select the tables prior to running. Some taps default everything to 'selected' and others (especially databases) often expect you to 'opt in' to which tables you want replicated.

Hey @hugo_vargas, maybe this sandbox db->db meltano project helps: https://github.com/sbalnojan/meltano-example-el-db An example meltano.yml would for instance look like this:

...
plugins:
  extractors:
  - name: tap-postgres
    variant: transferwise
    pip_url: pipelinewise-tap-postgres
    config:
      host: host.docker.internal
      port: 5433
      user: admin
      password: password
      dbname: demo_source
      default_replication_method: FULL_TABLE
    select:
      - public-raw_customers.* # select all three attributes from the public schema inside the raw_customers table. 
                              # use meltano select tap-postgres --list --all to view all selectable attributes

  loaders: 
  - name: target-postgres
    variant: transferwise
    pip_url: pipelinewise-target-postgres
    config:
      host: host.docker.internal
      port: 5432
      user: admin
      password: password
      dbname: demo_target

Hey @hugo_vargas I wondered how simple this would be to setup - not 100% simple, each step takes a few minutes. Assuming you're on linux with these dependencies installed

RUN apt-get update && apt-get install -y --no-install-recommends \
    git \
    gcc \
    libpq-dev \
    openssh-client \
    python3-pip \
    python3.7-dev \
    python3.7-venv \
    && apt-get clean

# use python3.7 as default
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 1

RUN rm -rf /var/lib/apt/lists/*

RUN python3 -m pip install -U pip
RUN pip install setuptools wheel psycopg2-binary

Meltano add tap-mssql and target-postgres. We've forked the pipelinewise variants, so I only really know that these 100% work git+https://github.com/Matatika/pipelinewise-tap-mssql.git git+https://github.com/Matatika/pipelinewise-target-postgres@v0.1.0 Set your environment

TAP_MSSQL_PASSWORD=example
TAP_MSSQL_USER=example
TAP_MSSQL_HOST=example
TAP_MSSQL_DATABASE=example
TARGET_POSTGRES__TRANSFERWISE_USER=example
TARGET_POSTGRES__TRANSFERWISE_PASSWORD=example
TARGET_POSTGRES__TRANSFERWISE_HOST=example
TARGET_POSTGRES__TRANSFERWISE_PORT=5432
TARGET_POSTGRES__TRANSFERWISE_DBNAME=example
TARGET_POSTGRES__TRANSFERWISE_DEFAULT_TARGET_SCHEMA=example
DBT_TARGET=postgres_transferwise
DBT_SOURCE_SCHEMA=analytics
DBT_TARGET_SCHEMA=analytics
MELTANO_DATABASE_URI=<postgresql://example:example@example:5432/example?options=-csearch_path%3Dpublic>
MELTANO_ENVIRONMENT=dev

Run meltano to get your catalog and edit / update the tables to replicate (this doc give a good idea of how to edit your catalog.json https://github.com/Matatika/pipelinewise-tap-mssql/blob/master/README.md)

meltano invoke tap-mssql --discover > catalog.json

Run meltano

export TAP_MSSQL__CATALOG="catalog.json"
meltano run tap-mssql target-postgres--transferwise

Here is an example repo MSSQL to Postgres (meltano v.1.0 at the moment

pip install -U meltano==1.105.0

): https://github.com/MatatikaBytes/staging-MSSQL-to-Postgres-example-uhiwkvj

Thanks everyone