Prerequisites

Prerequisites #

In order to contribute to Meltano, you will need the following:

1. Python 3.7+. For more details about Python requirements, refer to the “Requirements” section of the Installation instructions, that also apply here.
2. Node 8.11.0+
3. Yarn

Setting Up Your Environment #

# Clone the Meltano repo
git clone git@github.com:meltano/meltano.git

# Change directory into the Meltano project
cd meltano

# Install the Poetry tool for managing dependencies and packaging
pip3 install poetry

# Install all the dependencies
poetry install

# Install the pre-commit hook
poetry run pre-commit install --install-hooks

# Bundle the Meltano UI into the `meltano` package
make bundle

# Obtain a shell in the poetry created virtual environment
poetry shell

Meltano is now installed and available at meltano, as long as you remain in your virtual environment that you access via poetry shell! Most editor’s like VSCode or PyCharm can also be configured to detect and make use of virtualenv’s, or even be configured to use poetry directly. That allows meltano commands to work as you expect in editor based terminals, and is also typically required to enable advanced editors features (debugging, code hints, etc).

You can also run meltano outside of an activated virtualenv by prefixing all commands with poetry run , e.g. poetry run meltano....

Note that for users who are using pyenv with the virtualenv plugin you will likely not need to prefix the commands with poetry as poetry will default to using the pyenv activated virtual environment.

This means that you’re ready to start Meltano CLI development. For API and UI development, read on.

# Add to `~/.bashrc`, `~/.zshrc`, etc, depending on the shell you use:
export MELTANO_SEND_ANONYMOUS_USAGE_STATS=False

System Database #

Meltano API and CLI are both supported by a database that is managed using Alembic migrations.

Migrations for the system database are located inside the meltano.migrations module.

To create a new migration, use the alembic revision -m "message" command, then edit the created file with the necessary database changes. Make sure to implement both upgrade and downgrade, so the migration is reversible, as this will be used in migration tests in the future.

Each migration should be isolated from the meltano module, so don’t import any model definition inside a migration.

To run the migrations, use meltano upgrade inside a Meltano project.

Testing #

End-to-End Testing with Cypress #

Our end-to-end tests are currently being built with Cypress.

How to Run Tests #

1. Initialize a new meltano project with meltano init $PROJECT_NAME
2. Change directory into $PROJECT_NAME
3. Start up project with meltano ui
4. Clone Meltano repo
5. Open Meltano repo in Terminal
6. Run yarn setup
7. Run yarn test:e2e

This will kick off a Cypress application that will allow you to run tests as desired by clicking each test suite (which can be found in /src/tests/e2e/specs/*.spec.js)

Tmuxinator #

Tmuxinator is a way for you to efficiently manage multiple services when starting up Meltano.

Why Tmuxinator? #

In order to run applications, you need to run multiple sessions and have to do a lot of repetitive tasks (like sourcing your virtual environments). So we have created a way for you to start and track everything in its appropriate panes with a single command.

1. Start up Docker
2. Start Meltano API
3. Start the web app

It’s a game changer for development and it’s worth the effort!

Requirements #

1. tmux - Recommended to install with brew if on macOS
2. tmuxinator

This config uses $MELTANO_VENV to source the virtual environment from. Set it to the correct directory before running tmuxinator.

Instructions #

1. Make sure you know what directory your virtual environment is. It is normally .venv by default.
2. Run the following commands. Keep in mind that the .venv in line 2 refers to your virtual environment directory in Step #1.

cd path/to/meltano
MELTANO_VENV=.venv tmuxinator local

Resources #

• Managing Multiple Python Versions With pyenv
• Managing environments with poetry
• Tmux Cheat Sheet & Quick Reference