Upgrade Airbyte from Docker Compose to abctl on Ubuntu

This guide provides a step-by-step process to upgrade your Airbyte setup from Docker Compose to abctl. Additionally, you'll learn how to access your Airbyte instance via PuTTY SSH tunneling from your local pc.

0. Stop Docker Compose

To upgrade Airbyte if you're coming from Docker Compose:

Stop the existing Docker Compose instance:

docker compose stop

1. Install abctl

The easiest method for installing abctl is by using the following command:

curl -LsfS https://get.airbyte.com | bash -

This will install the abctl command-line tool for managing your Airbyte instance.

2. Run Airbyte

Once abctl is installed, you can use it to install and run Airbyte locally:

abctl local install --migrate

If you are using a system with less than 8 GB of RAM and 4 CPUs, enable low resource mode to optimize Airbyte performance on your machine:

abctl local install --migrate --low-resource-mode

3. Set Credentials

You need to set up credentials to access your Airbyte instance.

1. Set up your email address:

abctl local credentials --email [email protected]

1. Set up your password:

abctl local credentials --password new_password

4. Access Airbyte Instance via Localhost

By default, the Airbyte instance can only be accessed from localhost:8000 or DNS. If you're working on a remote server, you can use SSH tunneling with PuTTY to access it through your local browser.

Using PuTTY to Create an SSH Tunnel

• Open PuTTY:

Launch PuTTY on your Windows machine.

• Enter the Hostname or IP Address:

In the "Host Name (or IP address)" field, enter the hostname or IP address of your server. Keep the port as 22 (default for SSH).

• Go to SSH Tunnels Configuration:

In the left-hand side menu, under Category, navigate to Connection > SSH > Tunnels.

• Set Up Local Port Forwarding:

In the "Source port" field, enter the local port that you want to use (for example, 8080).

In the "Destination" field, enter the server and port you want to forward traffic to. For example: - If you want to access the web server on your remote machine (port 80 for HTTP), enter: localhost:80. - If it's an HTTPS server, use: localhost:443.

After entering the details, click the "Add" button. This adds the rule to the list above.

• Initiate the SSH Connection:

Go back to the "Session" category (top of the left-hand menu).

Save the session (if you want) and click "Open" to start the SSH connection.

You will be prompted to enter your username and password for the server.

• Access the Server in Your Browser:

Once you're connected to the server through PuTTY, open your web browser.

In the address bar, type: http://localhost:8080 (or whichever port you used in the "Source port" field).

This will forward your browser’s traffic through the SSH tunnel to the web server running on the remote machine.

5. Credentials for Airbyte Instance:

When you access the server in your browser, you will be prompted for credentials. If you haven't set them up yet, you can run the following command to configure your credentials:

abctl local credentials

After setting up credentials, you can log in to your Airbyte instance with the email and password you created.

6. Troubleshooting

If something goes wrong during the installation or configuration process, you can uninstall and reinstall Airbyte using the following command:

abctl local uninstall

Running this command will remove the Airbyte installation but will not delete your data. After uninstalling, you can reinstall Airbyte by running:

abctl local install

This will reinstall the Airbyte instance while keeping your data intact.

7. Production

For production cases, you may need to customize your Airbyte installation by using an external database, logging storage, or a secrets manager. You can define these customizations in a values.yaml file and apply them during the installation process.

• Create a values.yaml file on your local machine.

• Apply the custom values with the following command (adjust the path as necessary):

abctl local install --values ./values.yaml

For more Information visit https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart.