Upgrade to Airbyte v2 using abtcl
Upgrade to Airbyte v2 using abtcl
1. Stop Legacy Airbyte Installation
If you are upgrading from Docker Compose, stop the existing instance to free up resources and ports:
docker compose stop
or reset Airbyte v1 installation
abctl local uninstall
Install the Airbyte CLI tool using the following command:
curl -LsfS https://get.airbyte.com | bash -
Create a file named values.yaml in your working directory. This configuration is optimized for high-performance workloads and prevents memory crashes by setting explicit resource limits. It also uses an external postgres database.
Note on Server Capacity: The resource values provided in the values.yaml below are optimized for a high-capacity server (e.g., 4 CPUs and 64GB RAM). If your server has different specifications, please adjust the CPU and memory requests and limits accordingly.
Create values.yaml with this content:
postgresql:
enabled: false
global:
env_vars:
WORKLOAD_HEARTBEAT_TIMEOUT_MS: 1800000
DD_ENABLED: "false"
DD_TRACE_ENABLED: "false"
PUBLISH_METRICS: "false"
# Optimizes Java heap and ensures the process exits on OOM to trigger a restart
CONTAINER_ORCHESTRATOR_JAVA_OPTS: "-XX:+ExitOnOutOfMemoryError -XX:MaxRAMPercentage=85.0"
database:
type: external
host: xxx
port: '5432'
name: airbyte
user: xxx
password: xxx
auth:
cookieSecureSetting: 'false'
workloads:
resources:
mainContainer:
memory:
request: 2Gi
limit: 32Gi
cpu:
request: 500m
limit: 4000m
replication:
maxConcurrentJobs: "4"
memory:
request: 2Gi
limit: 32Gi
cpu:
request: 500m
limit: 4000m
discover:
memory:
request: 2Gi
limit: 8Gi
cpu:
request: 500m
limit: 2000m
jobs:
resources:
requests:
memory: 2Gi
cpu: 1000m
limits:
cpu: 2000m
memory: 8Gi
worker:
maxCheckWorkers: "5"
maxSyncWorkers: "5"
Remote Server Migration (Custom Host & Port)
If you are deploying on a remote server and need to specify a host (HTTP/HTTPS) and avoid browser pop-ups:
For HTTP without domain:
abctl local install --no-browser --insecure-cookies --values ./values.yaml
For HTTP (Port 80) with domain:
abctl local install --no-browser --insecure-cookies --values ./values.yaml --host your.domain.com --port 80
For HTTPS with domain:
abctl local install --no-browser --values ./values.yaml --host your.domain.com --port 443
5. Set Identity Credentials
Configure your access credentials for the Airbyte UI:
# Set Email
abctl local credentials --email user@company.example
# Set Password
abctl local credentials --password your_secure_password
6. Accessing via PuTTY SSH Tunneling
If your Airbyte instance is on a remote server, use PuTTY to forward the traffic to your local machine.
- Open PuTTY and enter your server IP.
- Navigate to Connection > SSH > Tunnels.
- Source port:
8000 (the port you will use in your local browser).
- Destination:
localhost:8000 (or localhost:80 depending on your server configuration).
- Click Add, then click Open.
- Once logged into the shell, open your browser and go to:
http://localhost:8000
7. Troubleshooting
If you need to reset the installation or re-apply a modified values.yaml without losing your data, use the uninstall command with the persisted flag:
# Removes the installation but keeps your data intact
abctl local uninstall --persisted
# Reinstall with updated values
abctl local install --values ./values.yaml
For more information, visit the Official Airbyte OSS Documentation.