Cycles Top-Up Procedure

Last Updated: 2025-12-04 Alert: LowCyclesBalance, CriticalCyclesBalance Severity: Warning / Critical Response Time: < 15 minutes for critical

Overview

This runbook covers the procedure for topping up canister cycles when balances fall below thresholds.

Alert Thresholds

Level	Threshold	Response Time
Warning	< 1T cycles	< 1 hour
Critical	< 500B cycles	< 15 minutes

Symptoms

• Alert: LowCyclesBalance or CriticalCyclesBalance
• Canister calls failing with "out of cycles" error
• 502/503 errors on frontend
• Grafana dashboard shows cycles below threshold

Diagnosis

Step 1: Verify Current Balance

# Check canister status
export DFX_WARNING=-mainnet_plaintext_identity
dfx canister --network ic status <canister-id>

# Example output:
# Canister status: Running
# Balance: 234_567_890_123 Cycles

Step 2: Check ICP Wallet Balance

# Check ICP balance available for top-ups
dfx ledger --network ic balance

Step 3: Identify Affected Canisters

Check the monitoring workflow output or Grafana dashboard to see all affected canisters.

Resolution

Option A: Automated Top-Up (Recommended)

The monitoring workflow includes automatic top-up for canisters below threshold:

1. Go to GitHub Actions > monitor-metrics workflow
2. Verify auto top-up executed
3. Check workflow logs for success/failure

Option B: Manual Top-Up via dfx

# Set up identity
dfx identity use <controller-identity>
export DFX_WARNING=-mainnet_plaintext_identity

# Top up specific canister (amount in ICP)
dfx ledger --network ic top-up <canister-id> --amount 0.1

# Verify new balance
dfx canister --network ic status <canister-id>

Option C: Manual Top-Up via GitHub Actions

1. Go to Actions > monitor-metrics workflow
2. Click "Run workflow"
3. Set verbose to true for detailed output
4. Monitor workflow execution

Option D: Emergency Top-Up (Critical)

If automatic methods fail:

# Convert ICP to cycles directly
dfx cycles --network ic convert --amount 0.5

# Send cycles to canister
dfx canister --network ic deposit-cycles 500000000000 <canister-id>

Canister IDs

Canister	ID	Network
Frontend (Staging)	vlmti-wqaaa-aaaad-acoiq-cai	IC Mainnet
Add more as deployed

ICP Funding

If ICP wallet is empty:

1. Contact team lead for ICP transfer
2. Transfer ICP to dfx identity account:

   dfx ledger --network ic account-id

3. Transfer ICP from exchange or team wallet
4. Verify balance received

Post-Resolution

Step 1: Verify Resolution

# Confirm canister is running
dfx canister --network ic status <canister-id>

# Test canister functionality
dfx canister --network ic call <canister-id> <test-method>

Step 2: Monitor

• Watch Grafana dashboard for next 30 minutes
• Verify no new alerts fire
• Check that cycles balance is stable

Step 3: Document

If this was a critical incident:

1. Create incident report
2. Update cycle consumption baseline
3. Adjust monitoring thresholds if needed

Prevention

Cycle Consumption Monitoring

• Review weekly cycle consumption trends
• Set up proactive alerts before reaching warning threshold
• Consider increasing thresholds for high-traffic canisters

ICP Reserve

• Maintain minimum 1 ICP in wallet for emergencies
• Set up alerts for low ICP balance
• Document ICP funding process

Automation

The monitor-metrics workflow runs every 6 hours and auto-tops-up canisters below threshold. Verify it's enabled and working.

Escalation

Condition	Action
ICP wallet empty	Contact team lead
dfx identity issues	Contact DevOps
Top-up fails repeatedly	Contact DFINITY support
Canister still failing after top-up	Check Canister Unresponsive runbook

Related Documentation

• Monitoring Setup
• Incident Response Runbook
• DFINITY Cycles Documentation