📊 Monitoring Your Aztec Sequencer Node

After your sequencer is running, here's how to monitor health, performance, and network participation.

Mainnet Operations

For mainnet, monitoring is critical. Downtime affects your staking rewards and can lead to penalties.

---

✅ Quick Health Check

# All-in-one status check
echo "=== Container Status ===" && docker ps | grep aztec && \
echo "=== Current Block ===" && curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","method":"node_getL2Tips","params":[],"id":67}' \
    http://localhost:8080 | jq -r ".result.proven.number" && \
echo "=== Disk Usage ===" && df -h ~/aztec-node/data

---

🔍 Container & Logs

Check Container Status

# Is container running?
docker ps | grep aztec

# Container health details
docker compose ps

# Container resource usage
docker stats aztec-sequencer --no-stream

View Logs

# Real-time logs
docker compose logs -f aztec-sequencer

# Last 100 lines
docker logs aztec-sequencer --tail 100

# Filter for errors
docker logs aztec-sequencer 2>&1 | grep -i "error\|warn\|fail"

# Filter for sync progress
docker logs aztec-sequencer 2>&1 | grep -i "sync\|block\|proven"

---

📈 Sync Status

Check Current Block

# Get your node's proven block number
curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","method":"node_getL2Tips","params":[],"id":67}' \
    http://localhost:8080 | jq -r ".result.proven.number"

Compare With Network

# Your block vs network (manual check)
MY_BLOCK=$(curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","method":"node_getL2Tips","params":[],"id":67}' \
    http://localhost:8080 | jq -r ".result.proven.number")

echo "Your node: Block $MY_BLOCK"
echo "Check network: https://aztecscan.xyz/"

Full Node Info

# Get complete node information
curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","method":"node_getNodeInfo","params":[],"id":67}' \
    http://localhost:8080 | jq

---

🔗 Validator Status (On-Chain)

Use Foundry's cast to query your validator status:

# Query AttesterView struct
# Replace with actual staking contract address and your attester address
cast call STAKING_CONTRACT_ADDRESS \
    "getAttesterView(address)" YOUR_ATTESTER_ADDRESS \
    --rpc-url https://eth-mainnet.g.alchemy.com/v2/YOUR_API_KEY

Status Values

Status	Meaning
NONE	Not registered as validator
VALIDATING	Active and validating blocks
ZOMBIE	Inactive but not exited
EXITING	Exit in progress

Monitor Effective Balance

Your effective balance must stay above the ejection threshold:

# Check in AttesterView response:
# - effectiveBalance: current stake
# - exitInfo: if exiting, shows details

---

🌐 Network Connectivity

Check P2P Ports

# Verify ports are listening
sudo netstat -tulnp | grep -E "(40400|8080)"

# Test external connectivity
nc -zv $(curl -s ifconfig.me) 40400

Get Peer ID

# Your node's P2P identity
docker logs aztec-sequencer 2>&1 | grep -i "peerId" | grep -o '"peerId":"[^"]*"' | cut -d'"' -f4 | head -n 1

Check P2P Connections

# Monitor peer connections
docker logs aztec-sequencer 2>&1 | grep -i "peer\|connection" | tail -20

---

🔄 Maintenance Operations

Restart Node (Safe)

# Graceful restart
docker compose restart aztec-sequencer

# Or full stop/start
docker compose down
docker compose up -d

# Verify restart
docker compose logs -f aztec-sequencer

Update to New Version

# Check current version
docker images | grep aztecprotocol/aztec

# Pull new version (verify on GitHub/Discord first!)
docker compose pull

# Restart with new version
docker compose down
docker compose up -d

# Verify update
docker images | grep aztecprotocol/aztec

Clean Restart (Reset Data)

# ⚠️ This will resync from scratch
docker compose down
sudo rm -rf ./data
docker compose up -d

# Monitor sync progress
docker compose logs -f aztec-sequencer

---

💾 Backup Procedures

Backup Configuration

# Create backup directory
mkdir -p ~/aztec-backups/$(date +%Y%m%d)

# Backup configs (NOT keys - those should be offline!)
cp ~/aztec-node/.env ~/aztec-backups/$(date +%Y%m%d)/
cp ~/aztec-node/docker-compose.yml ~/aztec-backups/$(date +%Y%m%d)/

# List backups
ls -la ~/aztec-backups/

Backup Keys (Critical!)

Secure Key Backup

Keys should be backed up to offline storage (encrypted USB, paper backup of mnemonic). Never store keys on cloud services or shared drives.

# Encrypt keys backup
tar -czf - ~/aztec-node/keys/ | gpg -c > aztec-keys-backup.tar.gz.gpg

# Store the encrypted file OFFLINE

---

📊 Performance Monitoring

System Resources

# Container CPU/Memory
docker stats aztec-sequencer --no-stream

# System overview
htop

# Disk usage
df -h ~/aztec-node/data
du -sh ~/aztec-node/data/*

Disk Space Management

# Check data directory growth
watch -n 60 'du -sh ~/aztec-node/data'

# Clean Docker resources
docker system prune -f

# Check for large log files
find ~/aztec-node -name "*.log" -size +100M

---

📱 Create Monitoring Script

Basic Health Check Script

nano ~/aztec-health.sh

#!/bin/bash
# Aztec Node Health Check

LOG_FILE="$HOME/aztec-health.log"
DATE=$(date '+%Y-%m-%d %H:%M:%S')

# Check container
if ! docker ps | grep -q aztec-sequencer; then
    echo "[$DATE] CRITICAL: Container not running!" >> $LOG_FILE
    # Optionally auto-restart:
    # docker compose -f $HOME/aztec-node/docker-compose.yml up -d
    exit 1
fi

# Check sync
BLOCK=$(curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","method":"node_getL2Tips","params":[],"id":67}' \
    http://localhost:8080 | jq -r ".result.proven.number" 2>/dev/null)

if [[ "$BLOCK" == "null" ]] || [[ -z "$BLOCK" ]]; then
    echo "[$DATE] WARNING: Cannot get block number" >> $LOG_FILE
else
    echo "[$DATE] OK: Block $BLOCK" >> $LOG_FILE
fi

# Check disk
DISK_PCT=$(df -h ~/aztec-node/data | tail -1 | awk '{print $5}' | sed 's/%//')
if [[ $DISK_PCT -gt 90 ]]; then
    echo "[$DATE] WARNING: Disk usage ${DISK_PCT}%" >> $LOG_FILE
fi

# Make executable
chmod +x ~/aztec-health.sh

# Add to crontab (every 10 min)
(crontab -l 2>/dev/null; echo "*/10 * * * * $HOME/aztec-health.sh") | crontab -

---

🔧 Troubleshooting

Common Issues

Issue	Solution
Container keeps restarting	Check logs: docker logs aztec-sequencer --tail 50
Sync stuck	Reset data: rm -rf ./data && docker compose up -d
RPC errors	Verify L1 RPC is working, check API limits
High memory	Restart container, check for memory leaks in logs
Port blocked	Check firewall: sudo ufw status

Test RPC Endpoints

# Test L1 Execution RPC
curl -X POST -H "Content-Type: application/json" \
    --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
    $ETHEREUM_HOSTS

# Test L1 Consensus RPC
curl -s "$L1_CONSENSUS_HOST_URLS/eth/v1/node/health"

# Test local Aztec RPC
curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","method":"node_getL2Tips","params":[],"id":67}' \
    http://localhost:8080

---

📈 Success Checklist

Daily monitoring checklist:

• Container running (docker ps | grep aztec)
• Synced to latest block (compare with AztecScan)
• No errors in recent logs
• Disk usage under 90%
• P2P ports accessible externally
• Validator status: VALIDATING (on-chain query)

---

🔗 Useful Resources

• AztecScan: https://aztecscan.xyz/ - Compare your block number
• Aztec Explorer: https://aztec.nethermind.io/ - Network overview
• Discord #node-operators: https://discord.com/invite/aztec - Support

---

🎯 Next Steps

• Advanced Monitoring: Aztec Ops Toolkit with Telegram alerts (coming soon)
• Self-hosted L1: Reduce dependency on external RPCs
• Backup Strategy: Implement automated encrypted backups

---

Keep your node healthy, keep the network secure! 🔮