# R5C Boot Failure - Filesystem Corruption

## Status
❌ **Boot Failed** - SD card filesystem corrupted
- Journal superblock corrupt
- ext4 journal version incompatible with standard e2fsck (1.47.2)
- Filesystem inaccessible from Linux host

## Root Cause Analysis

**Likely trigger:** Podman homeassistant container startup during resolv.conf changes
- Heavy I/O: container image pull (~1.5GB)
- System unstable: DNS changes, network issues, package installation in progress
- Result: Filesystem corruption, unclean shutdown

## Recovery Options

### Option 1: Fresh Armbian 6.18 Install (Recommended)
1. Download latest Armbian 6.18 for R5C from armbian.com
2. Flash to new SD card
3. Re-run deployment:
   ```bash
   cd /home/andreas/vanlink
   sudo ./deploy.sh
   ```
4. Restore configs:
   - `/srv/homeassistant` (if still on USB/external)
   - DNS fix: `/etc/resolv.conf` (see HOMEASSISTANT.md)

### Option 2: Boot Armbian on R5C to Repair
1. Need working Armbian installation to use Armbian's e2fsck
2. Mount corrupted SD as secondary drive
3. Run Armbian's fsck to repair journal

### Option 3: Data Recovery
- Mount with `noload` flag (skip journal):
  ```bash
  sudo mount -t ext4 -o noload /dev/sda1 /mnt/r5c
  ```
- May allow readonly access to copy critical files

## Lessons Learned

1. **Don't install heavy services during initial boot**
   - Podman pull during unstable network = corruption risk
   - Should have stabilized DNS first, then tested, then deployed containers

2. **systemd-resolved disable needs better handling**
   - Created broken symlink situation
   - Should have been handled in deploy.sh post-processing

3. **Network instability is critical**
   - 1000+ ms latency on Starlink caused cascading failures
   - Package downloads failing during container startup = corruption

## Prevention

For future R5C deployments:

1. **Boot → Verify → Stabilize → Deploy**
   - Boot fresh Armbian
   - Verify network (DNS, latency)
   - Run deploy.sh
   - Test services individually

2. **Add resolv.conf fix to deploy.sh:**
   ```bash
   # Remove broken systemd-resolved symlink if it exists
   rm -f /etc/resolv.conf
   cat > /etc/resolv.conf << EOF
   nameserver 10.42.1.1
   nameserver 1.1.1.1
   nameserver 8.8.8.8
   EOF
   ```

3. **Delay container startup:**
   - Deploy podman/containers separately
   - Test WiFi/network/DHCP first
   - Then start resource-heavy services

## Current State

- All config/scripts committed to git
- R5C SD card: corrupted, unrecoverable
- Homeassistant setup: scripts ready, needs fresh SD card
- DNS fix: documented, ready to deploy
- **Recommendation: Flash fresh Armbian 6.18, run deploy.sh**