Document R5C boot failure and recovery plan
SD card filesystem corrupted: ext4 journal version incompatible with standard e2fsck (1.47.2 from Linux host). Filesystem inaccessible. Root cause: Podman homeassistant container startup during network instability and system reconfiguration caused unclean shutdown and filesystem corruption. Recovery: Flash fresh Armbian 6.18, run deploy.sh with fixes: - DNS resolution fix added to HOMEASSISTANT.md - resolv.conf fix should be added to deploy.sh post-processing - Deploy containers after network is stable All scripts and configs are ready and committed. Only need fresh SD card. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
+88
@@ -0,0 +1,88 @@
|
||||
# R5C Boot Failure - Filesystem Corruption
|
||||
|
||||
## Status
|
||||
❌ **Boot Failed** - SD card filesystem corrupted
|
||||
- Journal superblock corrupt
|
||||
- ext4 journal version incompatible with standard e2fsck (1.47.2)
|
||||
- Filesystem inaccessible from Linux host
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
**Likely trigger:** Podman homeassistant container startup during resolv.conf changes
|
||||
- Heavy I/O: container image pull (~1.5GB)
|
||||
- System unstable: DNS changes, network issues, package installation in progress
|
||||
- Result: Filesystem corruption, unclean shutdown
|
||||
|
||||
## Recovery Options
|
||||
|
||||
### Option 1: Fresh Armbian 6.18 Install (Recommended)
|
||||
1. Download latest Armbian 6.18 for R5C from armbian.com
|
||||
2. Flash to new SD card
|
||||
3. Re-run deployment:
|
||||
```bash
|
||||
cd /home/andreas/vanlink
|
||||
sudo ./deploy.sh
|
||||
```
|
||||
4. Restore configs:
|
||||
- `/srv/homeassistant` (if still on USB/external)
|
||||
- DNS fix: `/etc/resolv.conf` (see HOMEASSISTANT.md)
|
||||
|
||||
### Option 2: Boot Armbian on R5C to Repair
|
||||
1. Need working Armbian installation to use Armbian's e2fsck
|
||||
2. Mount corrupted SD as secondary drive
|
||||
3. Run Armbian's fsck to repair journal
|
||||
|
||||
### Option 3: Data Recovery
|
||||
- Mount with `noload` flag (skip journal):
|
||||
```bash
|
||||
sudo mount -t ext4 -o noload /dev/sda1 /mnt/r5c
|
||||
```
|
||||
- May allow readonly access to copy critical files
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
1. **Don't install heavy services during initial boot**
|
||||
- Podman pull during unstable network = corruption risk
|
||||
- Should have stabilized DNS first, then tested, then deployed containers
|
||||
|
||||
2. **systemd-resolved disable needs better handling**
|
||||
- Created broken symlink situation
|
||||
- Should have been handled in deploy.sh post-processing
|
||||
|
||||
3. **Network instability is critical**
|
||||
- 1000+ ms latency on Starlink caused cascading failures
|
||||
- Package downloads failing during container startup = corruption
|
||||
|
||||
## Prevention
|
||||
|
||||
For future R5C deployments:
|
||||
|
||||
1. **Boot → Verify → Stabilize → Deploy**
|
||||
- Boot fresh Armbian
|
||||
- Verify network (DNS, latency)
|
||||
- Run deploy.sh
|
||||
- Test services individually
|
||||
|
||||
2. **Add resolv.conf fix to deploy.sh:**
|
||||
```bash
|
||||
# Remove broken systemd-resolved symlink if it exists
|
||||
rm -f /etc/resolv.conf
|
||||
cat > /etc/resolv.conf << EOF
|
||||
nameserver 10.42.1.1
|
||||
nameserver 1.1.1.1
|
||||
nameserver 8.8.8.8
|
||||
EOF
|
||||
```
|
||||
|
||||
3. **Delay container startup:**
|
||||
- Deploy podman/containers separately
|
||||
- Test WiFi/network/DHCP first
|
||||
- Then start resource-heavy services
|
||||
|
||||
## Current State
|
||||
|
||||
- All config/scripts committed to git
|
||||
- R5C SD card: corrupted, unrecoverable
|
||||
- Homeassistant setup: scripts ready, needs fresh SD card
|
||||
- DNS fix: documented, ready to deploy
|
||||
- **Recommendation: Flash fresh Armbian 6.18, run deploy.sh**
|
||||
Reference in New Issue
Block a user