Files

Floppyrj45 151d7461d3 docs: plan + spec finition cosma-nav (superpowers)

2026-04-26 01:21:43 +02:00

27 KiB

Raw Blame History

COSMA QC Platform Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Déployer cosma-nav comme QC viewer complet (carte GPS/USBL, viewer 3D, graphes nav) sur cosma-vm (.83), avec archivage automatique des données sur NAS .156 après chaque job.

Architecture: cosma-qc (pipeline) et cosma-nav (viewer) tournent en parallèle sur cosma-vm, derrière Caddy. Le dispatcher cosma-qc déclenche post-job : pré-décimation PLY sur ml-stack → SCP vers cosma-vm, rsync frames → NAS .156.

Tech Stack: Flask, Leaflet.js, Chart.js, Three.js, open3d, h5py, CIFS mount, Caddy

Fichiers créés / modifiés

Fichier	Action	Rôle
`/etc/fstab` (ml-stack .84)	Modify	Mount NAS .156 via CIFS
`/etc/fstab` (cosma-vm .83)	Modify	Mount NAS .156 via CIFS
`/root/cosma-nav/scripts/pre_decimate.py`	Create	Décimation PLY + SCP vers cosma-vm
`/root/cosma-nav/scripts/archive_job.sh`	Create	rsync frames+PLY brut → NAS
`/root/cosma-nav/viz/server.py`	Modify	Ajouter routes /map, /nav, /api/job/{id}/nav, /api/job/{id}/ply-status
`/root/cosma-nav/viz/templates/map.html`	Create	Page carte Leaflet
`/root/cosma-nav/viz/templates/nav.html`	Create	Page graphes nav
`/root/cosma-nav/viz/static/js/map.js`	Create	Leaflet init + tracks + USBL markers
`/root/cosma-nav/viz/static/js/nav_charts.js`	Create	Chart.js depth/altitude/RTK
`/home/cosma/cosma-qc/scripts/dispatcher.py`	Modify	Ajouter step post_job() après done
`/home/cosma/cosma-qc/app/templates/_jobs_table.html`	Modify	Bouton "QC →" par job done
`/home/cosma/cosma-qc/app/main.py`	Modify	Route /jobs/{id}/qc-redirect
`/etc/caddy/Caddyfile` (cosma-vm)	Create	Reverse proxy :80 → cosma-qc + cosma-nav
`/home/cosma/cosma-nav/`	Create	Clone repo cosma-nav sur cosma-vm

Task 1 : Montage NAS .156 sur ml-stack

Files:

Modify: /etc/fstab sur ml-stack (.84)
Create: /mnt/nas-cosma/ directory
Step 1.1 : Créer le point de montage sur ml-stack

ssh root@192.168.0.84
mkdir -p /mnt/nas-cosma

Step 1.2 : Installer cifs-utils si absent

apt-get install -y cifs-utils

Step 1.3 : Créer le fichier credentials NAS

cat > /root/.nas-credentials << 'EOF'
username=admin
password=vj']C9yJA-jYt)U
EOF
chmod 600 /root/.nas-credentials

Step 1.4 : Ajouter le mount dans /etc/fstab

echo '//192.168.0.156/Public /mnt/nas-cosma cifs credentials=/root/.nas-credentials,iocharset=utf8,uid=root,gid=root,_netdev,nofail 0 0' >> /etc/fstab

Step 1.5 : Monter et vérifier

mount /mnt/nas-cosma
df -h /mnt/nas-cosma
# Attendu : ~1.3T dispo
mkdir -p /mnt/nas-cosma/cosma-archive
ls /mnt/nas-cosma/cosma-archive

Task 2 : Montage NAS .156 sur cosma-vm

Files:

Modify: /etc/fstab sur cosma-vm (.83)
Create: /mnt/nas-cosma/ et /data/cosma/
Step 2.1 : Sur cosma-vm

ssh cosma@192.168.0.83
sudo apt-get install -y cifs-utils
sudo mkdir -p /mnt/nas-cosma /data/cosma

Step 2.2 : Credentials NAS

sudo bash -c "cat > /root/.nas-credentials << 'EOF'
username=admin
password=vj']C9yJA-jYt)U
EOF
chmod 600 /root/.nas-credentials"

Step 2.3 : fstab + mount

echo '//192.168.0.156/Public /mnt/nas-cosma cifs credentials=/root/.nas-credentials,iocharset=utf8,uid=1000,gid=1000,_netdev,nofail 0 0' | sudo tee -a /etc/fstab
sudo mount /mnt/nas-cosma
df -h /mnt/nas-cosma

Step 2.4 : Commit fstab configs

# Sur ta machine Windows, dans Nextcloud2/Projects/cosma-nav :
git add docs/
git commit -m "docs: spec et plan déploiement COSMA QC Platform"
git push origin master

Task 3 : Valider les PLY/poses existants (11 jobs)

Files:

Create: /root/cosma-nav/scripts/check_jobs.py sur ml-stack
Step 3.1 : Créer le script de validation

Sur ml-stack, créer /root/cosma-nav/scripts/check_jobs.py :

#!/usr/bin/env python3
"""Valide PLY + poses pour chaque job et affiche un rapport."""
import os, glob, numpy as np

FRAMES_DIR = "/root/cosma-qc-frames"

for job_dir in sorted(glob.glob(f"{FRAMES_DIR}/job_*")):
    job_id = os.path.basename(job_dir).replace("job_", "")
    ply = f"{job_dir}/reconstruction.ply"
    poses = f"{job_dir}/lingbot_poses.npz"

    ply_ok = os.path.exists(ply) and os.path.getsize(ply) > 1_000_000
    poses_ok = False
    n_poses = 0
    if os.path.exists(poses):
        try:
            d = np.load(poses)
            n_poses = d["poses"].shape[0]
            poses_ok = d["poses"].shape[1:] == (3, 4) and n_poses > 10
        except Exception as e:
            print(f"  job_{job_id} poses ERROR: {e}")

    ply_size = os.path.getsize(ply) / 1e9 if ply_ok else 0
    status = "✓" if (ply_ok and poses_ok) else "✗"
    print(f"{status} job_{job_id}: PLY {ply_size:.1f}GB {'OK' if ply_ok else 'MISSING'} | poses {n_poses} {'OK' if poses_ok else 'MISSING/BAD'}")

Step 3.2 : Exécuter sur ml-stack

ssh root@192.168.0.84 'cd /root/cosma-nav && python3 scripts/check_jobs.py'

Attendu : tous les jobs avec PLY > 1 GB et poses (N, 3, 4) marqués ✓.

Step 3.3 : Noter les jobs problématiques

Si un job est ✗, noter son ID pour investigation séparée (ne bloque pas la suite).

Task 4 : Script de pré-décimation PLY

Files:

Create: /root/cosma-nav/scripts/pre_decimate.py sur ml-stack
Step 4.1 : Créer le script

#!/usr/bin/env python3
"""Pré-décime un PLY (185M pts → ~200k) et SCP vers cosma-vm."""
import sys, os, argparse
import numpy as np
import open3d as o3d

def decimate(ply_path: str, out_path: str, max_pts: int = 200_000):
    pcd = o3d.io.read_point_cloud(ply_path)
    n = len(pcd.points)
    if n > max_pts:
        pts = np.asarray(pcd.points)
        vol = float(np.prod(pcd.get_max_bound() - pcd.get_min_bound()))
        vox = max((vol / max_pts) ** (1/3), 0.02)
        pcd = pcd.voxel_down_sample(vox)
    o3d.io.write_point_cloud(out_path, pcd)
    print(f"Décimé: {n} → {len(pcd.points)} pts → {out_path} ({os.path.getsize(out_path)/1e6:.1f} MB)")

if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("job_id", type=int)
    ap.add_argument("--frames-dir", default="/root/cosma-qc-frames")
    ap.add_argument("--cosma-vm", default="cosma@192.168.0.83")
    ap.add_argument("--dest-dir", default="/data/cosma")
    args = ap.parse_args()

    job_dir = f"{args.frames_dir}/job_{args.job_id}"
    ply_in  = f"{job_dir}/reconstruction.ply"
    ply_out = f"/tmp/job_{args.job_id}_decimated.ply"
    poses   = f"{job_dir}/lingbot_poses.npz"

    if not os.path.exists(ply_in):
        print(f"PLY non trouvé: {ply_in}"); sys.exit(1)

    decimate(ply_in, ply_out)

    # SCP vers cosma-vm
    os.system(f"scp {ply_out} {args.cosma_vm}:{args.dest_dir}/job_{args.job_id}_decimated.ply")
    if os.path.exists(poses):
        os.system(f"scp {poses} {args.cosma_vm}:{args.dest_dir}/job_{args.job_id}_poses.npz")
    print(f"SCP job_{args.job_id} terminé → {args.cosma_vm}:{args.dest_dir}")

Step 4.2 : Tester sur job_17 (le plus léger)

ssh root@192.168.0.84 'cd /root/cosma-nav && python3 scripts/pre_decimate.py 17'
# Attendu: Décimé: 185339364 → ~200000 pts → /tmp/job_17_decimated.ply (~7 MB)
# Puis SCP OK

Step 4.3 : Vérifier sur cosma-vm

ssh cosma@192.168.0.83 'ls -lh /data/cosma/'
# Attendu: job_17_decimated.ply ~7MB, job_17_poses.npz ~68KB

Step 4.4 : Lancer sur tous les jobs existants

ssh root@192.168.0.84 'for i in 9 11 12 13 14 15 16 17 19 21; do
  echo "=== job_$i ==="; cd /root/cosma-nav && python3 scripts/pre_decimate.py $i; done'

Task 5 : Script d'archivage vers NAS

Files:

Create: /root/cosma-nav/scripts/archive_job.sh sur ml-stack
Step 5.1 : Créer le script

cat > /root/cosma-nav/scripts/archive_job.sh << 'EOF'
#!/bin/bash
# Archive frames + PLY brut d'un job vers NAS .156
JOB_ID=$1
FRAMES_DIR=/root/cosma-qc-frames
NAS_DIR=/mnt/nas-cosma/cosma-archive

if [ -z "$JOB_ID" ]; then echo "Usage: $0 <job_id>"; exit 1; fi

SRC="$FRAMES_DIR/job_$JOB_ID"
DST="$NAS_DIR/job_$JOB_ID"

if [ ! -d "$SRC" ]; then echo "Job dir not found: $SRC"; exit 1; fi

mkdir -p "$DST"
echo "[$(date)] Archivage job_$JOB_ID vers NAS..."

# rsync frames JPG (exclut PLY et poses)
rsync -av --progress "$SRC/" "$DST/" \
  --include="frame_*.jpg" \
  --include="reconstruction.ply" \
  --include="lingbot_poses.npz" \
  --exclude="*" \
  2>&1 | tail -5

echo "[$(date)] Archivage job_$JOB_ID terminé: $DST"
EOF
chmod +x /root/cosma-nav/scripts/archive_job.sh

Step 5.2 : Tester sur job_17

ssh root@192.168.0.84 '/root/cosma-nav/scripts/archive_job.sh 17'
# Attendu: rsync OK, fichiers dans /mnt/nas-cosma/cosma-archive/job_17/
ls /mnt/nas-cosma/cosma-archive/job_17/ | head -5

Step 5.3 : Archiver tous les jobs existants (background)

ssh root@192.168.0.84 'for i in 9 11 12 13 14 15 16 17 19 21; do
  nohup /root/cosma-nav/scripts/archive_job.sh $i >> /tmp/archive.log 2>&1 &
done; echo "Archivages lancés en background"'

Task 6 : Déployer cosma-nav sur cosma-vm

Files:

Create: /home/cosma/cosma-nav/ sur cosma-vm (clone git)
Create: /home/cosma/cosma-nav/.venv/
Step 6.1 : Cloner le repo

ssh cosma@192.168.0.83
cd /home/cosma
git clone http://floppyrj45:67e1615fcfd06cf2df7872ac25e824f3afdb2bc1@192.168.0.82:3000/floppyrj45/cosma-nav.git
cd cosma-nav

Step 6.2 : Créer le venv et installer les dépendances

python3 -m venv .venv
source .venv/bin/activate
pip install flask h5py numpy open3d
# open3d peut prendre 2-3 min
python3 -c "import open3d; print(open3d.__version__)"
# Attendu: 0.19.0 (ou supérieur)

Step 6.3 : Créer le dossier de données

sudo mkdir -p /data/cosma
sudo chown cosma:cosma /data/cosma
ls /data/cosma/
# Attendu: job_*_decimated.ply et job_*_poses.npz déjà là (Task 4)

Step 6.4 : Tester le viewer existant

cd /home/cosma/cosma-nav
source .venv/bin/activate
python3 viz/server.py --trajectory /data/cosma/job_17_decimated.ply --port 5051 &
sleep 3 && curl -s -o /dev/null -w "%{http_code}" http://localhost:5051/trajectory
# Attendu: 200
kill %1

Task 7 : Étendre cosma-nav — routes map et nav

Files:

Modify: /home/cosma/cosma-nav/viz/server.py
Create: /home/cosma/cosma-nav/viz/templates/map.html
Create: /home/cosma/cosma-nav/viz/templates/nav.html
Create: /home/cosma/cosma-nav/viz/static/js/map.js
Create: /home/cosma/cosma-nav/viz/static/js/nav_charts.js
Step 7.1 : Ajouter les routes dans server.py

Ajouter après les routes existantes dans viz/server.py :

import json
from pathlib import Path

DATA_DIR = Path(os.environ.get("COSMA_DATA_DIR", "/data/cosma"))

def _load_nav_data(job_id: int) -> dict:
    """Charge poses + PLY décimé pour un job."""
    out = {"job_id": job_id}
    poses_path = DATA_DIR / f"job_{job_id}_poses.npz"
    ply_path   = DATA_DIR / f"job_{job_id}_decimated.ply"
    if poses_path.exists():
        d = np.load(str(poses_path))
        poses = d["poses"]           # (N, 3, 4)
        t_ns  = d.get("timestamps_ns", np.zeros(len(poses), dtype=np.int64))
        # Positions caméra : colonne de translation
        xyz = poses[:, :3, 3]
        out["track"] = {"x": xyz[:, 0].tolist(), "y": xyz[:, 1].tolist(), "z": xyz[:, 2].tolist()}
        out["n_poses"] = len(poses)
    out["ply_ready"] = ply_path.exists()
    out["ply_path"]  = str(ply_path) if ply_path.exists() else None
    return out

@app.route("/map")
def map_view():
    return render_template("map.html")

@app.route("/nav")
def nav_view():
    return render_template("nav.html")

@app.route("/api/jobs")
def api_jobs():
    """Liste les jobs disponibles (PLY décimé présent)."""
    jobs = []
    for p in sorted(DATA_DIR.glob("job_*_decimated.ply")):
        jid = int(p.stem.split("_")[1])
        jobs.append({"id": jid, "ply": str(p), "poses": str(DATA_DIR / f"job_{jid}_poses.npz")})
    return jsonify(jobs)

@app.route("/api/job/<int:job_id>/nav")
def api_job_nav(job_id: int):
    return jsonify(_load_nav_data(job_id))

@app.route("/api/job/<int:job_id>/ply")
def api_job_ply(job_id: int):
    """Sert le PLY décimé sous forme x/y/z arrays pour Three.js."""
    ply_path = DATA_DIR / f"job_{job_id}_decimated.ply"
    if not ply_path.exists():
        return jsonify({"error": "PLY non disponible"}), 404
    import open3d as o3d
    pcd = o3d.io.read_point_cloud(str(ply_path))
    pts = np.asarray(pcd.points)
    return jsonify({"x": pts[:,0].tolist(), "y": pts[:,1].tolist(), "z": pts[:,2].tolist(), "n": len(pts)})

Step 7.2 : Créer map.html

Créer /home/cosma/cosma-nav/viz/templates/map.html :

<!doctype html>
<html lang="fr">
<head>
  <meta charset="utf-8">
  <title>COSMA NAV — Carte</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css">
  <script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js"></script>
  <link rel="stylesheet" href="/static/style.css">
  <style>
    #map { height: calc(100vh - 60px); }
    #job-select { margin: 8px; padding: 4px 8px; background: #1a2a1a; color: #ccc; border: 1px solid #4ade80; border-radius: 3px; }
  </style>
</head>
<body>
<header>
  <h1>COSMA NAV — Carte GPS</h1>
  <select id="job-select"><option value="">-- sélectionner job --</option></select>
  <a href="/trajectory">3D →</a>
  <a href="/nav">Graphes →</a>
</header>
<div id="map"></div>
<script src="/static/js/map.js"></script>
</body>
</html>

Step 7.3 : Créer nav.html

Créer /home/cosma/cosma-nav/viz/templates/nav.html :

<!doctype html>
<html lang="fr">
<head>
  <meta charset="utf-8">
  <title>COSMA NAV — Données nav</title>
  <link rel="stylesheet" href="/static/style.css">
  <script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.0/dist/chart.umd.min.js"></script>
  <style>
    .charts { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; padding: 16px; }
    canvas { background: #0d1f0d; border-radius: 6px; }
    #job-select { margin: 8px; padding: 4px 8px; background: #1a2a1a; color: #ccc; border: 1px solid #4ade80; border-radius: 3px; }
  </style>
</head>
<body>
<header>
  <h1>COSMA NAV — Données navigation</h1>
  <select id="job-select"><option value="">-- sélectionner job --</option></select>
  <a href="/map">Carte →</a>
  <a href="/trajectory">3D →</a>
</header>
<div class="charts">
  <canvas id="chart-track-xy" height="300"></canvas>
  <canvas id="chart-depth" height="300"></canvas>
</div>
<script src="/static/js/nav_charts.js"></script>
</body>
</html>

Step 7.4 : Créer map.js

Créer /home/cosma/cosma-nav/viz/static/js/map.js :

const map = L.map('map').setView([43.17, 5.70], 13);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
  attribution: '© OSM contributors', maxZoom: 19
}).addTo(map);

let trackLayer = null;

async function loadJobs() {
  const res = await fetch('/api/jobs');
  const jobs = await res.json();
  const sel = document.getElementById('job-select');
  jobs.forEach(j => {
    const opt = document.createElement('option');
    opt.value = j.id; opt.textContent = `job_${j.id}`;
    sel.appendChild(opt);
  });
  if (jobs.length > 0) { sel.value = jobs[0].id; loadJob(jobs[0].id); }
}

async function loadJob(jobId) {
  const res = await fetch(`/api/job/${jobId}/nav`);
  const d = await res.json();
  if (trackLayer) map.removeLayer(trackLayer);
  if (!d.track) return;
  // Coordonnées locales lingbot → affichage relatif (pas de géoréf)
  // Si trajectory_world.h5 dispo → utiliser easting/northing
  const pts = d.track.x.map((x, i) => [43.17 + d.track.y[i] * 0.00001, 5.70 + x * 0.00001]);
  trackLayer = L.polyline(pts, { color: '#4ade80', weight: 2 }).addTo(map);
  map.fitBounds(trackLayer.getBounds());
}

document.getElementById('job-select').addEventListener('change', e => {
  if (e.target.value) loadJob(parseInt(e.target.value));
});

loadJobs();

Note: Les coordonnées lingbot sont locales (non géoréférencées). La carte affiche le track relatif jusqu'à ce que fuse_trajectory.py produise un trajectory_world.h5 avec easting/northing réels (logs La Ciotat). Une fois les données USBL disponibles, mettre à jour loadJob() pour appeler /api/job/{id}/trajectory avec les vraies coordonnées UTM.

Step 7.5 : Créer nav_charts.js

Créer /home/cosma/cosma-nav/viz/static/js/nav_charts.js :

let chartXY = null, chartDepth = null;

function initCharts() {
  chartXY = new Chart(document.getElementById('chart-track-xy'), {
    type: 'scatter',
    data: { datasets: [{ label: 'Track XY lingbot', data: [], borderColor: '#4ade80', pointRadius: 1 }] },
    options: { responsive: true, plugins: { title: { display: true, text: 'Track XY (m, local)', color: '#ccc' } },
      scales: { x: { ticks: { color: '#888' } }, y: { ticks: { color: '#888' } } } }
  });
  chartDepth = new Chart(document.getElementById('chart-depth'), {
    type: 'line',
    data: { datasets: [{ label: 'Z lingbot (profondeur approx)', data: [], borderColor: '#60a5fa', pointRadius: 0, borderWidth: 1 }] },
    options: { responsive: true, plugins: { title: { display: true, text: 'Z caméra (m, local)', color: '#ccc' } },
      scales: { x: { ticks: { color: '#888' } }, y: { reverse: true, ticks: { color: '#888' } } } }
  });
}

async function loadJobs() {
  const res = await fetch('/api/jobs');
  const jobs = await res.json();
  const sel = document.getElementById('job-select');
  jobs.forEach(j => {
    const opt = document.createElement('option');
    opt.value = j.id; opt.textContent = `job_${j.id} (${j.id})`;
    sel.appendChild(opt);
  });
  if (jobs.length > 0) { sel.value = jobs[0].id; loadJob(jobs[0].id); }
}

async function loadJob(jobId) {
  const res = await fetch(`/api/job/${jobId}/nav`);
  const d = await res.json();
  if (!d.track) return;
  const n = d.track.x.length;
  chartXY.data.datasets[0].data = d.track.x.map((x, i) => ({ x, y: d.track.y[i] }));
  chartXY.update();
  chartDepth.data.datasets[0].data = d.track.z.map((z, i) => ({ x: i, y: z }));
  chartDepth.update();
}

document.getElementById('job-select').addEventListener('change', e => {
  if (e.target.value) loadJob(parseInt(e.target.value));
});

initCharts();
loadJobs();

Step 7.6 : Tester les nouvelles routes

ssh cosma@192.168.0.83 'cd /home/cosma/cosma-nav && source .venv/bin/activate && \
  COSMA_DATA_DIR=/data/cosma python3 viz/server.py --port 5051 &'
sleep 3
curl -s http://192.168.0.83:5051/api/jobs | python3 -c "import sys,json; print(json.load(sys.stdin))"
curl -s http://192.168.0.83:5051/api/job/17/nav | python3 -c "import sys,json; d=json.load(sys.stdin); print('poses:', d.get('n_poses'), 'ply:', d.get('ply_ready'))"
# Attendu: n_poses 1217, ply_ready True

Step 7.7 : Commit

cd /root/cosma-nav  # sur ml-stack
git add viz/server.py viz/templates/map.html viz/templates/nav.html \
        viz/static/js/map.js viz/static/js/nav_charts.js \
        scripts/pre_decimate.py scripts/archive_job.sh scripts/check_jobs.py
git -c user.email="floppyrj45@gitea" -c user.name="Floppyrj45" commit -m "feat: routes map/nav + scripts pré-décimation + archivage NAS"
TOKEN=67e1615fcfd06cf2df7872ac25e824f3afdb2bc1
git push http://floppyrj45:${TOKEN}@192.168.0.82:3000/floppyrj45/cosma-nav.git master

Task 8 : Systemd service cosma-nav sur cosma-vm

Files:

Create: /etc/systemd/system/cosma-nav.service
Step 8.1 : Créer le service

ssh cosma@192.168.0.83
sudo tee /etc/systemd/system/cosma-nav.service << 'EOF'
[Unit]
Description=COSMA NAV QC Viewer
After=network.target

[Service]
User=cosma
WorkingDirectory=/home/cosma/cosma-nav
Environment=COSMA_DATA_DIR=/data/cosma
ExecStart=/home/cosma/cosma-nav/.venv/bin/python3 viz/server.py --port 5051
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable cosma-nav
sudo systemctl start cosma-nav
sudo systemctl status cosma-nav --no-pager

Step 8.2 : Vérifier

curl -s -o /dev/null -w "%{http_code}" http://192.168.0.83:5051/map
# Attendu: 200

Task 9 : Caddy reverse proxy

Files:

Create: /etc/caddy/Caddyfile sur cosma-vm
Step 9.1 : Installer Caddy

ssh cosma@192.168.0.83
sudo apt-get install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt-get update && sudo apt-get install -y caddy

Step 9.2 : Configurer le Caddyfile

sudo tee /etc/caddy/Caddyfile << 'EOF'
:80 {
    # cosma-qc pipeline dashboard (racine)
    handle /nav/* {
        reverse_proxy localhost:5051
    }
    handle /nav {
        reverse_proxy localhost:5051
    }
    handle {
        reverse_proxy localhost:3849
    }
}
EOF

Step 9.3 : Démarrer Caddy

sudo systemctl reload caddy || sudo systemctl start caddy
sudo systemctl enable caddy
sleep 2
curl -s -o /dev/null -w "/ → %{http_code}\n" http://192.168.0.83/
curl -s -o /dev/null -w "/nav → %{http_code}\n" http://192.168.0.83/nav
# Attendu: / → 200, /nav → 200 (redirect vers /nav/)

Note: Le Caddy écoute sur :80. cosma-qc reste aussi accessible directement sur :3849. cosma-nav reste aussi sur :5051 pour le viewer 3D.

Task 10 : Bouton "QC →" dans cosma-qc dashboard

Files:

Modify: /home/cosma/cosma-qc/app/templates/_jobs_table.html
Modify: /home/cosma/cosma-qc/app/main.py
Step 10.1 : Ajouter la route redirect dans main.py

Ajouter dans /home/cosma/cosma-qc/app/main.py après les routes existantes :

from fastapi.responses import RedirectResponse

@app.get("/jobs/{job_id}/qc")
async def qc_redirect(job_id: int):
    """Redirige vers cosma-nav viewer pour ce job."""
    return RedirectResponse(url=f"http://192.168.0.83:5051/trajectory?job={job_id}", status_code=302)

Step 10.2 : Modifier le template _jobs_table.html

Dans le bloc {% elif j.status == 'done' %} de col-progress, remplacer :

{% if j.viser_url %}
  <a class="btn-viser" href="{{ j.viser_url }}" target="_blank" rel="noopener">viser ↗</a>
{% else %}
  <button class="viewer-btn" data-view-url="jobs/{{ j.id }}/view" title="Charger PLY dans viser">PLY ↗</button>
{% endif %}
{% if j.ply_url %}
  <a class="btn-ply-dl" href="{{ j.ply_url }}" download>PLY ↓</a>
{% endif %}

par :

<a class="btn-qc" href="jobs/{{ j.id }}/qc" target="_blank" title="Ouvrir QC viewer">QC ↗</a>
<button class="viewer-btn" data-view-url="jobs/{{ j.id }}/view" title="PLY viser standalone">PLY ↗</button>
{% if j.ply_url %}
  <a class="btn-ply-dl" href="{{ j.ply_url }}" download>PLY ↓</a>
{% endif %}

Step 10.3 : Ajouter le style btn-qc dans style.css

ssh cosma@192.168.0.83 "cat >> /home/cosma/cosma-qc/app/static/style.css << 'EOF'

.btn-qc { background: #1a2a3a; color: #60a5fa; border: 1px solid #60a5fa; border-radius: 3px; padding: 2px 8px; text-decoration: none; font-size: 0.8rem; }
.btn-qc:hover { background: #60a5fa; color: #0a1020; }
.viewer-btn { background: #1a3a2a; color: #4ade80; border: 1px solid #4ade80; border-radius: 3px; padding: 2px 8px; cursor: pointer; font-size: 0.8rem; }
.viewer-btn:hover { background: #4ade80; color: #0a1a10; }
EOF"

Step 10.4 : Rebuild Docker cosma-qc

ssh cosma@192.168.0.83 'cd /home/cosma/cosma-qc && docker compose build --no-cache && docker compose up -d'
sleep 5
curl -s -o /dev/null -w "%{http_code}" http://192.168.0.83:3849/
# Attendu: 200

Task 11 : Post-job hook dans le dispatcher cosma-qc

Files:

Modify: /home/cosma/cosma-qc/scripts/dispatcher.py
Step 11.1 : Ajouter la fonction post_job()

Dans /home/cosma/cosma-qc/scripts/dispatcher.py, ajouter après _maybe_create_per_auv_stitch() :

def post_job_sync(job_id: int, worker: dict, frames_dir: str) -> None:
    """Pré-décime PLY + SCP vers cosma-vm + archivage NAS (non-bloquant)."""
    import threading

    def _run():
        try:
            # 1. Pré-décimation + SCP vers cosma-vm
            rc, out, err = ssh(
                worker["ssh_alias"],
                f"cd /root/cosma-nav && python3 scripts/pre_decimate.py {job_id}",
                timeout=600
            )
            if rc != 0:
                print(f"[post_job] pre_decimate job_{job_id} failed: {err[:200]}")
                return
            print(f"[post_job] pre_decimate job_{job_id} OK")

            # 2. Archivage NAS (background)
            ssh(
                worker["ssh_alias"],
                f"nohup /root/cosma-nav/scripts/archive_job.sh {job_id} >> /tmp/archive_{job_id}.log 2>&1 &",
                timeout=10
            )
            print(f"[post_job] archive job_{job_id} lancé en background")
        except Exception as e:
            print(f"[post_job] ERROR job_{job_id}: {e}")

    threading.Thread(target=_run, daemon=True).start()

Step 11.2 : Appeler post_job_sync() après chaque job done

Dans dispatcher.py, trouver le bloc qui marque un job comme done (après set_status(job["id"], status="done", ...)). Ajouter :

# Après set_status done :
post_job_sync(job["id"], worker, frames_dir)
_maybe_create_per_auv_stitch(job["id"])

Step 11.3 : Redémarrer le dispatcher

ssh cosma@192.168.0.83 'sudo systemctl restart cosma-qc-dispatcher && \
  sudo systemctl status cosma-qc-dispatcher --no-pager | head -5'
# Attendu: active (running)

Task 12 : Vérification finale end-to-end

Step 12.1 : Dashboard cosma-qc accessible

http://192.168.0.83/        → cosma-qc pipeline dashboard (via Caddy)
http://192.168.0.83:3849/   → cosma-qc direct

Step 12.2 : QC viewer accessible

http://192.168.0.83/nav/trajectory  → Three.js viewer
http://192.168.0.83/nav/map         → Carte Leaflet
http://192.168.0.83/nav/nav         → Graphes nav
http://192.168.0.83:5051/trajectory → cosma-nav direct

Step 12.3 : Cliquer bouton "QC →" dans le dashboard

Ouvrir http://192.168.0.83/, choisir un job done, cliquer "QC ↗" → doit ouvrir le viewer Three.js sur ce job dans un nouvel onglet.

Step 12.4 : Vérifier archivage NAS

ssh root@192.168.0.84 'ls -lh /mnt/nas-cosma/cosma-archive/ | head -15'
# Attendu: dossiers job_N avec frames + PLY brut

Step 12.5 : Commit final cosma-nav

cd /root/cosma-nav
git add -A
git -c user.email="floppyrj45@gitea" -c user.name="Floppyrj45" \
  commit -m "feat: déploiement cosma-vm — viewer map/nav/3D + post-job sync NAS"
TOKEN=67e1615fcfd06cf2df7872ac25e824f3afdb2bc1
git push http://floppyrj45:${TOKEN}@192.168.0.82:3000/floppyrj45/cosma-nav.git master

27 KiB Raw Blame History

COSMA QC Platform Implementation Plan

Fichiers créés / modifiés

Task 1 : Montage NAS .156 sur ml-stack

Task 2 : Montage NAS .156 sur cosma-vm

Task 3 : Valider les PLY/poses existants (11 jobs)

Task 4 : Script de pré-décimation PLY

Task 5 : Script d'archivage vers NAS

Task 6 : Déployer cosma-nav sur cosma-vm

Task 7 : Étendre cosma-nav — routes map et nav

Task 8 : Systemd service cosma-nav sur cosma-vm

Task 9 : Caddy reverse proxy

Task 10 : Bouton "QC →" dans cosma-qc dashboard

Task 11 : Post-job hook dans le dispatcher cosma-qc

Task 12 : Vérification finale end-to-end

27 KiB

Raw Blame History