nexus-scribe-hailo

Hailo 10-H NPU integration and driver interface.

Important Notice

This crate provides a stub implementation for development without real Hailo hardware. For actual transcription:

  • Use SimpleTranscriptionEngine from nexus-scribe-transcription
  • This works on any platform via whisper-rs (whisper.cpp bindings)

When to Use

Only use this crate if you have:

  1. Real Hailo 10-H NPU hardware installed
  2. Hailo drivers and runtime properly configured
  3. Device accessible at /dev/hailo0

Configuration 

use nexus_scribe_hailo::HailoConfig;

let config = HailoConfig {
    device_id: 0,
    batch_size: 1,
    timeout_ms: 5000,
    enable_power_management: true,
};

Device Management 

use nexus_scribe_hailo::HailoDevice;

let device = HailoDevice::new(config)?;

// Health check
if device.health_check()? {
    println!("NPU is healthy");
}

// Device info
let info = device.device_info()?;
println!("Firmware: {}", info.firmware_version);
println!("Temperature: {}C", info.temperature_celsius);

Model Loading 

use std::path::Path;

let model = device.load_model(Path::new("model.hef")).await?;

Inference 

// Single inference
let output = device.infer(&model, &input_features).await?;

// Batched inference
let outputs = device.infer_batch(&model, inputs).await?;

Model Management 

// Unload model to free NPU resources
device.unload_model(&model)?;

// Reset device
device.reset()?;

Performance Monitoring 

let stats = device.performance_stats();

println!("Inferences: {}", stats.total_inferences);
println!("Avg latency: {:.2}ms", stats.avg_latency_ms);
println!("P99 latency: {:.2}ms", stats.p99_latency_ms);

Device Info 

pub struct HailoDeviceInfo {
    pub device_id: u32,
    pub firmware_version: String,
    pub architecture: String,
    pub core_clock_mhz: u32,
    pub memory_size_mb: u32,
    pub temperature_celsius: f32,
    pub power_consumption_watts: f32,
}

HailoModel 

pub struct HailoModel {
    pub name: String,
    pub input_shape: Vec<usize>,
    pub output_shape: Vec<usize>,
    pub loaded: bool,
}

Current Limitations

Without real Hailo hardware:

  • run_inference() returns vec![0.0; size]
  • get_info() returns hardcoded values
  • load_model() does not load HEF files

For transcription without Hailo:

// Option 1: SimpleTranscriptionEngine (recommended)
use nexus_scribe_transcription::SimpleTranscriptionEngine;

let engine = SimpleTranscriptionEngine::new(
    "/path/to/ggml-base.bin",
    Some("en".to_string()),
)?;

// Option 2: CPU-only TranscriptionEngine
use nexus_scribe_transcription::TranscriptionEngine;

let engine = TranscriptionEngine::new_cpu_only(config, model_path)?;

Usage 

[dependencies]
nexus-scribe-hailo = { path = "../nexus-scribe-hailo" }

# Or in transcription crate:
nexus-scribe-transcription = {
    path = "../nexus-scribe-transcription",
    features = ["hailo"]
}

Hardware Requirements

  • Hailo 10-H NPU (40 TOPS)
  • Raspberry Pi 5 or compatible host
  • HailoRT runtime installed
  • Device permissions for /dev/hailo0