Building a Real-time Data Visualization Extension for DuckDB with Rust and Iced

[nk_Enuke]

Introduction​

This article demonstrates how to create a DuckDB extension that generates interactive charts using Rust and the Iced GUI framework. We'll build a system that executes SQL queries and displays the results as bar charts in a native window, bridging the gap between database analytics and visual representation.

Table of Contents​

1. Project Setup and Architecture
2. Creating the Rust Library with FFI
3. Building the C++ Extension Wrapper
4. Implementing the Iced Chart Viewer
5. Handling macOS Threading Constraints
6. Dynamic Data Visualization
7. Troubleshooting and Lessons Learned

Chapter 1: Project Setup and Architecture​

Overview​

Our architecture consists of three main components:

• A C++ DuckDB extension that provides SQL functions
• A Rust library exposed via FFI (Foreign Function Interface)
• A standalone Iced application for rendering charts

Directory Structure​

iced_duck/
├── src/                    # C++ extension
│   ├── quack_extension.cpp
│   └── include/
│       └── quack_extension.hpp
├── rust_hello_duck/        # Rust library
│   ├── Cargo.toml
│   ├── src/
│   │   ├── lib.rs
│   │   └── bin/
│   │       └── chart_viewer.rs
├── CMakeLists.txt
└── Makefile

Chapter 2: Creating the Rust Library with FFI​

Basic FFI Setup​

We started with a simple Rust library exposing C-compatible functions:

use std::ffi::{c_char, CString};

#[no_mangle]
pub extern "C" fn rust_hello_world() -> *const c_char {
    let message = CString::new("Hello from Duck!").unwrap();
    message.into_raw()
}

#[no_mangle]
pub extern "C" fn rust_hello_free(s: *mut c_char) {
    unsafe {
        if !s.is_null() {
            let _ = CString::from_raw(s);
        }
    }
}

Key Learnings​

1. Memory Management: Always provide a free function for allocated strings
2. Safety Attributes: Modern Rust requires explicit unsafe markers for FFI functions
3. Cross-platform Compatibility: Handle different library extensions (.dylib, .so, .dll)

Chapter 3: Building the C++ Extension Wrapper​

Dynamic Library Loading​

The C++ extension loads the Rust library at runtime:

void* rust_lib = dlopen(lib_path.c_str(), RTLD_LAZY);
if (!rust_lib) {
    printf("Warning: Could not load Rust library\n");
    return;
}

// Get function pointers
auto rust_hello_world = (rust_hello_world_fn)dlsym(rust_lib, "rust_hello_world");

Registering SQL Functions​

auto rust_hello_func = ScalarFunction("rust_hello", {}, LogicalType::VARCHAR, RustHelloScalarFun);
ExtensionUtil::RegisterFunction(instance, rust_hello_func);

Chapter 4: Implementing the Iced Chart Viewer​

Chart Application Structure​

We created a separate binary for the chart viewer to handle GUI rendering:

struct ChartApp {
    data: ChartData,
}

impl Application for ChartApp {
    type Message = Message;
    type Theme = Theme;
    type Executor = iced::executor::Default;
    type Flags = ChartData;

    fn view(&self) -> Element<Message> {
        let chart = canvas(self as &Self)
            .width(Length::Fill)
            .height(Length::Fill);

        container(
            column![
                text(&self.data.title).size(24),
                chart,
            ]
            .spacing(10)
        )
        .padding(20)
        .into()
    }
}

Canvas Drawing​

The actual chart rendering uses Iced's canvas API:

impl<Message> canvas::Program<Message> for ChartApp {
    fn draw(&self, _state: &Self::State, renderer: &iced::Renderer, 
            _theme: &Theme, bounds: Rectangle, _cursor: iced::mouse::Cursor) -> Vec<Geometry> {
        let mut frame = Frame::new(renderer, bounds.size());

        // Draw bars
        for (i, &value) in self.data.y_data.iter().enumerate() {
            let height = (value / max_value) * chart_height * 0.8;
            frame.fill_rectangle(
                Point::new(x, y),
                Size::new(bar_width, height as f32),
                Color::from_rgb(0.2, 0.6, 0.9),
            );
        }

        vec![frame.into_geometry()]
    }
}

Chapter 5: Handling macOS Threading Constraints​

The Problem​

On macOS, GUI applications must run on the main thread. Our initial approach using thread::spawn resulted in:

thread '<unnamed>' panicked at:
on macOS, `EventLoop` must be created on the main thread!

The Solution​

We launched the chart viewer as a separate process:

pub extern "C" fn rust_show_chart() -> *const c_char {
    match Command::new("rust_hello_duck/target/release/chart_viewer")
        .spawn() {
        Ok(_) => {
            let message = CString::new("Chart viewer launched").unwrap();
            message.into_raw()
        }
        Err(e) => {
            let message = CString::new(format!("Failed: {}", e)).unwrap();
            message.into_raw()
        }
    }
}

Chapter 6: Dynamic Data Visualization​

SQL Function Interface​

We created a bar_chart function that accepts SQL queries:

SELECT bar_chart(
    'SELECT category FROM sales', 
    'SELECT amount FROM sales', 
    'Sales by Category'
);

Data Transfer​

Instead of complex JSON serialization, we used a simple text format:

// Save data to file
std::ofstream file("/tmp/duckdb_chart_data.txt");
file << title_str << "\n";
file << x_data << "\n";  // comma-separated
file << y_data << "\n";  // comma-separated
file.close();

Chapter 7: Troubleshooting and Lessons Learned​

Common Issues Encountered​

1. Library Path Resolution: Always try both relative and absolute paths
2. Deadlocks in Query Execution: The context.Query() method can cause deadlocks when called from within a DuckDB function
3. Type Compatibility: Careful handling of numeric types between C++ and Rust
4. Build System Complexity: Managing both CMake and Cargo builds requires coordination

Best Practices​

1. Start Simple: Begin with basic FFI functions before adding complexity
2. Debug Incrementally: Add print statements at each stage of execution
3. Handle Errors Gracefully: Always provide fallback behavior
4. Test Cross-platform Early: Platform-specific issues can be significant

Conclusion​

This project demonstrates the power of combining different technologies:

• DuckDB for SQL processing
• Rust for safe systems programming
• Iced for modern GUI development
• FFI for language interoperability

While the integration presents challenges, particularly around threading and build systems, the result is a powerful system that can transform SQL query results into interactive visualizations.

The complete source code and additional examples are available in the project repository. Feel free to extend this foundation with additional chart types, improved error handling, or real-time data updates.

Future Enhancements​

• Support for multiple chart types (line, pie, scatter)
• Real-time data streaming
• Interactive chart features (zoom, pan, tooltips)
• Better error handling and user feedback
• Cross-platform installer generation

Happy coding, and may your data always be beautifully visualized!

Continue reading...

 

Join 𝕋𝕄𝕋 on Telegram
Channel PREVIEW: