commit 03074280f75e2fe223bc12db75aa6f7775ef815f
parent 2f40f4b6d4b6bf56459e910fdd52f3ff10b2be66
Author: Erik Loualiche <[email protected]>
Date: Fri, 6 Feb 2026 10:57:09 -0600
Update documentation for nickel_eval_native
- Add nickel_eval_native to public API docs
- Update FFI guide with two-function comparison
- Document type mapping from Nickel to Julia
- Update TODO with completed status
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Diffstat:
3 files changed, 81 insertions(+), 139 deletions(-)
diff --git a/TODO.md b/TODO.md
@@ -1,131 +1,61 @@
-# NickelEval.jl - Next Session TODOs
+# NickelEval.jl - Status & TODOs
-## Goal
+## Completed
-**Parse Nickel directly into Julia native types** via FFI binary protocol.
+### Core Features
+- **Subprocess evaluation** - `nickel_eval`, `nickel_eval_file` via CLI
+- **FFI native evaluation** - `nickel_eval_native` via Rust binary protocol
+- **Type preservation** - Int64 vs Float64 from Nickel types directly
+- **Typed evaluation** - `nickel_eval(code, T)` for Dict, NamedTuple, etc.
+- **Export functions** - JSON, TOML, YAML via subprocess
+- **Documentation** - VitePress site at https://louloulibs.github.io/NickelEval/
-The Rust FFI encodes Nickel values with type tags:
-- `TYPE_INT (2)` → `Int64`
-- `TYPE_FLOAT (3)` → `Float64`
-- `TYPE_STRING (4)` → `String`
-- `TYPE_BOOL (1)` → `Bool`
-- `TYPE_NULL (0)` → `nothing`
-- `TYPE_ARRAY (5)` → `Vector`
-- `TYPE_RECORD (6)` → `Dict{String, Any}` or `NamedTuple`
-
-This preserves Nickel's type semantics directly—no JSON round-trip.
-
----
-
-## Current State
-
-**Done:**
-- Rust FFI library (`rust/nickel-jl/src/lib.rs`) - encodes to binary protocol
-- 33 Rust tests passing
-- Julia FFI skeleton (`src/ffi.jl`) - calls Rust, but only JSON path implemented
-
-**TODO:**
-- Julia binary decoder (`decode_native`)
-- Build Rust library
-- Test end-to-end
+### Test Coverage
+- 94 tests passing (53 subprocess + 41 FFI)
---
-## Next Session Tasks
+## Next Steps
-### 1. Add Julia Binary Decoder
+### 1. Cross-Platform FFI Distribution
+Currently FFI requires local Rust build. Options:
+- **BinaryBuilder.jl** - Create `NickelEval_jll` for automatic binary distribution
+- Support Linux (x86_64, aarch64), macOS (x86_64, aarch64), Windows
-In `src/ffi.jl`, add:
+### 2. CI FFI Testing
+Update CI workflow to build Rust library and run FFI tests.
+### 3. Performance Benchmarks
+Compare subprocess vs FFI:
```julia
-const TYPE_NULL = 0x00
-const TYPE_BOOL = 0x01
-const TYPE_INT = 0x02
-const TYPE_FLOAT = 0x03
-const TYPE_STRING = 0x04
-const TYPE_ARRAY = 0x05
-const TYPE_RECORD = 0x06
-
-function decode_native(data::Vector{UInt8})
- io = IOBuffer(data)
- return _decode_value(io)
-end
-
-function _decode_value(io::IOBuffer)
- tag = read(io, UInt8)
- if tag == TYPE_NULL
- return nothing
- elseif tag == TYPE_BOOL
- return read(io, UInt8) != 0
- elseif tag == TYPE_INT
- return read(io, Int64)
- elseif tag == TYPE_FLOAT
- return read(io, Float64)
- elseif tag == TYPE_STRING
- len = read(io, UInt32)
- return String(read(io, len))
- elseif tag == TYPE_ARRAY
- len = read(io, UInt32)
- return [_decode_value(io) for _ in 1:len]
- elseif tag == TYPE_RECORD
- len = read(io, UInt32)
- dict = Dict{String, Any}()
- for _ in 1:len
- key_len = read(io, UInt32)
- key = String(read(io, key_len))
- dict[key] = _decode_value(io)
- end
- return dict
- else
- error("Unknown type tag: $tag")
- end
-end
+using BenchmarkTools
+@benchmark nickel_eval("{ x = 1 }") # subprocess
+@benchmark nickel_eval_native("{ x = 1 }") # FFI
```
-### 2. Add `nickel_eval_native_ffi`
-
-```julia
-function nickel_eval_native_ffi(code::String)
- if !FFI_AVAILABLE
- error("FFI not available. Build with: cd rust/nickel-jl && cargo build --release")
- end
-
- buffer = ccall((:nickel_eval_native, LIB_PATH),
- NativeBuffer, (Cstring,), code)
-
- if buffer.data == C_NULL
- error_ptr = ccall((:nickel_get_error, LIB_PATH), Ptr{Cchar}, ())
- throw(NickelError(unsafe_string(error_ptr)))
- end
+---
- data = unsafe_wrap(Array, buffer.data, buffer.len; own=false)
- result = decode_native(copy(data))
+## Nice-to-Have
- ccall((:nickel_free_buffer, LIB_PATH), Cvoid, (NativeBuffer,), buffer)
+- File watching for config reload
+- Multi-file evaluation with imports
+- NamedTuple output option for records
+- Nickel contracts integration
- return result
-end
-```
+---
-### 3. Build & Test
+## Quick Reference
+**Build FFI locally:**
```bash
cd rust/nickel-jl && cargo build --release
-mkdir -p ../../deps
-cp target/release/libnickel_jl.dylib ../../deps/ # macOS
+cp target/release/libnickel_jl.dylib ../deps/ # macOS
+cp target/release/libnickel_jl.so ../deps/ # Linux
```
+**Test FFI:**
```julia
using NickelEval
-nickel_eval_native_ffi("42") # => 42::Int64
-nickel_eval_native_ffi("3.14") # => 3.14::Float64
-nickel_eval_native_ffi("{ x = 1 }") # => Dict("x" => 1)
+check_ffi_available() # true if library found
+nickel_eval_native("42") # => 42::Int64
```
-
----
-
-## Later (nice-to-have)
-
-- Cross-platform distribution via BinaryBuilder.jl
-- TOML/YAML export (already works via subprocess)
-- File watching
diff --git a/docs/src/lib/public.md b/docs/src/lib/public.md
@@ -23,6 +23,7 @@ nickel_to_yaml
```@docs
check_ffi_available
nickel_eval_ffi
+nickel_eval_native
```
## String Macro
diff --git a/docs/src/man/ffi.md b/docs/src/man/ffi.md
@@ -1,32 +1,45 @@
# FFI Mode (High Performance)
-For repeated evaluations, NickelEval provides native FFI bindings to a Rust library that wraps `nickel-lang-core`. This eliminates subprocess overhead.
+For repeated evaluations, NickelEval provides native FFI bindings to a Rust library that wraps `nickel-lang-core`. This eliminates subprocess overhead and preserves Nickel's type semantics.
-## Checking FFI Availability
+## Two FFI Functions
-```julia
-using NickelEval
+### `nickel_eval_native` - Native Types (Recommended)
-check_ffi_available() # => true or false
+Parses Nickel directly into Julia native types using a binary protocol:
+
+```julia
+nickel_eval_native("42") # => 42::Int64
+nickel_eval_native("3.14") # => 3.14::Float64
+nickel_eval_native("true") # => true::Bool
+nickel_eval_native("\"hello\"") # => "hello"::String
+nickel_eval_native("null") # => nothing
+
+nickel_eval_native("[1, 2, 3]") # => Any[1, 2, 3]
+nickel_eval_native("{ x = 1 }") # => Dict("x" => 1)
```
-FFI is available when the compiled Rust library exists in the `deps/` folder.
+**Key benefit:** Type preservation. Integers stay `Int64`, decimals become `Float64`.
+
+### `nickel_eval_ffi` - JSON-based
-## Using FFI Evaluation
+Uses JSON serialization internally, supports typed parsing:
```julia
-# Basic evaluation
-nickel_eval_ffi("1 + 2") # => 3
+nickel_eval_ffi("{ a = 1, b = 2 }") # JSON.Object with dot-access
+nickel_eval_ffi("{ a = 1 }", Dict{String, Int}) # Typed Dict
+```
-# With dot-access
-config = nickel_eval_ffi("{ host = \"localhost\", port = 8080 }")
-config.host # => "localhost"
+## Checking FFI Availability
-# Typed evaluation
-nickel_eval_ffi("{ a = 1, b = 2 }", Dict{String, Int})
-# => Dict{String, Int64}("a" => 1, "b" => 2)
+```julia
+using NickelEval
+
+check_ffi_available() # => true or false
```
+FFI is available when the compiled Rust library exists in the `deps/` folder.
+
## Building the FFI Library
### Requirements
@@ -54,6 +67,20 @@ cp target/release/libnickel_jl.so ../../deps/
cp target/release/nickel_jl.dll ../../deps/
```
+## Type Mapping
+
+| Nickel | Julia (native) |
+|--------|----------------|
+| Integer numbers | `Int64` |
+| Decimal numbers | `Float64` |
+| Bool | `Bool` |
+| String | `String` |
+| null | `nothing` |
+| Array | `Vector{Any}` |
+| Record | `Dict{String, Any}` |
+
+Note: Nickel has a single `Number` type. Whole numbers (like `42` or `42.0`) become `Int64`. Only true decimals (like `3.14`) become `Float64`.
+
## Performance Comparison
FFI mode is faster for repeated evaluations because it:
@@ -64,22 +91,6 @@ FFI mode is faster for repeated evaluations because it:
For single evaluations, the difference is minimal. For batch processing or interactive use, FFI mode is significantly faster.
-## Binary Protocol
-
-The FFI uses a binary protocol that preserves type information:
-
-| Type Tag | Nickel Type |
-|----------|-------------|
-| 0 | Null |
-| 1 | Bool |
-| 2 | Int64 |
-| 3 | Float64 |
-| 4 | String |
-| 5 | Array |
-| 6 | Record |
-
-This allows direct conversion to Julia types without JSON parsing overhead.
-
## Fallback Behavior
If FFI is not available, you can still use the subprocess-based functions:
@@ -89,7 +100,7 @@ If FFI is not available, you can still use the subprocess-based functions:
nickel_eval("1 + 2")
# Requires FFI library
-nickel_eval_ffi("1 + 2") # Error if not built
+nickel_eval_native("1 + 2") # Error if not built
```
## Troubleshooting
