Buffer

Luma provides binary data handling through the buffer type.
A buffer builds or parses binary data — write to build packets, read to parse them.
Both write() and read() are fully duck-typed. No named variants.

Creating a buffer

Empty buffer (for writing)

buf: buffer = buffer()

Creates an empty buffer. Write calls append data. Extract the result with buf.to_bytes().

buf: buffer = buffer()
buf.write(0x03)                  // COM_QUERY command — byte
buf.write("SELECT 1")           // query text — str
packet: [byte] = buf.to_bytes() // ready to send

Buffer from bytes (for reading)

buf: buffer = buffer(data)

Creates a buffer from an existing [byte] slice. The read cursor starts at position 0 and advances with each read.

data: [byte] = sock.read(length)
buf: buffer = buffer(data)
protocol: byte = buf.read()              // first field
version: str = buf.read(0x00)            // read until null
thread_id: int = buf.read(4).to_int32()  // read 4 bytes, interpret as int

A single buffer type handles both modes. The usage is clear from context:

  • buffer() then write() — building
  • buffer(data) then read() — parsing

Writing data

buf.write(data)

The type of the argument determines what happens:

Arg type Behavior
byte Appends a single byte (0-255)
[byte] Appends raw bytes
str Appends UTF-8 bytes, no terminator
buffer Appends the other buffer’s contents
buf: buffer = buffer()
buf.write(0x0A)            // byte
buf.write(payload)         // [byte]
buf.write("SELECT 1")     // str
buf.write(other_buf)       // buffer

Null-terminated strings

No special method — write the string, then write the null byte:

buf.write(username)
buf.write(0x00)

Writing integers

Integer byte-order conversion is a type method on int, not a buffer method.
Convert the int to [byte], then write:

buf.write(payload_len.to_int24())     // 3 bytes, little-endian
buf.write(cap_flags.to_int32())       // 4 bytes, little-endian
buf.write(value.to_int32_be())        // 4 bytes, big-endian

The .to_int*() methods return [byte], so duck-typed write() handles it naturally.
Buffer doesn’t need to know about byte order — that’s the int’s job.

Composing buffers

Appending one buffer to another uses the same write():

header: buffer = buffer()
header.write(body.len().to_int24())
header.write(seq_id)
header.write(body)                     // body is buffer — Luma knows
sock.write(header.to_bytes())

Reading data

buf.read()
buf.read(n)
buf.read(delimiter)

A single read() method dispatches on both argument type and return type:

Call Arg type Return type Behavior
buf.read() none byte Read one byte
buf.read() none [byte] Read all remaining bytes
buf.read() none str Read all remaining as string
buf.read(4) int [byte] Read N bytes
buf.read(4) int str Read N bytes as string
buf.read(0x00) byte [byte] Read until delimiter
buf.read(0x00) byte str Read until delimiter as string

Read one byte

v: byte = buf.read()

Reads a single byte and advances the cursor by 1.

command: byte = buf.read()
if command == 0x00 {
    // OK packet
} else if command == 0xFF {
    // ERR packet
}

Read N bytes

data: [byte] = buf.read(8)
text: str = buf.read(4)

When the argument is an int, reads exactly N bytes.
The declared type controls the format — [byte] or str.

Read until delimiter

text: str = buf.read(0x00)
data: [byte] = buf.read(0xFF)

When the argument is a byte (not int), scans forward until it finds the delimiter byte.
Returns everything before the delimiter and advances past it.
Panics if the delimiter is not found.

version: str = buf.read(0x00)               // "8.0.32"
thread_id: int = buf.read(4).to_int32()     // next field
auth_data: [byte] = buf.read(8)             // next 8 bytes

Read all remaining

rest: [byte] = buf.read()
rest: str = buf.read()

When read() has no argument and the return type is [byte] or str, reads everything from the cursor to the end of the buffer.

// Error message is everything after the fixed fields
error_msg: str = buf.read()

Reading integers

Integer byte-order conversion is a type method on [byte], not a buffer method.
Read the bytes, then convert:

length: int = buf.read(3).to_int24()       // 3 bytes, little-endian
flags: int = buf.read(4).to_int32()        // 4 bytes, little-endian
value: int = buf.read(4).to_int32_be()     // 4 bytes, big-endian

Buffer reads bytes — interpretation is the bytes’ job.

Utility methods

Extract all bytes

data: [byte] = buf.to_bytes()

Returns the entire buffer contents as [byte].
Does not affect the read cursor. Used after building a packet to get the final bytes for sending.

Buffer length

size: int = buf.len()

Returns the total number of bytes in the buffer (not remaining — total).

Cursor position

pos: int = buf.position()

Returns the current read cursor position (0-based).
After reading a 4-byte header, position() returns 4.

Remaining bytes

left: int = buf.remaining()

Returns the number of bytes between the cursor and the end.
remaining() == len() - position().

Skip bytes

buf.skip(n)

Advances the cursor by N bytes without returning anything.
Used for reserved fields, padding, and filler bytes.

buf.skip(10)    // skip 10 reserved bytes

Reset cursor

buf.reset()

Resets the read cursor to position 0. Allows re-reading the buffer from the beginning.

Error handling

Buffer errors are always panics. There are no optional variants.

Unlike TCP reads (where network failures are expected), buffer operations work on data that’s already in memory. A read failure means the data is malformed or the code has a bug. Panicking immediately surfaces the problem.

Operation Panics when
buf.read() / buf.read(n) Not enough bytes remaining
buf.read(delimiter) Delimiter not found before end
buf.skip(n) Not enough bytes remaining

Examples

Build a protocol packet

body: buffer = buffer()
body.write(0x03)               // command byte
body.write("SELECT 1")        // query string

pkt: buffer = buffer()
pkt.write(body.len().to_int24())  // 3-byte payload length
pkt.write(0x00)                   // sequence ID
pkt.write(body)                   // payload

sock.write(pkt.to_bytes())

Parse a MySQL server greeting

data: [byte] = sock.read(packet_length)
buf: buffer = buffer(data)

protocol: byte = buf.read()                // protocol version (10)
version: str = buf.read(0x00)              // "8.0.32"
thread_id: int = buf.read(4).to_int32()   // connection ID
auth_data1: [byte] = buf.read(8)          // first 8 bytes of nonce
buf.skip(1)                                // filler

print("Server version: ${version}")
print("Thread ID: ${thread_id}")

Round-trip test

original_id: int = 305419896       // 0x12345678
original_name: str = "Alice"

// Write
w: buffer = buffer()
w.write(original_id.to_int32())
w.write(original_name)
w.write(0x00)

// Read back
r: buffer = buffer(w.to_bytes())
read_id: int = r.read(4).to_int32()
read_name: str = r.read(0x00)

print("ID match: ${read_id == original_id}")       // true
print("Name match: ${read_name == original_name}") // true
print("All bytes consumed: ${r.remaining() == 0}") // true

Method summary

Method Arguments Returns Description
buffer() buffer Create empty write buffer
buffer(data) data: [byte] buffer Create read buffer from bytes
buf.write() byte, [byte], str, or buffer Append data (duck-typed on arg)
buf.read() byte Read one byte
buf.read() [byte] or str Read all remaining
buf.read(n) n: int [byte] or str Read N bytes
buf.read(d) d: byte [byte] or str Read until delimiter
buf.to_bytes() [byte] Extract all bytes
buf.len() int Total buffer size
buf.position() int Current cursor position
buf.remaining() int Bytes left to read
buf.skip(n) n: int Advance cursor without reading
buf.reset() Reset cursor to 0

Design notes

  • write() dispatches by argument type — pass a byte, bytes, string, or buffer, Luma figures it out
  • read() dispatches by both argument type and return type — the most duck-typed method in Luma
  • Integer byte-order is handled by type methods on int and [byte], not by buffer — each concern owns its own logic
  • Buffer always panics on errors — data in memory is either correct or a bug, no middle ground
  • One buffer type for both building and parsing — mode is clear from context, no type choice needed
Last updated on