Constructing DST pointers in Rust

Recently I was working on a Rust library that implements several data structures. One of the things I needed to do was to allocate variable-length structs. I didn't immediately find a good way to do this (even in Rustonomicon), so I think it's worth documenting how I ended up implementing it.

I found some very useful hints in the code of slice-dst and zerocopy crates.

Background

Rust has several kinds of dynamically sized types (DSTs):

• slices: [T]
• trait objects: dyn Trait
• structs that end with a DST:

  struct LenPrefixedBuf {
    len: usize,
    data: [u8]
  }
  

In this post, I'll ignore trait objects and focus on the other two kinds of DSTs: slices and structs with slices.

One way to create a pointer to a DST is to create a pointer to a statically sized type and cast it to a pointer to a compatible DST. For my use case, this was not possible because the size of slices has to be dynamic.

Constructing pointers to slices

Pointers to slices can be created using std::ptr::slice_from_raw_parts:

let n: usize = 10;
let layout = Layout::array::<i32>(n).unwrap();
let allocated: *mut u8 = unsafe { alloc(layout) };

let slice_ptr: *mut [u8] = slice_from_raw_parts_mut(allocated, n);

Constructing pointers to variable-length structs

Constructing pointers to a variable-length struct is more complicated, and there is no straightforward way to do it. There's an unstable feature that should make it easier, but I want to avoid using unstable features unnecessarily.

The way to construct a pointer is as follows:

1. First, make a slice pointer with the address of a struct and the correct number of elements. The element type of a slice does not matter.
2. Then cast the slice pointer to a struct pointer.

This works because of the way pointer casting is specified for DSTs (see https://doc.rust-lang.org/reference/expressions/operator-expr.html#pointer-to-pointer-cast):

If T and U are both unsized, the pointer is also returned unchanged. In particular, the metadata is preserved exactly.

#[repr(C)]
struct BufWithLen<T> {
    header: BufWithLenHeader,
    data: [T],
}
#[repr(C)]
struct BufWithLenHeader {
    len: usize,
}
impl<T> BufWithLen<T> {
    fn layout(n: usize) -> Layout {
        Layout::new::<BufWithLenHeader>()
            .extend(Layout::array::<i32>(n).unwrap())
            .unwrap()
            .0
            .pad_to_align()
    }
}

let n: usize = 10;
let layout = BufWithLen::<i32>::layout(n);
let allocated: *mut u8 = unsafe { alloc(layout) };

// 1) First create pointer with the right address and the right metadata
let dummy_slice: *mut [u8] = slice_from_raw_parts_mut(allocated, n);
// 2) Then cast pointer to the correct type
let buf_ptr: *mut BufWithLen<i32> = dummy_slice as *mut _;

References to DSTs

A pointer can be converted to a reference by dereferencing it:

let p: *const [T] = ...;
let r: &[T] = unsafe { &*p };

Of course, the unsafe block requires establishing the appropriate safety guarantees.