The lowest level parsing API in this package; supports streaming input with a low memory footprint. The memory requirement is O(d) where d is the nesting depth of [] or {} containers in the input. Specifically d/8 bytes are required for this purpose, with some extra buffer according to the implementation of std.ArrayList.

This scanner can emit partial tokens; see std.json.Token. The input to this class is a sequence of input buffers that you must supply one at a time. Call feedInput() with the first buffer, then call next() repeatedly until error.BufferUnderrun is returned. Then call feedInput() again and so forth. Call endInput() when the last input buffer has been given to feedInput(), either immediately after calling feedInput(), or when error.BufferUnderrun requests more data and there is no more. Be sure to call next() after calling endInput() until Token.end_of_document has been returned.

Fields

state: State = .value,
string_is_object_key: bool = false,
stack: BitStack,
value_start: usize = undefined,
unicode_code_point: u21 = undefined,
input: []const u8 = "",
cursor: usize = 0,
is_end_of_input: bool = false,
diagnostics: ?*Diagnostics = null,

Functions

fn allocNextIntoArrayList(self: *@This(), value_list: *ArrayList(u8), when: AllocWhen) AllocIntoArrayListError!?[]const u8

Equivalent to `allocNextIntoArrayListMax(value_list, when, default_max_value_len…

Equivalent to allocNextIntoArrayListMax(value_list, when, default_max_value_len);

fn allocNextIntoArrayListMax(self: *@This(), value_list: *ArrayList(u8), when: AllocWhen, max_value_len: usize) AllocIntoArrayListError!?[]const u8

The next token type must be either .number or .string. See `peekNextTokenTyp…

The next token type must be either .number or .string. See peekNextTokenType(). When allocation is not necessary with .alloc_if_needed, this method returns the content slice from the input buffer, and value_list is not touched. When allocation is necessary or with .alloc_always, this method concatenates partial tokens into the given value_list, and returns null once the final .number or .string token has been written into it. In case of an error.BufferUnderrun, partial values will be left in the given value_list. The given value_list is never reset by this method, so an error.BufferUnderrun situation can be resumed by passing the same array list in again. This method does not indicate whether the token content being returned is for a .number or .string token type; the caller of this method is expected to know which type of token is being processed.

fn deinit(self: *@This()) void

No documentation provided.

fn enableDiagnostics(self: *@This(), diagnostics: *Diagnostics) void

No documentation provided.

fn endInput(self: *@This()) void

Call this when you will no longer call feedInput() anymore. This can be calle…

Call this when you will no longer call feedInput() anymore. This can be called either immediately after the last feedInput(), or at any time afterward, such as when getting error.BufferUnderrun from next(). Don’t forget to call next*() after endInput() until you get .end_of_document.

fn ensureTotalStackCapacity(self: *@This(), height: usize) Allocator.Error!void

Pre allocate memory to hold the given number of nesting levels. stackHeight()

Pre allocate memory to hold the given number of nesting levels. stackHeight() up to the given number will not cause allocations.

fn feedInput(self: *@This(), input: []const u8) void

Call this whenever you get error.BufferUnderrun from next(). When there is …

Call this whenever you get error.BufferUnderrun from next(). When there is no more input to provide, call endInput().

fn initCompleteInput(allocator: Allocator, complete_input: []const u8) @This()

Use this if your input is a single slice. This is effectively equivalent to: `…

Use this if your input is a single slice. This is effectively equivalent to:

initStreaming(allocator);
feedInput(complete_input);
endInput();
fn initStreaming(allocator: Allocator) @This()

The allocator is only used to track [] and {} nesting levels.

fn next(self: *@This()) NextError!Token

See std.json.Token for documentation of this function.

fn nextAlloc(self: *@This(), allocator: Allocator, when: AllocWhen) AllocError!Token

Equivalent to nextAllocMax(allocator, when, default_max_value_len); This func…

Equivalent to nextAllocMax(allocator, when, default_max_value_len); This function is only available after endInput() (or initCompleteInput()) has been called. See also std.json.Token for documentation of nextAlloc*() function behavior.

fn nextAllocMax(self: *@This(), allocator: Allocator, when: AllocWhen, max_value_len: usize) AllocError!Token

This function is only available after endInput() (or initCompleteInput()) ha…

This function is only available after endInput() (or initCompleteInput()) has been called. See also std.json.Token for documentation of nextAlloc*() function behavior.

fn peekNextTokenType(self: *@This()) PeekError!TokenType

Seeks ahead in the input until the first byte of the next token (or the end of t…

Seeks ahead in the input until the first byte of the next token (or the end of the input) determines which type of token will be returned from the next next*() call. This function is idempotent, only advancing past commas, colons, and inter-token whitespace.

fn skipUntilStackHeight(self: *@This(), terminal_stack_height: usize) NextError!void

Skip tokens until an .object_end or .array_end token results in a `stackHeig…

Skip tokens until an .object_end or .array_end token results in a stackHeight() equal the given stack height. Unlike skipValue(), this function is available in streaming mode.

fn skipValue(self: *@This()) SkipError!void

This function is only available after endInput() (or initCompleteInput()) ha…

This function is only available after endInput() (or initCompleteInput()) has been called. If the next token type is .object_begin or .array_begin, this function calls next() repeatedly until the corresponding .object_end or .array_end is found. If the next token type is .number or .string, this function calls next() repeatedly until the (non .partial_*) .number or .string token is found. If the next token type is .true, .false, or .null, this function calls next() once. The next token type must not be .object_end, .array_end, or .end_of_document; see peekNextTokenType().

fn stackHeight(self: *const @This()) usize

The depth of {} or [] nesting levels at the current position.

DocTests

test Scanner {
    var scanner = Scanner.initCompleteInput(testing.allocator, "{\"foo\": 123}\n");
    defer scanner.deinit();
    try testing.expectEqual(Token.object_begin, try scanner.next());
    try testing.expectEqualSlices(u8, "foo", (try scanner.next()).string);
    try testing.expectEqualSlices(u8, "123", (try scanner.next()).number);
    try testing.expectEqual(Token.object_end, try scanner.next());
    try testing.expectEqual(Token.end_of_document, try scanner.next());
}