The lowest level parsing API in this package; supports streaming input with a low memory footprint. The memory requirement is O(d)
where d is the nesting depth of []
or {}
containers in the input. Specifically d/8
bytes are required for this purpose, with some extra buffer according to the implementation of std.ArrayList
.
This scanner can emit partial tokens; see std.json.Token
. The input to this class is a sequence of input buffers that you must supply one at a time. Call feedInput()
with the first buffer, then call next()
repeatedly until error.BufferUnderrun
is returned. Then call feedInput()
again and so forth. Call endInput()
when the last input buffer has been given to feedInput()
, either immediately after calling feedInput()
, or when error.BufferUnderrun
requests more data and there is no more. Be sure to call next()
after calling endInput()
until Token.end_of_document
has been returned.
Fields
state: State = .value,
string_is_object_key: bool = false,
stack: BitStack,
value_start: usize = undefined,
unicode_code_point: u21 = undefined,
input: []const u8 = "",
cursor: usize = 0,
is_end_of_input: bool = false,
diagnostics: ?*Diagnostics = null,
Functions
fn allocNextIntoArrayList(self: *@This(), value_list: *ArrayList(u8), when: AllocWhen) AllocIntoArrayListError!?[]const u8
Equivalent to `allocNextIntoArrayListMax(value_list, when, default_max_value_len…
Equivalent to
allocNextIntoArrayListMax(value_list, when, default_max_value_len);
fn allocNextIntoArrayListMax(self: *@This(), value_list: *ArrayList(u8), when: AllocWhen, max_value_len: usize) AllocIntoArrayListError!?[]const u8
The next token type must be either
.number
or.string
. See `peekNextTokenTyp…The next token type must be either
.number
or.string
. SeepeekNextTokenType()
. When allocation is not necessary with.alloc_if_needed
, this method returns the content slice from the input buffer, andvalue_list
is not touched. When allocation is necessary or with.alloc_always
, this method concatenates partial tokens into the givenvalue_list
, and returnsnull
once the final.number
or.string
token has been written into it. In case of anerror.BufferUnderrun
, partial values will be left in the given value_list. The givenvalue_list
is never reset by this method, so anerror.BufferUnderrun
situation can be resumed by passing the same array list in again. This method does not indicate whether the token content being returned is for a.number
or.string
token type; the caller of this method is expected to know which type of token is being processed.fn endInput(self: *@This()) void
Call this when you will no longer call
feedInput()
anymore. This can be calle…Call this when you will no longer call
feedInput()
anymore. This can be called either immediately after the lastfeedInput()
, or at any time afterward, such as when gettingerror.BufferUnderrun
fromnext()
. Don’t forget to callnext*()
afterendInput()
until you get.end_of_document
.fn ensureTotalStackCapacity(self: *@This(), height: usize) Allocator.Error!void
Pre allocate memory to hold the given number of nesting levels.
stackHeight()
…Pre allocate memory to hold the given number of nesting levels.
stackHeight()
up to the given number will not cause allocations.fn feedInput(self: *@This(), input: []const u8) void
Call this whenever you get
error.BufferUnderrun
fromnext()
. When there is …Call this whenever you get
error.BufferUnderrun
fromnext()
. When there is no more input to provide, callendInput()
.fn initCompleteInput(allocator: Allocator, complete_input: []const u8) @This()
Use this if your input is a single slice. This is effectively equivalent to: `…
Use this if your input is a single slice. This is effectively equivalent to:
initStreaming(allocator); feedInput(complete_input); endInput();
fn initStreaming(allocator: Allocator) @This()
The allocator is only used to track
[]
and{}
nesting levels.fn nextAlloc(self: *@This(), allocator: Allocator, when: AllocWhen) AllocError!Token
Equivalent to
nextAllocMax(allocator, when, default_max_value_len);
This func…Equivalent to
nextAllocMax(allocator, when, default_max_value_len);
This function is only available afterendInput()
(orinitCompleteInput()
) has been called. See alsostd.json.Token
for documentation ofnextAlloc*()
function behavior.fn nextAllocMax(self: *@This(), allocator: Allocator, when: AllocWhen, max_value_len: usize) AllocError!Token
This function is only available after
endInput()
(orinitCompleteInput()
) ha…This function is only available after
endInput()
(orinitCompleteInput()
) has been called. See alsostd.json.Token
for documentation ofnextAlloc*()
function behavior.fn peekNextTokenType(self: *@This()) PeekError!TokenType
Seeks ahead in the input until the first byte of the next token (or the end of t…
Seeks ahead in the input until the first byte of the next token (or the end of the input) determines which type of token will be returned from the next
next*()
call. This function is idempotent, only advancing past commas, colons, and inter-token whitespace.fn skipUntilStackHeight(self: *@This(), terminal_stack_height: usize) NextError!void
Skip tokens until an
.object_end
or.array_end
token results in a `stackHeig…Skip tokens until an
.object_end
or.array_end
token results in astackHeight()
equal the given stack height. UnlikeskipValue()
, this function is available in streaming mode.fn skipValue(self: *@This()) SkipError!void
This function is only available after
endInput()
(orinitCompleteInput()
) ha…This function is only available after
endInput()
(orinitCompleteInput()
) has been called. If the next token type is.object_begin
or.array_begin
, this function callsnext()
repeatedly until the corresponding.object_end
or.array_end
is found. If the next token type is.number
or.string
, this function callsnext()
repeatedly until the (non.partial_*
).number
or.string
token is found. If the next token type is.true
,.false
, or.null
, this function callsnext()
once. The next token type must not be.object_end
,.array_end
, or.end_of_document
; seepeekNextTokenType()
.fn stackHeight(self: *const @This()) usize
The depth of
{}
or[]
nesting levels at the current position.
DocTests
test Scanner { var scanner = Scanner.initCompleteInput(testing.allocator, "{\"foo\": 123}\n"); defer scanner.deinit(); try testing.expectEqual(Token.object_begin, try scanner.next()); try testing.expectEqualSlices(u8, "foo", (try scanner.next()).string); try testing.expectEqualSlices(u8, "123", (try scanner.next()).number); try testing.expectEqual(Token.object_end, try scanner.next()); try testing.expectEqual(Token.end_of_document, try scanner.next()); }