fn findTagAtCurrentIndex(self: *Tokenizer, tag: Token.Tag) Token

This is a workaround to the fact that the tokenizer can queue up ‘pending_invalid_token’s when parsing literals, which means that we need to scan from the start of the current line to find a matching tag - just in case it was an invalid character generated during literal tokenization. Ideally this processing of this would be pushed to the AST parser or another later stage, both to give more useful error messages with that extra context and in order to be able to remove this workaround.

Parameters

self: *Tokenizer,
tag: Token.Tag,