Enum Token 

pub enum Token {
    Heading(Vec<Token>, usize),
    Emphasis {
        level: usize,
        content: Vec<Token>,
    },
    StrongEmphasis(Vec<Token>),
    Code(String, String),
    BlockQuote(String),
    ListItem {
        content: Vec<Token>,
        ordered: bool,
        number: Option<usize>,
    },
    Link(String, String),
    Image(String, String),
    Text(String),
    Table {
        headers: Vec<Vec<Token>>,
        aligns: Vec<Alignment>,
        rows: Vec<Vec<Vec<Token>>>,
    },
    TableAlignment(Alignment),
    HtmlComment(String),
    Math {
        content: String,
        display: bool,
    },
    Newline,
    HorizontalRule,
    Unknown(String),
}

Represents the different types of tokens that can be parsed from Markdown text. Each variant captures both the semantic meaning and associated content/metadata needed to properly render the element.

Variants

Heading(Vec<Token>, usize)

A heading with nested content and level (e.g., # h1, ## h2)

Emphasis

Emphasized text with configurable level (1-3) for * or _ delimiters

Fields

level: usize

content: Vec<Token>

StrongEmphasis(Vec<Token>)

Strong emphasis (bold) text using ** or __ delimiters

Code(String, String)

Code block with optional language specification and content

BlockQuote(String)

Block quote containing quoted text

ListItem

List item with nested content and type information

Fields

content: Vec<Token>

ordered: bool

number: Option<usize>

Link(String, String)

Link with display text and URL

Image(String, String)

Image with alt text and URL

Text(String)

Plain text content

Table

Table with header, alignment info, and rows

Fields

headers: Vec<Vec<Token>>

aligns: Vec<Alignment>

rows: Vec<Vec<Vec<Token>>>

TableAlignment(Alignment)

Text alignment for table columns

HtmlComment(String)

HTML comment content

Math

LaTeX math block (inline $…$ or display $$…$$)

Fields

content: String

display: bool

Newline

Line break

HorizontalRule

Horizontal rule (—)

Unknown(String)

Unknown or malformed token

Implementations

impl Token

pub fn save_to_json_file(tokens: Vec<Token>, file_path: &str) -> Result<()>

Saves tokens to a JSON file for visualization. Recursively formats nested tokens with proper indentation.

Arguments

• tokens - The tokens to save
• file_path - Path to the output JSON file (e.g., “tokens.json”)

Returns

Result indicating success or IO error

Example

use markdown2pdf::markdown::Lexer;

let mut lexer = Lexer::new("# Title".to_string());
let tokens = lexer.parse().unwrap();
markdown2pdf::markdown::Token::save_to_json_file(tokens, "tokens.json").unwrap();

fn to_readable_json(&self, indent_level: usize) -> String

Converts a token into a readable JSON representation for visualization. Recursively formats nested tokens with proper indentation.

fn tokens_to_readable_json(tokens: Vec<Token>) -> String

Convenience method to convert a vector of tokens into a readable JSON array.

impl Token

pub fn collect_all_text(tokens: &[Token]) -> String

Recursively extracts all text content from a token and its nested tokens. This is useful for collecting all characters used in a document for font subsetting.

Returns

A string containing all text content from this token and any nested tokens.

Example

use markdown2pdf::markdown::Token;

let tokens = vec![
    Token::Heading(vec![Token::Text("Title".to_string())], 1),
    Token::Text("Body text with ăâîșț".to_string()),
];

let all_text = Token::collect_all_text(&tokens);
assert!(all_text.contains("Title"));
assert!(all_text.contains("ăâîșț"));

fn collect_text_recursive(&self, result: &mut String)

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token