langchain-ai/docs
1
0
Fork 0
mirror of https://github.com/langchain-ai/docs.git synced 2026-07-22 20:35:34 -04:00
Code Issues 490 Packages Projects Releases Wiki Activity

[GH-ISSUE #2069] Autolink preprocessor matches patterns inside code blocks #267

New Issue
Closed
opened 2026-02-17 17:19:31 -05:00 by yindo · 5 comments
yindo commented 2026-02-17 17:19:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @mdrxy on GitHub (Jan 5, 2026).
Original GitHub issue: https://github.com/langchain-ai/docs/issues/2069

Originally assigned to: @Devpatel1901 on GitHub.

Problem

The autolink preprocessor (pipeline/preprocessors/handle_auto_links.py) transforms @[link_name] syntax into markdown links, but it doesn't skip fenced code blocks. This causes false positive warnings when code contains regex patterns that happen to include @[ followed by characters and ].

Examples

src/langsmith/mask-inputs-outputs.mdx - Email regex patterns trigger warnings:

{ "pattern": r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+.[a-zA-Z]{2,}", "replace": "<email-address>" }
Warning: Link 'a-zA-Z0-9.-' not found in scope 'python'

src/oss/python/integrations/llms/google_vertex_ai.mdx - Email validation regex:

regex = re.compile(r"[^@]+@[^@]+\.[^@]+")
Warning: Link '^@' not found in scope 'python'

Root cause

The replace_autolinks function in handle_auto_links.py processes every line looking for the @[...] pattern. It correctly tracks :::language fence scopes but doesn't track or skip triple-backtick code blocks.

Originally created by @mdrxy on GitHub (Jan 5, 2026). Original GitHub issue: https://github.com/langchain-ai/docs/issues/2069 Originally assigned to: @Devpatel1901 on GitHub. ## Problem The autolink preprocessor (`pipeline/preprocessors/handle_auto_links.py`) transforms `@[link_name]` syntax into markdown links, but it doesn't skip fenced code blocks. This causes false positive warnings when code contains regex patterns that happen to include `@[ followed by characters and ]`. ## Examples `src/langsmith/mask-inputs-outputs.mdx` - Email regex patterns trigger warnings: ``` { "pattern": r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+.[a-zA-Z]{2,}", "replace": "<email-address>" } Warning: Link 'a-zA-Z0-9.-' not found in scope 'python' ``` `src/oss/python/integrations/llms/google_vertex_ai.mdx` - Email validation regex: ``` regex = re.compile(r"[^@]+@[^@]+\.[^@]+") Warning: Link '^@' not found in scope 'python' ``` ## Root cause The `replace_autolinks` function in `handle_auto_links.py` processes every line looking for the `@[...]` pattern. It correctly tracks `:::language` fence scopes but doesn't track or skip triple-backtick code blocks.
yindo added the internal label 2026-02-17 17:19:31 -05:00
yindo commented 2026-02-17 17:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@Devpatel1901 commented on GitHub (Jan 22, 2026):

Hi @mdrxy, If this issue is still pending then could you please assign this to me? Thank you.

@Devpatel1901 commented on GitHub (Jan 22, 2026): Hi @mdrxy, If this issue is still pending then could you please assign this to me? Thank you.
yindo commented 2026-02-17 17:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@Devpatel1901 commented on GitHub (Jan 23, 2026):

Hi @mdrxy, Before I submit a PR, I would like to propose my solution. If we agree on below proposed solution, then I will go ahead and generate a PR which will fixes this issue.

Approach:

Track code block state as a simple boolean toggle, when encountering backtick at the start of line, we should toggle in_code_block state, skip auto link processing while in_code_block == True and set in_code_block == false until we found another backtick occurrence.

It's a simple to implement but it assumes that we have well-formed markdown and in addition to that it doesn't distinguish between opening and closing backticks explicitly.

I search for occurrences of backtick in doc repo, and I found that it mostly used to either code examples or command examples. (like ```python, ```javascript, ```bash npm, ```bash yarn, ```typescript, ```text)

Let me know if this is a good solution to proceed with or if you feel another way then please suggest any alternatives.

@Devpatel1901 commented on GitHub (Jan 23, 2026): Hi @mdrxy, Before I submit a PR, I would like to propose my solution. If we agree on below proposed solution, then I will go ahead and generate a PR which will fixes this issue. **Approach:** Track code block state as a simple boolean toggle, when encountering **backtick** at the start of line, we should toggle `in_code_block` state, skip auto link processing while `in_code_block == True` and set `in_code_block == false` until we found another **backtick** occurrence. It's a simple to implement but it assumes that we have well-formed markdown and in addition to that it doesn't distinguish between opening and closing backticks explicitly. I search for occurrences of backtick in `doc` repo, and I found that it mostly used to either code examples or command examples. (like \```python, \```javascript, \```bash npm, \```bash yarn, \```typescript, \```text) Let me know if this is a good solution to proceed with or if you feel another way then please suggest any alternatives.
yindo commented 2026-02-17 17:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@mdrxy commented on GitHub (Jan 23, 2026):

@Devpatel1901 I think in this case it may make sense to consider a markdown parser library to identify code blocks, then only run autolink replacement on non-code text nodes. I think this would probably do a better job of handling edge cases

@mdrxy commented on GitHub (Jan 23, 2026): @Devpatel1901 I think in this case it may make sense to consider a markdown parser library to identify code blocks, then only run autolink replacement on non-code text nodes. I think this would probably do a better job of handling edge cases
yindo commented 2026-02-17 17:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@Devpatel1901 commented on GitHub (Jan 24, 2026):

Hi @mdrxy,

As per your suggestion, I explored using markdown-it-py to parse code blocks. However, to proceed, we’ll need to add markdown-it-py >= 3.0.0 as a project dependency. Are we okay with introducing this new dependency?

Below are some sample inputs and their corresponding parsed results to demonstrate my understanding of how the library behaves:

from markdown_it import MarkdownIt

md = MarkdownIt("commonmark")

_samples = [
    """```python {highlight={8-9,20-30}} title="src/security/auth.py""",
    """```"""
]

for s in _samples:
    tokens = md.parse(s)
    for t in tokens:
        print(t)

Results:

Token(type='fence', tag='code', nesting=0, attrs={}, map=[0, 1], level=0, children=None, content='', markup='```', info='python {highlight={8-9,20-30}} title="src/security/auth.py', meta={}, block=True, hidden=False)
Token(type='fence', tag='code', nesting=0, attrs={}, map=[0, 1], level=0, children=None, content='', markup='```', info='', meta={}, block=True, hidden=False)

From these results, we can apply our existing logic — that is, toggle the in_code_block flag to True when encountering a token with type == "fence" and markup == "```", and ignore all subsequent lines until in_code_block is set back to False.

Let me know your views on this.

@Devpatel1901 commented on GitHub (Jan 24, 2026): Hi @mdrxy, As per your suggestion, I explored using [markdown-it-py](https://markdown-it-py.readthedocs.io/en/latest/index.html) to parse code blocks. However, to proceed, we’ll need to add markdown-it-py >= 3.0.0 as a project dependency. Are we okay with introducing this new dependency? Below are some sample inputs and their corresponding parsed results to demonstrate my understanding of how the library behaves: ```python from markdown_it import MarkdownIt md = MarkdownIt("commonmark") _samples = [ """```python {highlight={8-9,20-30}} title="src/security/auth.py""", """```""" ] for s in _samples: tokens = md.parse(s) for t in tokens: print(t) ``` Results: ```bash Token(type='fence', tag='code', nesting=0, attrs={}, map=[0, 1], level=0, children=None, content='', markup='```', info='python {highlight={8-9,20-30}} title="src/security/auth.py', meta={}, block=True, hidden=False) Token(type='fence', tag='code', nesting=0, attrs={}, map=[0, 1], level=0, children=None, content='', markup='```', info='', meta={}, block=True, hidden=False) ``` From these results, we can apply our existing logic — that is, toggle the `in_code_block` flag to True when encountering a token with **type == "fence"** and **markup == "```",** and ignore all subsequent lines until `in_code_block` is set back to False. Let me know your views on this.
yindo commented 2026-02-17 17:19:31 -05:00
Author
Owner
Copy Link
Copy Source

@mdrxy commented on GitHub (Jan 26, 2026):

@Devpatel1901 sounds good - excited to review

@mdrxy commented on GitHub (Jan 26, 2026): @Devpatel1901 sounds good - excited to review
yindo changed title from Autolink preprocessor matches patterns inside code blocks to [GH-ISSUE #2069] Autolink preprocessor matches patterns inside code blocks 2026-06-05 17:25:57 -04:00
yindo closed this issue 2026-06-05 17:25:57 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
deepagents
docs-infra
external
external
help wanted
integration
internal
internal
javascript
js/ts
langchain
langgraph
langsmith
needs-title
pull-request
Mirrored from GitHub Pull Request
 python
reference
No Label internal
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/docs#267
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?