feat: highlight SQL in CodeBlock with the ClickHouse WASM lexer - #1164
feat: highlight SQL in CodeBlock with the ClickHouse WASM lexer#1164alexey-milovidov wants to merge 2 commits into
Conversation
Port the SQL highlighting of the ClickHouse Web UI (programs/server/play.html) to CodeBlock: the query is tokenized by ClickHouse's own lexer (src/Parsers/Lexer.cpp) compiled to WebAssembly (~9 KB, byte-identical to the payload embedded in play.html), and the tokens are classified exactly as the Web UI and clickhouse-client do it: keywords bold, functions distinguished from identifiers by peeking at the next token, strings, numbers, quoted identifiers, comments, rainbow brackets by nesting depth (type-aware, so broken nesting colors nothing), digit-group underlines in long numbers, and an error style for an un-lexable tail. The colors are the play.html palette for both the light and the dark theme. The rendering mirrors the DOM of the react-syntax-highlighter path (pre > code > one span per line, same inline line-number spans), so line numbers, wrapping and the copy button behave identically. While the WASM module instantiates - or when WebAssembly is unavailable or the lexer rejects the text - CodeBlock falls back to the previous highlight.js rendering, so a block is never left unstyled. Also fix setupTests.ts: the browserify util package shadows Node's built-in, so the TextEncoder polyfill was silently assigning undefined; import from node:util and add TextDecoder. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: ba8a879 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8f89803. Configure here.
| </span> | ||
| )} | ||
| {showLineNumbers ? <span>{content}</span> : content} | ||
| </span> |
There was a problem hiding this comment.
Wrapped SQL lines render extra blanks
Medium Severity
When wrap is on and line numbers are shown, each SQL line becomes a flex row while a trailing newline stays inside the content span. With pre-wrap that newline still creates a line box, so rows look double-spaced and numbers no longer line up with the code.
Reviewed by Cursor Bugbot for commit 8f89803. Configure here.
There was a problem hiding this comment.
fix suggestion: skip the '\n' when the line renders as its own row as the row already provides the break.
see: https://github.com/ClickHouse/click-ui/pull/1164/changes/BASE..8f89803527e6311a16cac4e24b71a9976caf4815#diff-ee99fd60d2e3215cf7178d23af1c7080c27dc6e07dcf7901f95978a2ce5da86fR149
| style={lineNumberStyle} | ||
| > | ||
| {lineIndex + 1} | ||
| </span> |
There was a problem hiding this comment.
SQL line numbers lose numbers color
Low Severity
SQL line-number spans copy highlighter class names but only set layout styles. Token colors are inline, and those classes have no CSS, so the numbers inherit the pre text color instead of --codeblock-numbers (or the highlight.js comment color the other path uses).
Reviewed by Cursor Bugbot for commit 8f89803. Configure here.
| 'MONTH', | ||
| 'MOVE', | ||
| 'MUTATION', | ||
| 'NAN_SQL', |
There was a problem hiding this comment.
NAN keyword never classified
Low Severity
The keyword set stores NAN_SQL (the C++ enum name used to avoid the NAN macro) instead of the SQL lexeme NAN. tokenKind uppercases the bare word and looks it up, so nan/NaN never match and are painted as identifiers.
Reviewed by Cursor Bugbot for commit 8f89803. Configure here.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Storybook Preview Deployed✅ Preview URL: https://click-8fxq8s778-clickhouse.vercel.app Built from commit: |
2 participants
What
CodeBlocknow highlightslanguage="sql"with ClickHouse's own lexer (src/Parsers/Lexer.cppin ClickHouse/ClickHouse) compiled to WebAssembly — the same embedded payload (~9 KB base64, byte-identical), token classification and colors as the ClickHouse Web UI (programs/server/play.html) and the ClickHouse documentation. This ports ClickHouse/clickhouse-fiddle#81 to Click-UI's static code display, as discussed there.sql/lexer.ts): the exacttokenizeSyncfrom play.html, including the__heap_basebuffer layout,max_query_size = 0and the forward-progress guard.sql/highlight.ts): the same keyword table and logic — keywords bold, function calls distinguished from identifiers by peeking at the next non-whitespace/non-comment token (sum(vs.x), strings, numbers, quoted identifiers, comments in italic.SELECT ([)]colors nothing), digit-group underlines in numbers with ≥5 integer digits (1̲234̲567), and a red-wavy error style for a tail the lexer cannot process. The cursor-dependent features of the Web UI (matched-pair emphasis, identifier-under-cursor underline) do not apply to a read-only block and are not ported.clickhouse-clientrenders to).Rendering & fallback
The lexer path mirrors the DOM of the
react-syntax-highlighterpath (pre > code > one span per line, the same inline line-number spans), so line numbers, wrapping and the copy button behave identically. While the WASM module instantiates — or when WebAssembly is unavailable or the lexer rejects the text — CodeBlock keeps the previous highlight.js rendering, so a block is never left unstyled. Other languages are untouched.Screenshots
New
ClickHouseSqlHighlightingstory (dark / light):Existing
Playgroundstory, before → after (notearrayJoin/arraySort/groupArraynow recognized as functions, identifiers colored, rainbow brackets, bold keywords):Also fixed
setupTests.tsimportedTextEncoderfrom"util", which resolves to the browserifyutilpackage innode_modules— it has noTextEncoder, so the polyfill silently assignedundefined. It now imports fromnode:utiland also polyfillsTextDecoder(the lexer needs both under jsdom).Verification
yarn typecheckpassesyarn vitest run: 550 tests pass (6 new CodeBlock tests: classification, function/identifier disambiguation, rainbow and broken-nesting brackets, digit-group underlines, error tail, line numbers, and the highlight.js fallback for other languages)yarn lint: only pre-existing warningsyarn buildpasses (including the dist health check)🤖 Generated with Claude Code