diff --git a/README.md b/README.md deleted file mode 100644 index 86e4c00547a42156b147e6b50aa245b9f371d276..0000000000000000000000000000000000000000 --- a/README.md +++ /dev/null @@ -1,114 +0,0 @@ -# `@clo/react-markdown` - -This package exports a React component to render Markdown using the [unified] -ecosystem's Markdown tools (remark, rehype). The exported `` -component is extremely memoized, making it suitable for streaming situations -such as LLM chat interfaces. - -- **Bring your Existing Pipeline**: The primary option for configuration is - providing a `unified.Processor` with remark and rehype plugins configured. - Additionally, you can override the element renderers with the `components` - attribute. By default, `@clo/react-markdown` applies a bare bones config so - you can get started with just ``. Configuration is - done either with a provider, or right at the component level. -- **Handle Partial Markdown**: When the `predict` prop is set, the document - will be treated as an incomplete stream at the end of a document. - `hello **world` will be emitted as `hello world`, perfect for - the LLM case. (standalone predictor exported as `@clo/react-markdown/Predict`) - -The motivation for this package is to have an easy to understand version of -[Streamdown]. With me banning all Vercel software at the company I work at, -we needed a simple, trustable solution for this problem. Some differences -to Streamdown: - -- **Better Memoization**: All inline components will preserve their state, even - as adjacent content changes. This is done to preserve remounts for things like - custom `` tags or other components. (For example, if a custom `` fetches - previewing data and provides a hover card, that card won't flicker). -- **Better Prediction**: While optimizing edge cases, the prediction system became - better than the Streamdown one on accident. -- **Headless UI**: No built in styles or components, bring your own CSS to blend - your markdown with your existing theme. `@clo/react-markdown` simply takes in - your existing `unified` pipeline and works off of that. -- **Easy to Audit**: Only ~1000 lines of modern TypeScript. 4 total files. - -[Streamdown]: https://streamdown.ai -[unified]: https://unifiedjs.com/ - -## Getting Started - -Install off of the [JSR package](https://jsr.io/clo/react-markdown): - -```sh -npx jsr add @clo/react-markdown -pnpm add jsr:@clo/react-markdown -``` - -```tsx -import { Markdown } from "@clo/react-markdown"; -import remarkGfm from "remark-gfm"; -import remarkParse from "remark-parse"; -import { unified } from "unified"; - -// Define any unified processing pipeline. We expect that your -// app already has one of these. @clo/react-markdown works with -// most pipelines, but some important things to note: -// - During parsing, all nodes have to have a source location. -// - The pipeline must output a remark (Markdown) AST. -// - You can also add `remarkRehype` and some rehype plugins. -// - Do not include the `rehypeReact` plugin in here. It is added for you. -const processor = unified().use(remarkParse).use(remarkGfm); - -// Define custom components. (maps to rehypeReact's option) -const components = { - a: MarkdownLink, - code: InlineCode, -}; - -export function HelloWorld() { - return ( - - ); -} -``` - -If you want to play with the example project, which offers a comparison to -Streamdown and the non-memoized `react-markdown` package. - -``` -git clone https://git.paperclover.net/clo/react-markdown -cd react-markdown -pnpm demo # auto-install -``` - -## Architecture - -The `Markdown` component is built out of a memoizer which uses the following tricks to improve performance: - -- Using proper `React.memo()` calls. Obviously. -- Prediction is implemented in a stateful way that only parses the tail end of - the document, marking how much of the document is stable and where possible - incomplete syntax may live. Since prediction only applies at the end, changing - text midway through can invalidate the whole predictor. -- Separate the parsed document into "blocks", noting the source location of - where each block lives. -- Only start parsing after the first changed character, rounded to the nearest - block. In the append-only stream, this essentially means the last two blocks - are the only things being re-parsed. -- Similarly, run AST transforms only on the changed data. This step has a couple - of slow paths for when reference link definitions are added or edited, since - it means any places that might have used a reference link may now have to - reflect it. -- After all that, a special AST -> React node transform is used that diffs the - new ast with the last ast, reusing React nodes whenever possible. It supports - nested children as well as re-ordering top level blocks. This is what prevents - most rerenders and is the "secret sauce". - -All put together, basically nothing rerenders except what actually changed, and the document is beautiful. diff --git a/readme.changes.md b/readme.changes.md new file mode 100644 index 0000000000000000000000000000000000000000..e594d8123d9c5707ba2746c55ed6f5fff480c662 --- /dev/null +++ b/readme.changes.md @@ -0,0 +1 @@ +# notable changes in React Markdown diff --git a/readme.md b/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..86e4c00547a42156b147e6b50aa245b9f371d276 --- /dev/null +++ b/readme.md @@ -0,0 +1,114 @@ +# `@clo/react-markdown` + +This package exports a React component to render Markdown using the [unified] +ecosystem's Markdown tools (remark, rehype). The exported `` +component is extremely memoized, making it suitable for streaming situations +such as LLM chat interfaces. + +- **Bring your Existing Pipeline**: The primary option for configuration is + providing a `unified.Processor` with remark and rehype plugins configured. + Additionally, you can override the element renderers with the `components` + attribute. By default, `@clo/react-markdown` applies a bare bones config so + you can get started with just ``. Configuration is + done either with a provider, or right at the component level. +- **Handle Partial Markdown**: When the `predict` prop is set, the document + will be treated as an incomplete stream at the end of a document. + `hello **world` will be emitted as `hello world`, perfect for + the LLM case. (standalone predictor exported as `@clo/react-markdown/Predict`) + +The motivation for this package is to have an easy to understand version of +[Streamdown]. With me banning all Vercel software at the company I work at, +we needed a simple, trustable solution for this problem. Some differences +to Streamdown: + +- **Better Memoization**: All inline components will preserve their state, even + as adjacent content changes. This is done to preserve remounts for things like + custom `` tags or other components. (For example, if a custom `` fetches + previewing data and provides a hover card, that card won't flicker). +- **Better Prediction**: While optimizing edge cases, the prediction system became + better than the Streamdown one on accident. +- **Headless UI**: No built in styles or components, bring your own CSS to blend + your markdown with your existing theme. `@clo/react-markdown` simply takes in + your existing `unified` pipeline and works off of that. +- **Easy to Audit**: Only ~1000 lines of modern TypeScript. 4 total files. + +[Streamdown]: https://streamdown.ai +[unified]: https://unifiedjs.com/ + +## Getting Started + +Install off of the [JSR package](https://jsr.io/clo/react-markdown): + +```sh +npx jsr add @clo/react-markdown +pnpm add jsr:@clo/react-markdown +``` + +```tsx +import { Markdown } from "@clo/react-markdown"; +import remarkGfm from "remark-gfm"; +import remarkParse from "remark-parse"; +import { unified } from "unified"; + +// Define any unified processing pipeline. We expect that your +// app already has one of these. @clo/react-markdown works with +// most pipelines, but some important things to note: +// - During parsing, all nodes have to have a source location. +// - The pipeline must output a remark (Markdown) AST. +// - You can also add `remarkRehype` and some rehype plugins. +// - Do not include the `rehypeReact` plugin in here. It is added for you. +const processor = unified().use(remarkParse).use(remarkGfm); + +// Define custom components. (maps to rehypeReact's option) +const components = { + a: MarkdownLink, + code: InlineCode, +}; + +export function HelloWorld() { + return ( + + ); +} +``` + +If you want to play with the example project, which offers a comparison to +Streamdown and the non-memoized `react-markdown` package. + +``` +git clone https://git.paperclover.net/clo/react-markdown +cd react-markdown +pnpm demo # auto-install +``` + +## Architecture + +The `Markdown` component is built out of a memoizer which uses the following tricks to improve performance: + +- Using proper `React.memo()` calls. Obviously. +- Prediction is implemented in a stateful way that only parses the tail end of + the document, marking how much of the document is stable and where possible + incomplete syntax may live. Since prediction only applies at the end, changing + text midway through can invalidate the whole predictor. +- Separate the parsed document into "blocks", noting the source location of + where each block lives. +- Only start parsing after the first changed character, rounded to the nearest + block. In the append-only stream, this essentially means the last two blocks + are the only things being re-parsed. +- Similarly, run AST transforms only on the changed data. This step has a couple + of slow paths for when reference link definitions are added or edited, since + it means any places that might have used a reference link may now have to + reflect it. +- After all that, a special AST -> React node transform is used that diffs the + new ast with the last ast, reusing React nodes whenever possible. It supports + nested children as well as re-ordering top level blocks. This is what prevents + most rerenders and is the "secret sauce". + +All put together, basically nothing rerenders except what actually changed, and the document is beautiful.