While adding highlight.js I found my home page had been loading lodash for over a year
After adding code block highlighting I checked the home page requests and found the 71KB highlight.js used only on post pages. The cause was Rollup's CommonJS helper, and for the same reason lodash had been loading on the home page for over a year.
#Vite #Build #Performance #React #Frontend
A few days ago I added syntax highlighting to the code blocks in my blog posts. With a language label, line numbers, a wrap button and a copy button, posts became a lot easier to read. But before deploying, while checking what the home page actually downloads as I always do, I noticed something odd. Highlighting is only used on the post detail page, yet the home page was also loading the highlight.js chunk. Digging in, it turned out not to be a highlighting problem. I also found out that for the same reason the home page had been loading lodash for over a year. In this post I go over what caused it, and why it's a problem you will never find by looking at the build output alone. Why is a library used only on post pages on the home page Code blocks are stored in the DB without any language info, so at render time I guess the language from the content and color it with highlight.js. That code lives only in ContentBody , which draws the post body, and the post detail route is loaded separately with lazy() . So the home page has no reason at all to load highlight.js. But when I opened the built index.html , the very first line of the preload list was highlight.js. link rel="modulepreload" crossorigin href="/assets/highlight.js-e6918b47.js" link rel="modulepreload" crossorigin href="/assets/react-f12e2e30.js" link rel="modulepreload" crossorigin href="/assets/scheduler-765c72db.js" Being in modulepreload means Vite decided it is needed to draw the first screen. I assumed I had statically imported highlight.js somewhere and went through the whole source, but there was no such code. You only see it in the imports at the top of a chunk If it isn't in the source, it's in the build output. I opened the import statements at the top of each chunk file one by one. // react-dom chunk import{g as fa}from"./highlight.js-e6918b47.js" // lottie-web chunk import{g as getDefaultExportFromCjs,c as commonjsGlobal}from"./highlight.js-e6918b47.js" // quill chunk import{c as vt}from"./highlight.js-e6918b47.js" react-dom was importing something called g from the highlight.js chunk. react-dom has no reason to use syntax highlighting, so the question was what that g was. On the lottie-web side the names weren't minified, so it was obvious right away: getDefaultExportFromCjs and commonjsGlobal . These two are helpers Rollup uses when it converts CommonJS packages into ES modules. react-dom, lottie-web, quill and highlight.js are all published as CommonJS, so they all use this helper. The problem was that this helper had ended up inside the highlight.js chunk. So to use react-dom, the whole highlight.js chunk had to be downloaded. Why did the helper end up in the highlight.js chunk of all places The blog's chunk split rule was simple. Anything under a node_modules path gets a chunk named after its package. manualChunks(id) { if (id.includes('node_modules')) { return id.toString().split('node_modules/')[1].split('/')[0].toString(); } } But the CommonJS helper isn't a real file. It's a virtual module generated by a plugin. Inside Vite its id is \0commonjsHelpers.js , which doesn't contain the text node_modules . So it doesn't match my rule and is left as a module with no owner. Rollup doesn't leave ownerless modules alone. It goes through the chunks you assigned one by one and puts any dependency of that chunk that has no owner yet into the same chunk. And it does this walk in alphabetical order of chunk names . Sorting the CommonJS package chunk names gives this. highlight.js lodash lottie-web quill react react-dom h comes first. So the highlight.js chunk grabbed the helper first, and every other chunk that needs the helper ended up importing the highlight.js chunk. react-dom is essential for the first screen, so highlight.js went onto the first screen list along with it. To confirm it really was the name order, I renamed only the highlight.js chunk to zz-highlight and built again. The helper moved to the lodash chunk. Only after seeing the build output change from renaming one chunk was I convinced. So where was it before That led to one more thing. Even before highlight.js existed, the helper must have been living somewhere. I built the commit right before highlighting was added. // build of the commit before highlighting, react-dom chunk import{g as fa}from"./lodash-c86a663f.js" // index.html link rel="modulepreload" crossorigin href="/assets/lodash-c86a663f.js" It was in lodash. Before highlight.js arrived, lodash came first alphabetically, so that's exactly what you'd expect. The blog's own code doesn't even use lodash. It's just a dependency pulled in by react-quill, the editor on the admin writing screen. Even so, every visitor to the home page was downloading the lodash chunk (13.23KB, 5.32KB gzipped). I split chunks by package last June, so it had been like that for over a year. It was small, so nobody, including me, ever noticed. highlight.js only stood out because it's 71KB. The fix was one line Give the helper its own chunk. Once it has an owner, Rollup doesn't slip it into another chunk. manualChunks(id) { // Rollup's CommonJS helper isn't under a node_modules path, so it misses the rule // and lands on whichever package chunk sorts first. Move it to its own small chunk. if (id.includes('commonjsHelpers')) return 'commonjs-helpers'; if (id.includes('node_modules')) { return id.toString().split('node_modules/')[1].split('/')[0].toString(); } } The new commonjs-helpers chunk is 0.24KB. Two helper functions are all it has. Now react-dom, lottie-web and quill all take the helper from this small chunk. // react-dom chunk after the fix import{g as fa}from"./commonjs-helpers-725317a4.js" The number of chunks referencing the highlight.js chunk also dropped a lot. Before the fix, 25 chunks referenced highlight.js, from react, react-dom, lodash, lottie-web and quill to route chunks like login and the guestbook. After the fix only 4 remain: the 3 on the post detail and admin write preview side that actually use highlighting, and the entry chunk that holds the lazy import list. I counted the JS the home page downloads with playwright. To compare under the same conditions, this is from a local server running without compression. Home JS (uncompressed) files total highlight.js lodash without the helper rule 17 691,297 B loaded not loaded with the helper rule 17 620,702 B not loaded not loaded In terms of what the production server actually transfers, the home page JS is now 187KB, and highlight.js (22.8KB) is only downloaded once you open a post. Without the rule, the home page JS would have grown by about 12%. I also registered only the languages I use There was one more thing I paid attention to while adding highlighting. If you just import highlight.js with import hljs from 'highlight.js' , all 192 supported languages come along. Import uncompressed gzip full highlight.js (192 languages) 1,081,310 B 311,671 B lib/core only 21,130 B 8,504 B lib/core + my 11 languages 71,335 B 22,573 B Importing everything is about 14 times bigger even gzipped. The code in my posts is roughly JS, TS, HTML, terminal, SQL, JSON, CSS, Apache, Nginx, PHP and YAML, so I registered only those. import hljs from 'highlight.js/lib/core'; import javascript from 'highlight.js/lib/languages/javascript'; import typescript from 'highlight.js/lib/languages/typescript'; import bash from 'highlight.js/lib/languages/bash'; // ... xml, sql, json, css, apache, nginx, php, yaml hljs.registerLanguage('javascript', javascript); hljs.registerLanguage('typescript', typescript); hljs.registerLanguage('bash', bash); Sizes differ quite a bit by language too. CSS is the biggest at 13KB and JSON is 0.4KB. If I end up writing about Go or Python, I can add them one at a time then. You can't find it from the build output What makes this tricky is that there's no signal at all. The build shows no warning, and chunk sizes barely change. The highlight.js chunk with the helper was 70.98KB, and 70.80KB without it. If I had only been looking at the size table in the build log, I would never have found it. There were only three places it showed up: the import statements at the top of chunk files, the modulepreload list in index.html , and the list of files the browser actually requested. Ever since I found my home page was downloading 19.7MB , I've had the habit of checking the browser's request list before deploying, and that's how I caught it. And this isn't something you fix once and forget. If you add a new CommonJS package whose name sorts earlier, the helper can move there again. It's fine for now since the helper has its own chunk, but if I change the chunk split rules or the bundler changes, like Rolldown in Vite 8 , I'll need to check again. I wrote up how I first decided on manualChunks in an earlier post . If you split chunks per package with manualChunks, I recommend opening the imports at the top of your entry chunk or react-dom chunk once. If you see an unrelated package name there, the same thing is happening to you.