Enhancement/esbuild mode

Dockerfile

@@ -9,7 +9,7 @@ FROM --platform=$TARGETPLATFORM node:lts-alpine as builder
 
 WORKDIR /app
 
-# Install only production dependencies (yarn)
+# Install only production dependencies
 COPY --from=deps /app/node_modules ./node_modules
 COPY . .
 
@@ -26,4 +26,4 @@ COPY --from=builder /app .
 EXPOSE 8080
 
 ENTRYPOINT ["/sbin/tini", "--"]
-CMD ["node", "server.js"]
+CMD ["node", "server.js"]

README.md

@@ -26,6 +26,8 @@ The application requires a configuration file `./config.json` to be able to run.
 | cleanThreshold | The amount of downloaded branches that will trigger the clean up job. Defaults to 1000 |
 | tmpLifetime | How many hours since last request to keep a branch when cleaning up |
 
+**Note on esbuild mode**: esbuild compilation requires no additional configuration. The feature is enabled by the `?esbuild` query parameter and uses the same cache cleanup settings as standard builds, but with separate `output-esbuild/` directories.
+
 Example config file:
 ```json
 {
@@ -52,17 +54,93 @@ Open a CLI and run the command: `npm start`
 Open `http://localhost:80` in a browser and you should see the index page of the app.
 It is possible to configure which port the application listens to, see [Configure settings](#configure-settings).
 
+## Usage
+
+### Basic Examples
+
+```
+# Master branch
+https://github.highcharts.com/master/highcharts.src.js
+
+# Version tag
+https://github.highcharts.com/v10.3.3/highcharts.src.js
+
+# Commit SHA (full or short)
+https://github.highcharts.com/abc1234/highcharts.src.js
+
+# Feature branch
+https://github.highcharts.com/feature/my-branch/highcharts.src.js
+
+# Modules
+https://github.highcharts.com/master/modules/exporting.src.js
+
+# Stock/Maps/Gantt
+https://github.highcharts.com/master/highstock.src.js
+https://github.highcharts.com/master/highmaps.src.js
+https://github.highcharts.com/master/highcharts-gantt.src.js
+```
+
+### esbuild Mode
+
+Add `?esbuild` to any request to use esbuild compilation instead of the standard TypeScript + assembler pipeline:
+
+```
+# esbuild compilation
+https://github.highcharts.com/master/highcharts.src.js?esbuild
+https://github.highcharts.com/v11.4.0/modules/exporting.src.js?esbuild
+https://github.highcharts.com/feature/my-branch/highstock.src.js?esbuild
+```
+
+**Benefits:**
+- **Faster compilation**: esbuild typically 10-100x faster than TypeScript compiler
+- **Same output format**: UMD bundles compatible with browsers, AMD, and CommonJS
+- **Separate caching**: Uses `output-esbuild/` directory to avoid conflicts
+- **Easy identification**: `X-Built-With: esbuild` response header indicates esbuild compilation
+- **Error handling**: Compilation errors return JavaScript with helpful `console.error()` messages
+- **Legacy compatibility**: Automatic support for older Highcharts versions via plugins
+- **Performance logging**: Console output shows compilation time for debugging
+
+**Response headers for esbuild requests:**
+```
+X-Built-With: esbuild
+ETag: {commit-sha}
+```
+
+**Technical Implementation:**
+- **Primary files** (e.g., `highcharts.src.js`, `highstock.src.js`) get full UMD wrappers
+- **Module files** (e.g., `modules/exporting.src.js`) get dependency-aware UMD wrappers
+- **ES modules support**: Files in `/es-modules/` paths receive ES module treatment
+- **Version detection**: Automatic legacy plugin application for versions < 11.2.0
+- **Namespace mapping**: Smart replacement of core dependencies to use existing globals
+- **Error resilience**: Compilation failures return executable JavaScript with error logging
+
+**Dependencies:**
+- `esbuild` ^0.25.0 - Core compilation engine
+- `esbuild-plugin-replace-regex` ^0.0.2 - Legacy compatibility patches
+- `semver` ^7.6.0 - Version detection for compatibility features
+
+### Supported Build Modes
+
+- **Classic builds** (default): TypeScript → Assembler → UMD bundles
+- **Webpack builds**: Detected when `tsconfig.json` has `"outDir": "code/es-modules/"`
+- **esbuild builds**: Use `?esbuild` query parameter for faster compilation with esbuild
+
 ## Code documentation
 Each file contains a descriptive header, mentioning its author, purpose and so on. Every function should contain a descriptive JSDoc header.
 
 ### File Structure
+
 | Path | Description |
 |---|---|
 | app | Contains all the application JS code. |
+| app/esbuild.js | esbuild compilation engine for faster builds with UMD wrapper generation |
 | assets | Contains assets like CSS, images, etc. |
 | scripts | Tooling scripts used for deployment and such. Should not be deployed with the application. |
 | test | Contains all the unit-tests for the application. Should not be deployed with the application. |
+| test/esbuild.js | Unit tests for esbuild compilation functionality |
 | tmp | Where the temporary files used in the application is written. |
+| tmp/{branch}/output | Final assembled files (classic builds) |
+| tmp/{branch}/output-esbuild | esbuild compiled files (separate cache) |
 | static | Where the HTML files are located. |
 
 ## Update the Highcharts assembler
@@ -91,6 +169,43 @@ Open a CLI and run the following command:
 `npm run build`
 The application will be packed into an archive named `github.highcharts-<version>.zip`. The zip is ready to be uploaded and unpacked on your server.
 
+## Build Process
+
+The application supports multiple build modes to compile TypeScript source files into JavaScript bundles:
+
+### Standard Build Process (Default)
+
+1. **Download**: Downloads TypeScript source files from GitHub for the specified branch/commit
+2. **TypeScript Compilation**: Compiles `.ts` files to JavaScript using TypeScript compiler
+3. **Assembly**: Uses `@highcharts/highcharts-assembler` to create UMD bundles with dependencies
+4. **Caching**: Results are cached to speed up subsequent requests
+
+### esbuild Process (with `?esbuild`)
+
+1. **Download**: Downloads TypeScript source files from GitHub for the specified branch/commit
+2. **esbuild Compilation**: Compiles TypeScript directly to JavaScript with UMD wrappers
+3. **Module Resolution**: Applies post-processing for module compatibility and namespace mappings
+4. **Caching**: Results are cached in separate `output-esbuild/` directory
+
+**Key differences:**
+- **Performance**: esbuild is significantly faster than tsc + assembler
+- **Direct compilation**: TypeScript → JavaScript compilation with built-in bundling
+- **UMD compatibility**: Same output format for browser/AMD/CommonJS compatibility
+- **Isolated cache**: Separate cache to avoid conflicts with standard builds
+- **Legacy support**: Compatibility for older Highcharts versions via plugins
+
+**esbuild-specific features:**
+- **Smart path resolution**: Automatic mapping of master file paths (e.g., `/es-modules/masters` → TypeScript sources)
+- **Dual UMD modes**: Primary files get full UMD wrappers, modules get dependency-aware wrappers
+- **Namespace injection**: Runtime replacement of core modules with existing global references
+- **Version-aware processing**: Automatic legacy patches for TypeScript syntax changes
+- **Build context preservation**: Maintains version strings and asset prefixes during compilation
+- **Error recovery**: Failed compilations return executable error-reporting JavaScript
+
+### Webpack Builds
+
+Automatically detected when `tsconfig.json` has `"outDir": "code/es-modules/"`. Uses webpack for module bundling.
+
 ## Nice to know
 The application does not do a full clone of the `highcharts` repo. It fetches only certain folders within that repo.
 The files are downloaded via GitHub Contents API and stored in a local folder per branch/tag/ref the first time a particular branch/tag/ref is requested. For every subseqent request the local version will be used, which, means that the state of a particular branch won't be updated unless the application is redeployed.
@@ -101,4 +216,25 @@ Note that TypeScript files are not being served by this application.
 For bundling master files, or custom files the `highcharts-assembler` is used.
 
 ### Troubleshooting
-The temporary folder `tmp/` folder may become bloated or have a partial state if something goes wrong. This may cause unexpected behaviour. If you experience something similar then try to delete everything in the  `tmp/` folder and retry your request.
+
+**Build Failures:**
+- Check if the branch/commit exists on GitHub
+- Verify the file path exists in that branch
+- Check server logs for compilation errors
+- Some older commits may not have TypeScript sources
+- For esbuild mode: compilation errors are returned as JavaScript with `console.error()` messages
+
+**esbuild-specific troubleshooting:**
+- **Legacy version issues**: Versions < 11.2.0 automatically receive compatibility patches
+- **Missing dependencies**: Check browser console for namespace resolution errors
+- **UMD wrapper problems**: Verify that primary files use correct global names (`Highcharts`, `Dashboards`, etc.)
+- **Performance debugging**: Check browser console for compilation time logs
+- **Module conflicts**: esbuild cache is isolated in `output-esbuild/` to prevent cross-contamination
+
+**Cache Issues:**
+The temporary folder `tmp/` folder may become bloated or have a partial state if something goes wrong. This may cause unexpected behaviour. If you experience something similar then try to delete everything in the `tmp/` folder and retry your request.
+
+**Performance:**
+- Use `?esbuild` for faster compilation times
+- esbuild cache is separate from standard builds (`output-esbuild/` vs `output/`)
+- Both build modes support the same file types and UMD output format

app/esbuild.js

@@ -0,0 +1,112 @@
+// @ts-check
+/**
+ * Esbuild-based compile-on-demand functionality.
+ * Uses @highcharts/highcharts-utils/lib/compile-on-demand-core.js for core compilation.
+ *
+ * @module esbuild
+ */
+
+const { join } = require('node:path')
+const { readFile, writeFile, mkdir } = require('node:fs/promises')
+const { existsSync } = require('node:fs')
+const { log } = require('./utilities')
+
+/**
+ * Cached promise for the dynamically imported compile-on-demand-core module
+ * @type {Promise<typeof import('@highcharts/highcharts-utils/lib/compile-on-demand-core.js')> | null}
+ */
+let coreModulePromise = null
+
+/**
+ * Dynamically import the compile-on-demand-core module (ESM)
+ * @returns {Promise<typeof import('@highcharts/highcharts-utils/lib/compile-on-demand-core.js')>}
+ */
+async function getCompileCore () {
+  if (!coreModulePromise) {
+    coreModulePromise = import('@highcharts/highcharts-utils/lib/compile-on-demand-core.js')
+  }
+  return coreModulePromise
+}
+
+/**
+ * Get package version from package.json in the source directory
+ * @param {string} highchartsDir - Path to the Highcharts source directory
+ * @returns {Promise<string>} The package version
+ */
+async function getPackageVersion (highchartsDir) {
+  try {
+    const packageJsonPath = join(highchartsDir, 'package.json')
+    const packageJson = JSON.parse(await readFile(packageJsonPath, 'utf-8'))
+    return packageJson.version || '0.0.0'
+  } catch (e) {
+    return '999.0.0' // Default to a high version to skip legacy plugins
+  }
+}
+
+/**
+ * Compile a file and write it to the output directory
+ * @param {string} branch - The branch/commit SHA
+ * @param {string} requestFilename - The requested filename (e.g., 'highcharts.src.js')
+ * @returns {Promise<{file?: string, body?: string, status: number}>} Result with file path or error
+ */
+async function compileWithEsbuild (branch, requestFilename) {
+  const core = await getCompileCore()
+
+  const pathCacheDirectory = join(__dirname, '../tmp', branch)
+  const outputDir = join(pathCacheDirectory, 'output-esbuild')
+
+  // Normalize the filename to have a leading slash for the compile function
+  const normalizedFilename = requestFilename.startsWith('/')
+    ? requestFilename
+    : `/${requestFilename}`
+
+  // Output file path
+  const outputFilePath = join(outputDir, requestFilename)
+
+  // Check if already compiled
+  if (existsSync(outputFilePath)) {
+    log(0, `Serving cached esbuild file: ${outputFilePath}`)
+    return { file: outputFilePath, status: 200 }
+  }
+
+  // Ensure output directory exists (use try/catch to handle race conditions)
+  const outputFileDir = join(outputDir, ...requestFilename.split('/').slice(0, -1))
+  try {
+    await mkdir(outputFileDir, { recursive: true })
+  } catch (error) {
+    // EEXIST is fine; directory already exists (race condition with parallel requests)
+    if (error.code !== 'EEXIST') {
+      throw error
+    }
+  }
+
+  log(0, `Compiling with esbuild: ${normalizedFilename} for ${branch}`)
+
+  /** @type {{code: string, duration: number}} */
+  let result
+  try {
+    result = await core.compile(normalizedFilename, {
+      highchartsDir: pathCacheDirectory,
+      branchName: branch,
+      getPackageVersion: () => getPackageVersion(pathCacheDirectory)
+    })
+
+    // Write to output directory
+    await writeFile(outputFilePath, result.code, 'utf-8')
+
+    log(0, `esbuild compilation complete: ${normalizedFilename} (${result.duration}ms)`)
+
+    return { file: outputFilePath, status: 200 }
+  } catch (error) {
+    log(2, `esbuild compilation failed: ${error.message}`)
+    const errorJs = `console.error('esbuild compilation failed: ${error.message.replace(/'/g, "\\'")}');`
+    await writeFile(outputFilePath, errorJs, 'utf-8')
+    return { file: outputFilePath, status: 200 }
+  }
+}
+
+module.exports = {
+  compileWithEsbuild,
+  // Re-export from core for any external usage
+  getCompileCore
+}

app/handlers.js

@@ -38,6 +38,7 @@ const directoryTree = require('directory-tree')
 const { JobQueue } = require('./JobQueue')
 const { existsSync } = require('node:fs')
 const { shouldUseWebpack, compileWebpack } = require('./utils.js')
+const { compileWithEsbuild } = require('./esbuild.js')
 
 // Constants
 const PATH_TMP_DIRECTORY = join(__dirname, '../tmp')
@@ -143,6 +144,9 @@ async function handlerDefault (req, res) {
     return respondToClient(result, res, req)
   }
 
+  // Check for esbuild mode
+  const useEsbuild = /[?&]esbuild(?:[&=]|$)/.test(req.url)
+
   let branch = await getBranch(req.path)
   let url = req.url
   let useGitDownloader = branch === 'master' || /^\/v[0-9]/.test(req.path) // version tags
@@ -168,6 +172,16 @@ async function handlerDefault (req, res) {
     }
   }
 
+  // Handle esbuild mode
+  if (useEsbuild) {
+    const result = await serveEsbuildFile(branch, url, useGitDownloader)
+    if (!res.headersSent) {
+      res.header('ETag', branch)
+      res.header('X-Built-With', 'esbuild')
+    }
+    return respondToClient(result, res, req)
+  }
+
   // Serve a file depending on the request URL.
   // Try to serve  a static file.
   let result = await serveStaticFile(branch, url)
@@ -610,6 +624,59 @@ async function serveStaticFile (branch, requestURL) {
   return applyRateLimitMeta({ status: 200, file: pathFile }, rateLimitMeta)
 }
 
+/**
+ * Interprets the request URL and serves a file compiled with esbuild.
+ * Downloads the source files if needed, then compiles with esbuild.
+ * The Promise resolves with an object containing information on the response.
+ *
+ * @param {string} branch The branch/commit SHA
+ * @param {string} requestURL The url which the request was sent to.
+ * @param {boolean} useGitDownloader Whether to use the git downloader
+ */
+async function serveEsbuildFile (branch, requestURL, useGitDownloader = true) {
+  const type = getType(branch, requestURL)
+  const file = getFile(branch, type, requestURL)
+
+  // Respond with not found if the interpreter can not find a filename.
+  if (file === false) {
+    return response.missingFile
+  }
+
+  const pathCacheDirectory = join(PATH_TMP_DIRECTORY, branch)
+  const tsMastersDirectory = join(pathCacheDirectory, 'ts', 'masters')
+
+  // Check if source files are downloaded
+  const isAlreadyDownloaded = exists(tsMastersDirectory)
+
+  if (!isAlreadyDownloaded) {
+    const maybeResponse = await queue.addJob(
+      'download',
+      branch,
+      {
+        func: downloadSourceFolder,
+        args: [
+          pathCacheDirectory, URL_DOWNLOAD, branch
+        ]
+      }
+    ).catch(error => {
+      if (error.name === 'QueueFullError') {
+        return { status: 202, body: error.message }
+      }
+
+      log(2, `Queue addJob failed for ${branch}: ${error.message}`)
+      return { status: 500, body: 'Download failed' }
+    })
+
+    if (maybeResponse.status && maybeResponse.status !== 200) {
+      return maybeResponse
+    }
+  }
+
+  // Compile with esbuild
+  const result = await compileWithEsbuild(branch, file)
+  return result
+}
+
 function printTreeChildren (children, level = 1, carry) {
   return children.reduce((carry, child) => {
     let padding = ''