Compatibility suites
Acropolis is tested against other builders' published test suites, not only its own apps.
We don’t trust our own test apps: we use other builders’ tests as they publish them. Each case builds the image with Acropolis, runs it with docker run, and checks the output or an HTTP request, with the same timeouts as the original. Results as of v0.3.1:
| suite | what it is | result |
|---|---|---|
| Railpack examples | 131 apps and 156 cases: Node (npm, pnpm, yarn, bun, Next, Nuxt, Astro, SvelteKit, Remix, Angular, Nx, Turborepo), Python (pip, uv, poetry, pdm), Go, Rust, Ruby/Rails, PHP/Laravel, Java, .NET, Elixir, Gleam, Deno, static sites and scripts | 149 pass; 2 are for arm64 and are skipped on x86; the other 5 fail with v0.2.1 too since 2026-10-13 (the Ruby images’ apt-get hits expired Debian Release files, and greenlet 3.2.4 has no wheel for the Python it gets and wants g++) |
Next.js and Turbopack (tests/suites/next) |
Next 15 with --turbopack, Next 16 with Turbopack by default, --webpack, output: standalone, output: export, next/image, file reads at runtime and a Turborepo monorepo; with npm, pnpm, yarn and bun |
8/8 |
Workspaces (tests/suites/workspaces) |
A member that depends on a member whose devDependency is a third member with only a tsconfig base, with bun, pnpm, npm, yarn 1 and yarn berry |
5/5 |
Next.js deploy tests (tests/suites/next-deploy) |
Next’s own harness with NEXT_TEST_MODE=deploy: 40 files spread over test/e2e/app-dir, built with Acropolis and run with docker; Next 15.5.27 (webpack) and 16.4.0 (Turbopack) |
15.5.27: 48 of 49 tests (35 of 36 files that run); 16.4.0: 154 of 160 tests (22 of 27 files); the rest are skipped in deploy mode |
TechEmpower (scripts/techempower-suite.py) |
297 frameworks in the languages we build, without their Dockerfile, checking /json and /plaintext |
55 pass; 98 fail by design (system packages or a runtime built from source in the Dockerfile, PHP extensions outside composer.json, Swoole/Workerman-style servers, C/C++ without CMake or Meson), 31 are tagged broken upstream, 13 need TechEmpower’s database at startup |
Heroku getting-started and Paketo samples (scripts/smoke-suite.py) |
6 Heroku apps and 43 Paketo samples (Java, Node, Go, Python, Ruby, PHP, .NET, static) | 22 pass; 7 fail by design (prebuilt .NET binaries, apps without package.json, requirements.txt or composer.json) |
Nixpacks tests (scripts/nixpacks-suite.py) |
73 cases converted from tests/docker_run_tests.rs |
37 pass; the rest are languages Railpack doesn’t support either (Clojure, Crystal, Dart, Haskell, Scala, Scheme, Swift, Zig), Nixpacks-specific configs (nixpacks.toml, NIXPACKS_*), or apps that expect the compiler inside the final image |
Plan snapshots (tests/plans) |
acropolis plan --json for the 131 Railpack examples |
any detection change shows up as a diff before anything is built |
| Unit tests | package installation, layers, sandbox, registry, paths that escape the app, watch paths, errors | 132 |
What they found
These suites found bugs our own apps would never have shown: yarn v1 installing binaries for every platform (177 MB of sharp for darwin, windows and arm in a linux image), node_modules of pnpm workspace members not reaching the image, Next 15 needing typescript at runtime to read next.config.ts, patchedDependencies patches not being applied, duplicate hardlinks inflating layers, Go without go.mod, and Java with an old Gradle. Later ones: yarn berry workspace members without node_modules in the image, apps without a lockfile sharing each other’s cached node_modules layer and getting the musl builds of @next/swc and sharp (41 MB more per Next image), node-gyp missing when another plan had installed the same Node without npm, and layers with a path written twice, which docker pull refuses on overlay2.
Images are tested with docker pull from a local registry, and every layer of every image (base layers too) is checked for duplicate paths. With E2E_OCI=1 the harness also loads the --oci tar with docker load, which is how Olimpia consumes them.