Skip to main content

Screenshots, smoke tests and benchmarks

The repository has a few tools that keep the documentation and the performance numbers honest: tools/Talesmith.Screenshots renders every editor screenshot on this site and runs the editor stress benchmark, tools/smoke/editor-smoke.sh drives the real editor in an X11 window, benchmarks/Talesmith.Benchmarks holds the micro-benchmarks and the scale scenes, and the player has a headless benchmark mode. This page explains each one and how to read its output.

Talesmith.Screenshots​

The tool hosts the editor and the toolkit in headless Avalonia, renders each scene in the dark and light themes and saves the images. It never opens a window.

dotnet run -c Release --project tools/Talesmith.Screenshots                     # every scene, as PNG, into artifacts/screenshots
dotnet run -c Release --project tools/Talesmith.Screenshots -- --list           # list the scenes
dotnet run -c Release --project tools/Talesmith.Screenshots -- dock toolkit     # only scenes whose name contains a filter
dotnet run -c Release --project tools/Talesmith.Screenshots -- --output /tmp/shots --scale 1
OptionEffect
<filter> …Renders only scenes whose name contains one of the filters, ignoring case. With --docs, filters match docs names such as guide/project-hub.
--listLists the scene names.
--output <folder>, -oWhere PNG files go; artifacts/screenshots by default.
--scale <n>Pixels per logical unit, as on a high-DPI display; 2 by default, so images are twice their logical size. Layout is the same at every scale, so lists and popups show as they do in the editor.
--docs <folder>Renders the documentation screenshots as WebP into <folder>/screenshots/<section>/ and updates each section's manifest.json. npm run screenshots passes static/img.
--docs-listLists every documentation screenshot with its section.
--shortcuts <file>Writes the editor's registered commands and shortcuts as JSON. npm run shortcuts uses it for this site's shortcut tables.
--perf [columns rows entities]Runs the editor stress benchmark.

Scenes​

A scene derives from ScreenshotScene (Capture/ScreenshotScene.cs): Name, Size (1280 × 800 by default), Build to create the content, Prepare to get it into the right state (open a flyout, start a drag, select an entity), Region to crop to one control and Release to free what Build created, such as an open editor. One process renders every screenshot, so a scene that keeps its editor alive adds to the memory of the whole run. EditorWindowScene starts from the full editor with a project open.

EditorFixture creates template projects in a temporary folder and opens them in a headless editor. The samples are copied there too (EditorFixture.IsleHopper, EditorFixture.HexQuest), so opening them never writes .meta files or anything else into samples/. Scenes for this site are listed per section in Docs/GuideShots.cs, ScriptingShots.cs, PluginsShots.cs, DevelopersShots.cs and HomeShots.cs, each entry a name, a scene and default alt text.

Adding a documentation screenshot​

  1. Write a scene in tools/Talesmith.Screenshots/Scenes/, in a file for your section, or reuse an existing one.
  2. Register it in your section's list in tools/Talesmith.Screenshots/Docs/, with a kebab-case name and alt text that describes what the image shows.
  3. Render it from website/: npm run screenshots -- developers/my-shot.
  4. Use it in a page: <Screenshot name="developers/my-shot" />.

website/README.md has the details, and website/STYLE.md the rules for what a screenshot should show. Each image is added to its manifest as soon as it is saved, so an interrupted run loses nothing. A run deletes images of screenshots that are no longer listed, so never put hand-made images in static/img/screenshots. To render them all on a CI runner and get a pull request with the changes, use the Screenshots workflow.

The editor stress benchmark​

--perf generates a project with a hex map of columns × rows cells and entities entities (sprites, lights and particle emitters in groups of 100), opens it in a headless editor with a 1600 × 1000 window and measures how long each interaction keeps the UI thread busy:

dotnet run -c Release --project tools/Talesmith.Screenshots -- --perf 2000 1000 5000 > perf.log 2>&1
TALESMITH_PERF_ONLY=select dotnet run -c Release --project tools/Talesmith.Screenshots -- --perf 2000 1000 5000 > perf.log 2>&1

TALESMITH_PERF_ONLY runs only the named sections (idle, select, reveal, scroll, pan, paint), each five times as often, which helps when you work on one interaction. The project is generated in a temporary folder and deleted afterwards.

Reading the output​

Each line is one interaction, with the median, 95th percentile and maximum in milliseconds and, at the end, how often it ran:

ColumnMeaning
UI threadFrom the input until its work ran, the window was laid out and a frame was drawn.
Before the frameThe part spent in the interaction's own handlers and layout. This is what your change usually affects.
Game tickThe edit game's frames that ran during the interaction.
Scene renderDrawing the edit game's frame, reported separately and not counted, because in the real editor it runs on the render thread.

Headless Avalonia renders the window in software and copies each frame into a newly allocated framebuffer, so every frame has a fixed cost that a window drawn on the GPU does not have: about 9 ms to redraw a status text, 15 ms for the viewport and 25 ms for every control. The idle rows measure that cost. Panning, zooming and painting spend less than 1 ms before the frame, so almost all of their total is this fixed cost. Compare before the frame across runs, not totals against a real window. The Performance page has numbers measured in a real window.

The editor smoke test​

tools/smoke/editor-smoke.sh starts the real editor in an X11 window under Xvfb, drives it with xdotool and saves a screenshot after each step: it creates a project from a template, selects an entity in the hierarchy, moves it with the Move tool, undoes, plays and stops, searches the command palette and closes the editor. It fails when the editor exits early, does not close, or its log shows a crash.

nix shell nixpkgs#xvfb-run nixpkgs#xdotool nixpkgs#imagemagick --command tools/smoke/editor-smoke.sh
tools/smoke/editor-smoke.sh artifacts/smoke hex-adventure
ArgumentDefault
Output folderartifacts/smoke
Templateplatformer; also empty or hex-adventure

It needs dotnet, xvfb-run, xdotool and ImageMagick's import, builds src/Talesmith.App in Debug when needed, and runs with a temporary HOME, so your editor settings are not touched. Positions in the script assume the default layout at 1600 × 1000; if you change the default layout, update hero_row and viewport in the script. Look at the screenshots, not only the exit code: a panel that renders wrong does not fail the test.

Micro-benchmarks​

benchmarks/Talesmith.Benchmarks is a BenchmarkDotNet project for measuring single pieces in isolation:

dotnet run -c Release --project benchmarks/Talesmith.Benchmarks -- --list flat
dotnet run -c Release --project benchmarks/Talesmith.Benchmarks -- --filter '*ParticleBenchmarks*'
dotnet run -c Release --project benchmarks/Talesmith.Benchmarks -- --filter '*PhysicsStep*' --job short
ClassParametersMeasures
ParticleBenchmarks10,000 or 100,000 particles; Simple or Full modulesSimulate: one step; BuildSprites: writing the particles' sprite instances
PhysicsStepBenchmarks1,000 or 4,000 bodies; Pile, Swarm or Tiles; dynamic tree or spatial hash broadphaseFixedStep: one physics step
PhysicsQueryBenchmarksDynamic tree or spatial hashClosestRays, AllRays and CircleOverlaps

--job short gives rough numbers in a few minutes; drop it for publishable ones. CI runs every benchmark with the short job each night and compares it with the recent runs; see Benchmarks. Results and logs go to BenchmarkDotNet.Artifacts in the current folder, or the folder passed with --artifacts. The Performance page lists recent results.

The scale scenes​

The same project writes two game folders that put everything under load at once:

dotnet run -c Release --project benchmarks/Talesmith.Benchmarks -- scale-scene artifacts/scale
dotnet run -c Release --project src/Talesmith.Player -- artifacts/scale/scale --benchmark --frames 600 --renderer vulkan
dotnet run -c Release --project src/Talesmith.Player -- artifacts/scale/scale-flyover --benchmark --frames 600 --renderer vulkan
  • scale: 10,000 sprites (half sorted by Y, on four layers), ten particle emitters holding about 47,500 particles, and 32 flickering lights that all cast shadows, over a 1500 × 1400 hex map (2.1 million cells) whose collision layer casts shadows too. Everything is on screen at once.
  • scale-flyover: the same content, but the camera follows a body that crosses the map at 670 units per second, so chunks keep streaming in, decoding and building meshes, and the sprites and emitters come into view and leave it again.

Both folders are ordinary games, so you can also open them in the player with a window or in the editor.

The player's benchmark mode​

--benchmark runs a game without a window for a fixed number of frames, advancing exactly 1/60 of a second per frame, so two runs can be compared frame by frame. It waits for the start scene, runs warm-up frames, measures, prints a table and saves a JSON or CSV report. The options and how to read the report are on the Performance page.

Use it to check a change: run the same game with the same --size, --renderer and --frames before and after, and compare the game and render work and the markers of the systems you touched. The report keeps the last 600 frames, whatever --frames is.