MeshShaderEngine

A didactic engine for native GLSL shader experiments: a classical vertex/fragment pipeline and NVIDIA mesh/task shaders (GL_NV_mesh_shader). It wraps the PyOpenGL boilerplate that is irrelevant for teaching purposes and gives students a small, explicit interface made of TOML configuration, shader files, and optional model animation.

Target audience and scope

MeshShaderEngine is aimed at students and instructors who want to learn or teach shader-level graphics programming in isolation from a scene graph, physics, and interaction logic. It is explicitly not a general-purpose 3D engine and complements Godot in the teaching concept:

• MeshShaderEngine: native GLSL, classical vertex/fragment pipeline, NVIDIA mesh/task shaders, meshlets, shader-level experiments.
• Godot: scene graph, physics, interaction models, desktop applications, OpenXR/Quest Pro.

Details on goals, non-goals, and architecture are in PLAN_SHADER_ENGINE.md.

Project status

Version 0.2.0 (beta). Two render paths, four built-in objects plus support for a student's own custom mesh, a real meshlet data path with optional task-shader culling, real lighting with normal mapping, multiple loadable 2D RGB/RGBA textures in both render paths, shader hot reload, declarative animation, thirteen graduated teaching examples, and a GitLab CI pipeline. mse run displays an experiment interactively, selectable via pipeline.backend in experiment.toml; [object].type selects one of four built-in objects (plane/cube/teapot/bunny), each of which can be rendered in BOTH backends, or "custom" for your own model (see section "Custom Student Meshes"). Mouse orbit/zoom (each individually toggleable) is handled by the application independently of the render path and works identically in both:

• classic (vertex/fragment pipeline): classical VAO/VBO pipeline. Runs on any OpenGL-4.5-capable GPU.
• mesh_nv (NVIDIA GL_NV_mesh_shader): a real meshlet data path -- every object is deterministically partitioned into meshlets and handed to the mesh shader via fixed SSBO bindings (see section "Shader ABI"), with an optional task-shader stage for meshlet/frustum culling. Strictly requires an NVIDIA GPU with this extension (see below).

On top of that there is a package structure, a CLI, an error hierarchy, fully validated TOML configuration, pure math, deterministic geometry (plane/cube procedural, teapot/bunny from versioned assets, see section "Assets"), OpenGL context creation/diagnostics, shader compilation with ABI reflection, and small GL resource wrappers. mse validate stays OpenGL-free. The shader ABI and all asset/data formats are versioned and frozen for 1.x (see sections "Shader ABI" and "Assets"). Known limitation: the original, fixed object.base_color_texture/normal_texture pair is still not implemented (only its documented defaults, see section "Known limitations") -- the newer, general [[textures]] mechanism (see section "Textures") is unrelated to that pair and fully implemented.

Requirements and supported hardware

• Python 3.11 up to, but excluding, 4 (requires-python = ">=3.11,<4" in pyproject.toml; verified in CI on 3.11/3.12/3.13, see section "GitLab CI"), Linux.
• Window management/context via GLFW (Python binding glfw), GL calls via PyOpenGL, image decoding via Pillow (see section "Textures"). All three are declared runtime dependencies.
• For mse run/mse info with real capability data: a working OpenGL 4.5 core context (X11/Wayland + libGL). Without these prerequisites (e.g. in a pure CPU sandbox without /dev/dri), mse info/mse run end in a controlled way with an explanatory error message instead of a crash -- the package itself remains importable independently of that.
• Strictly required only for pipeline.backend = "mesh_nv": an NVIDIA GPU with a working GL_NV_mesh_shader implementation (reference hardware: RTX A4000). Without this extension, MeshNVBackend aborts in a controlled way with a GraphicsContextError, instead of risking a raw PyOpenGL symbol error. mse info reports the extension as GL_NV_mesh_shader along with its implementation limits (max_mesh_output_vertices etc.) as soon as it is available.

Installation

python -m pip install meshshader-engine

Development installation

To work on the engine itself (running tests, linting, building the docs), clone the repository and install it editable with its dev extras instead:

python -m pip install --upgrade pip
python -m pip install -e '.[dev]'

Command line

mse --help
mse run path/to/experiment.toml       # displays an experiment interactively in a window
mse validate path/to/experiment.toml  # checks configuration and file references without OpenGL
mse info                              # logs OpenGL/GPU/driver capabilities

mse info: diagnostics

Briefly creates an invisible window with an OpenGL context, reads out vendor, renderer, OpenGL/GLSL version, and extensions, and then releases the context again. On hardware with a working graphics driver:

Vendor:            NVIDIA Corporation
Renderer:          NVIDIA RTX A4000/PCIe/SSE2
OpenGL version:    4.6.0 NVIDIA 535.183.01
GLSL version:      4.60 NVIDIA
Debug context:     True
GL_NV_mesh_shader: True
  max_mesh_work_group_invocations: 32
  max_mesh_total_memory_size:       16384 Bytes
  max_mesh_output_vertices:         256
  max_mesh_output_primitives:       256
  max_draw_mesh_tasks_count:        4294967295
Total extensions: 312

The max_* values under GL_NV_mesh_shader only appear if the extension is actually reported (see gl.capabilities.query_mesh_nv_limits); the numbers shown here are example values, not guaranteed constants.

Without display/GPU access (e.g. in a pure CPU sandbox without /dev/dri), the command ends in a controlled way with exit code 1 and an explanatory message instead of a crash:

$ mse info
Error: GLFW could not be initialized (code 65544: X11: Failed
to load Xlib). This indicates missing display/GPU access.

Five-minute example

Minimal experiment.toml (see also tests/data/valid_minimal/):

[pipeline]
backend = "classic"

[pipeline.shaders]
vertex = "basic.vert"
fragment = "basic.frag"

[object]
type = "cube"

[light]
position = [2.0, 3.0, 4.0]

mse validate path/to/experiment.toml
# OK: /path/to/experiment.toml is valid.

mse run examples/01_fragment/experiment.toml

The first runnable example lives under examples/01_fragment/: a cube in the classical render path, whose fragment color is computed directly from the UV coordinates (→ learning objectives, task, reflection questions).

Interaction: left-click-drag orbits the camera. Zoom works two ways -- scrolling, or right-click-drag up (in) / down (out), so it also works on laptops without a scroll wheel. Both are individually toggleable via interaction.orbit_enabled/zoom_enabled in experiment.toml.

One example shows that the vertex shader can actively change geometry instead of just passing it through:

mse run examples/02_vertex/experiment.toml

examples/02_vertex/: every vertex is displaced along its normal by an amount that depends on both time AND object-space position -- a "breathing" cube (→ learning objectives, task, reflection questions).

Two more examples show real lighting in the classical path:

mse run examples/03_lighting/experiment.toml
mse run examples/04_normal_mapping/experiment.toml

examples/03_lighting/: a Lambert diffuse term plus an optional, toggleable Blinn-Phong specular highlight (defines.USE_BLINN_PHONG), computed entirely in the fragment shader (see section "Coordinate spaces for lighting"). examples/04_normal_mapping/: reconstructs the tangent-space basis (TBN matrix) from normal, tangent, and handedness, and uses it to apply a normal perturbation to the lighting -- for lack of real texture loading (see section "Known limitations"), combined with a small procedural, purely UV-based wave function instead of a real normal-map file. Both examples offer defines.DEBUG_MODE to make the normal/tangent/UV (and for 04 additionally the tangent-space normal-map value) directly visible as color instead of evaluating the lighting formula -- the exact meaning of each value is documented as a comment in the respective experiment.toml (→ learning objectives, tasks, reflection questions: 03_lighting, 04_normal_mapping).

Another example shows the NVIDIA-specific mesh-shader path:

mse info                                          # checks GL_NV_mesh_shader first
mse run examples/05_mesh_triangle/experiment.toml

examples/05_mesh_triangle/: a single triangle generated entirely in the mesh shader -- no vertex buffer. Strictly requires an NVIDIA GPU with GL_NV_mesh_shader; without this extension, mse run aborts in a controlled way with an understandable error message (see section "Requirements"). The hardcoded triangle is ported from a Panda3D mesh-shader example, originally used to verify GL_NV_mesh_shader on the RTX A4000 before this engine had its own mesh-NV backend (see shader.mesh, https://discourse.panda3d.org/t/simple-mesh-shader-using-callback-node/29751) (→ learning objectives, task, reflection questions).

A final example shows the real meshlet data path:

mse run examples/06_meshlets/experiment.toml

examples/06_meshlets/: the same mesh-NV path, but the mesh shader reads geometry via the fixed SSBO bindings (see section "Shader ABI") instead of hardcoding it, and colors each meshlet by its ID for debugging purposes (defines.SHADING_MODE = 0, default). With defines.SHADING_MODE = 1, the same mesh shader instead renders real Lambert/Blinn-Phong lighting (the same formula as examples/03_lighting) via the meshlet SSBOs -- see section "Coordinate spaces for lighting". type in experiment.toml can be changed to "plane"/"teapot" to observe the meshlet partitioning on differently sized objects (→ learning objectives, task, reflection questions).

An example with an optional task-shader stage shows meshlet/frustum culling:

mse run examples/07_task_culling/experiment.toml

examples/07_task_culling/: a task-shader stage decides per meshlet, based on its bounding sphere, whether a mesh workgroup is dispatched at all (defines.CULLING_MODE, see section "Task shader and culling"). The camera is deliberately narrow so that a noticeable part of the teapot actually lies outside the view cone. CULLING_MODE 0 and 1 look deliberately identical here (frustum culling never changes the image, only the GPU work) -- only CULLING_MODE 2 colors something visibly for visualization purposes, see section "Task shader and culling" for the explanation (→ learning objectives, task, reflection questions).

Five more examples load real image files as textures (see section "Textures"):

mse run examples/08_texture_color/experiment.toml
mse run examples/09_texture_normal_map/experiment.toml
mse run examples/10_texture_parallax/experiment.toml
mse run examples/11_multiple_textures/experiment.toml
mse run examples/12_mesh_nv_textures/experiment.toml

examples/08_texture_color/: the first example that loads a real image file, via [[textures]] in experiment.toml -- a generated four-color grid, so a flip_y mistake is immediately obvious. examples/09_texture_normal_map/: a real normal-map file, sampled with the same TBN reconstruction as examples/04_normal_mapping. examples/10_texture_parallax/: an RGBA image whose alpha channel carries an unrelated height field, visualized as grayscale (not a full parallax-occlusion implementation). examples/11_multiple_textures/: three textures bound simultaneously, declared out of unit order in the TOML on purpose, to demonstrate that binding depends only on unit/layout(binding = ...), never declaration order. examples/12_mesh_nv_textures/: the mesh-shader equivalent of examples/11_multiple_textures -- confirms [[textures]] binds identically in the mesh_nv backend, and is the first shader in the repository whose shader.mesh actually reads the VertexUVs/VertexTangents SSBOs (requires GL_NV_mesh_shader) (→ learning objectives, task, reflection questions: 08_texture_color, 09_texture_normal_map, 10_texture_parallax, 11_multiple_textures, 12_mesh_nv_textures). examples/13_bunny_meshlets/: the Stanford Bunny (69,451 triangles) dropped into the exact same shaders as examples/06_meshlets, to make the resulting meshlet count visually dramatic instead of the teapot's comparatively modest 40 meshlets (→ learning objectives, task, reflection questions: 13_bunny_meshlets).

Examples: learning path

The thirteen examples are deliberately graduated (PLAN_SHADER_ENGINE.md, Phase 25; PLAN_TEXTURES.md, Phase 9): each builds on concepts from the previous ones, and not every stage calls for equally open-ended changes -- early examples give a concrete, tightly guided task, later ones more of an observation with open-ended reflection. Each example folder contains its own README.md with learning objectives, the files to change, a concrete task, the expected result, and reflection questions.

#	Topic	Path	Degree of openness
1	Fragment color/UV	examples/01_fragment/	tightly guided
2	Vertex transformation	examples/02_vertex/	tightly guided
3	Point light (Lambert/Blinn-Phong)	examples/03_lighting/	guided
4	Normal mapping	examples/04_normal_mapping/	guided + advanced bonus exercise
5	GPU-generated mesh triangle	examples/05_mesh_triangle/	more open
6	Meshlets (SSBO data path)	examples/06_meshlets/	more open
7	Task shader/culling (optional)	examples/07_task_culling/	most open
8	RGB color texture	examples/08_texture_color/	tightly guided
9	RGB normal map (loaded)	examples/09_texture_normal_map/	guided
10	RGBA height/parallax data	examples/10_texture_parallax/	guided
11	Multiple textures together	examples/11_multiple_textures/	more open
12	Multiple textures in the mesh-shader path	examples/12_mesh_nv_textures/	more open
13	Meshlets at real scale (Stanford Bunny)	examples/13_bunny_meshlets/	more open

The advanced bonus exercise in example 4 deliberately opens up a single encapsulated engine part (gl.reflection/gl.program.Program -- program reflection), without turning the entire engine into the subject of study (Phase 25, task 5).

Data flow: classical vs. mesh pipeline

Both pipelines share configuration, scene state, and the fragment- shader call; the difference lies solely in how geometry reaches the GPU and becomes triangles there.

Classical path (examples 1-4, 8-11):

flowchart LR
    T[experiment.toml] --> C[ExperimentConfig]
    C --> M[MeshData: primitives/teapot/bunny]
    M --> VB[VBO/VAO: Position/Normal/UV/Tangent]
    C --> S[SceneState: camera/light/time]
    S --> U["bind_scene_uniforms (mse_*)"]
    VB --> VS[Vertex shader]
    U --> VS
    VS --> RA[Rasterization]
    RA --> FS[Fragment shader]
    FS --> FB[Framebuffer]

Mesh-NV path (examples 5-7, 12-13):

flowchart LR
    T[experiment.toml] --> C[ExperimentConfig]
    C --> M[MeshData: primitives/teapot/bunny]
    M --> BM[build_meshlets]
    BM --> SSBO["MeshletGPUBuffers: SSBO bindings 0-7"]
    C --> S[SceneState: camera/light/time]
    S --> U["bind_scene_uniforms (mse_*)"]
    SSBO --> TS["Task shader (optional, example 7)"]
    U --> TS
    TS -->|Payload + gl_TaskCountNV| MS[Mesh shader]
    SSBO --> MS
    U --> MS
    MS --> RA[Rasterization]
    RA --> FS[Fragment shader]
    FS --> FB[Framebuffer]

Without a task shader (examples 5/6/12/13), the TS stage is omitted, and glDrawMeshTasksNV dispatches the mesh workgroups directly (see section "Shader ABI").

Instructor notes

• No separate solution version. Every example already exists as a fully working reference implementation; the "task" in the respective README.md asks for a targeted change/extension, not for filling in blanks. Students therefore always see a working result first, before changing anything themselves.
• Grading/traceability: a git diff against the original example state directly shows what was changed -- enough to check an example's task without reading the engine's entire source code.
• Adding your own examples: new examples follow the same pattern (its own folder NN_topic/ under examples/, its own README.md following the template above) -- both data-flow diagrams apply unchanged to any further classical or mesh-NV example.
• GPU dependency: examples 5-7 and 12-13 strictly require GL_NV_mesh_shader (see section "Requirements"); without suitable hardware, they abort in a controlled way with an understandable error message, instead of making a graded task unsolvable -- for a course without lab access, examples 1-4 and 8-11 (all classical-path) are primarily suitable.
• Test levels: the patterns used in tests/gpu//tests/integration/ (see section "Test levels and runner capabilities") can be reused 1:1 as a template for your own acceptance tests for student solutions.

Usability and Teaching (Clean-Room Test)

This section documents a clean-room test based exclusively on this README (PLAN_SHADER_ENGINE.md, Phase 26; acceptance: "A knowledgeable tester can start the first experiment, change it, and fix a shader error without any additional verbal instruction.").

Installation Test (Fresh venv)

Only the steps from the "Installation" section, run against a fresh, empty Python venv (no access to already-installed packages):

Step	Duration
python -m pip install --upgrade pip	~1.5 s
python -m pip install -e '.[dev]'	~9.5 s
mse --help	~0.1 s
mse validate examples/01_fragment/experiment.toml	~0.1 s
mse info (without display/GPU)	~0.1 s, controlled failure
mse run examples/01_fragment/experiment.toml (without display/GPU)	~0.25 s, controlled failure

All steps ran exactly as described in this README, with no additional verbal instruction needed. Without display/GPU, mse info/mse run ended, as documented, in a controlled way with an understandable error message and exit code 1 -- no Python traceback. (The actual mse run frame rendering itself requires a display/GPU and is not part of the clean-room test at this point, see section "Test Levels and Runner Capabilities".)

Student's View: Which Files Actually Need to Be Read?

For a single example, it suffices to read:

1. This README, sections "Installation", "Five-Minute Example", and "Hot Reload".
2. The README.md of the respective example folder (learning objectives, task, reflection questions).
3. The 2-4 files of THIS example folder itself (experiment.toml and the referenced shader files).

No source code under src/ is needed for this -- reflection questions that specifically point to ABI details always point to individual, clearly named README sections rather than to implementation files.

Instructor's View: Updating Solutions, Assets, Tests

• Solutions: there is no separate solution version -- every example already IS the reference implementation (see section "Examples: Learning Path", "Instructor Notes"). A model solution for a task is simply the git diff since the last commit of the respective example folder.
• Assets: procedural objects (plane/cube) are generated at runtime, no update needed. The teapot and bunny are versioned as src/meshshader_engine/assets/data/teapot.npz/bunny.npz; changes are made via python tools/build_assets.py (optionally --only teapot/--only bunny), which regenerates the file(s) from their official sources (see section "Assets").
• Tests: new examples get their tests following the same pattern as the existing thirteen (Level A: configuration validation in tests/unit; Level B/C: rendering behavior in tests/integration/tests/gpu, see section "Test Levels and Runner Capabilities").

Error Messages: File and Line (Phase 26, tasks 2+3)

Four typical student shader errors are covered by automated tests (tests/integration/test_shader_compiler_integration.py):

Error type	Example	Test
Undeclared identifier	gl_Position = mse_UndeclaredVariable;	test_compile_error_reports_stage_file_and_original_line_number
Missing semicolon (pure syntax error)	gl_Position = vec4(...) without ;	test_compile_error_for_missing_semicolon_reports_stage_and_file (Phase 26)
Type mismatch between vertex out/fragment in	out vec3 vColor; vs. in vec4 vColor;	test_link_error_reports_involved_stages
ABI uniform with wrong type	uniform vec4 mse_ModelViewProjection; instead of mat4	test_reflection_rejects_abi_uniform_with_wrong_type

For semantic errors (undeclared identifier, ABI type error), the driver reliably reports the ORIGINAL file line number (thanks to the #line restoration, see section "Shader ABI"). For pure syntax errors (missing semicolon), the exact reported line varies by driver -- but the file and stage are correctly reported in every case, so test_compile_error_for_missing_semicolon_reports_stage_and_file deliberately checks only those two, not the exact line.

Black-Box Risk

The engine deliberately encapsulates render-path details (see sections "Security Boundaries" and "Examples: Learning Path"). To ensure students change shaders through understanding rather than trial and error alone, a short comprehension/code-review task is useful, e.g.:

1. "Explain in 2-3 sentences why mse_NormalMatrix is computed from the model matrix instead of from model-view (see section 'Coordinate Spaces for Lighting')."
2. "A classmate claims that CULLING_MODE 1 should LOOK faster THAN CULLING_MODE 0. Refute this using examples/07_task_culling/README.md."
3. Code review task: hand out shader.mesh from examples/06_meshlets without the associated MeshletDescriptors SSBO declaration and have students explain which line(s) would no longer compile as a result, and why.

These questions can be adopted directly into the respective example README.md files or posed as a standalone short assignment.

Hot Reload

mse run watches the shader files entered in pipeline.shaders and automatically reloads them as soon as their modification timestamp changes -- saving in an editor is enough, no restart needed. The check is done via polling (file timestamp, pathlib.Path.stat()) once per drawn frame, deliberately instead of via an additional file watcher process (e.g. watchdog) -- no new dependency needed, and the latency of one frame is imperceptible for interactive experimentation (see meshshader_engine.gl.hot_reload, meshshader_engine.gl.program.ShaderHotReloader).

Order on every reload: the new program is fully compiled and linked BEFORE the previous one is replaced. If that fails (syntax error, ABI violation), the last working program stays active unchanged, and the error message appears on stderr -- a broken shader does not terminate mse run and does not leave behind an empty/frozen image.

Hot reload affects only GLSL shader files. There is no way to load or execute Python code at runtime via experiment.toml or any other file (see section "Security Boundaries").

Security Boundaries

• Shader hot reload executes no Python code. Only GLSL files are reloaded, and they are compiled entirely normally by the driver -- the same trust boundary as the original mse run startup, no additional attack surface.
• There is currently no Python animation adapter. Declarative animations ([animation], see section "TOML Reference") are the only supported form of dynamic scene state; they are computed entirely from validated, type-checked TOML values (meshshader_engine.scene_state.SceneState.model_matrix), never from executable code.
• An optional Python animation adapter is planned, but deliberately not yet implemented (PLAN_SHADER_ENGINE.md, Phase 21, tasks 5-7 explicitly require separate confirmation before implementation). Should it be added, the following already applies as a binding commitment:
  • Such an API would provide only small, pure helper functions for matrices -- no filesystem, network, or process control.
  • No real in-process sandbox is promised. Python offers no secure in-process isolation for untrusted code; an animation script would run with the same privileges as the engine itself. Anyone who runs an animation script must therefore trust it to the same degree as any other Python code they run themselves.
  • Automated evaluation of third-party/not-self-written animation scripts is intended exclusively in a separate, restricted process or container (e.g. for a future CI test run with example scripts from a student submission) -- never in the same process as mse run.

TOML Reference

An experiment.toml consists exclusively of the following tables. Unknown keys in any table are an error — typos are caught early this way instead of being silently ignored. Every error message from mse validate names the affected key path, e.g. pipeline.shaders.vertex: ....

Table	Key	Type	Required	Default	Description
window	title	string	no	"MeshShaderEngine Experiment"	window title
	width, height	integer > 0	no	1280, 720	window size
	vsync	bool	no	true	vertical sync
pipeline	backend	"classic" | "mesh_nv"	yes	—	render path
pipeline.shaders	vertex, fragment	path	for classic	—	shader files
	mesh, fragment	path	for mesh_nv	—	shader files
	task	path	optional, mesh_nv only	—	optional task shader (Phase 23)
object	type	"plane" | "cube" | "teapot" | "bunny" | "custom"	yes	—	object to display
	mesh_path	path	for type = "custom" only	—	converted .npz (see section "Custom Student Meshes")
	base_color_texture, normal_texture	path	no	engine default (below)	textures
camera	distance	number > 0	no	3.0	orbit distance
	azimuth_degrees, elevation_degrees	number	no	0.0, 20.0	orbit angles (elevation: -89..89)
	fov_degrees	number (1..179)	no	45.0	vertical field of view
	near, far	number > 0, far > near	no	0.05, 100.0	clipping planes
	target	[x, y, z]	no	[0, 0, 0]	orbit target
interaction	orbit_enabled, zoom_enabled	bool	no	true, true	mouse control on/off
	zoom_min, zoom_max	number > 0, max > min	no	1.0, 10.0	zoom bounds
light	position	[x, y, z]	yes	—	point-light world position
	color	[r, g, b], each in [0, 1]	no	[1, 1, 1]	light color
	intensity	number >= 0	no	1.0	light intensity
animation	mode	"none" | "rotate" | "oscillate"	no	"none"	built-in animation
	speed, axis	number, [x,y,z]	no	30.0, [0,1,0]	"rotate" only
	amplitude, frequency	number, number >= 0	no	1.0, 1.0	"oscillate" only
defines	any names	bool | int | float	no	—	approved shader defines (see below)
[[textures]]	path	path	yes	—	image file to load (see section "Textures")
	unit	integer >= 0	yes	—	GL texture unit; 14/15 reserved, see below
	channels	"auto" | "rgb" | "rgba"	no	"auto"	forced channel count, or auto-detect
	color_space	"linear" | "srgb"	no	"linear"	selects the GL internal format
	flip_y	bool	no	true	flip image rows before upload

Defines whitelist: keys must follow the pattern ^[A-Z][A-Z0-9_]*$ (uppercase letters, digits, underscore, starting with a letter); values are restricted to bool, int, and float. The actual injection into the shader preamble follows in Phase 11.

Textures: [[textures]] is an array of tables -- any number of entries, each declaring one additional 2D texture, independent of object.base_color_texture/normal_texture above. An experiment.toml with no [[textures]] section at all remains fully valid and behaves exactly as before this feature existed. Numbering can start at unit 0: only units 14 and 15 are permanently reserved, for the fixed mse_BaseColorTexture/mse_NormalTexture ABI slots (see section "Shader ABI") -- placed deliberately out of the way so this common case does not immediately hit a reserved unit; using either is rejected here with a clear error. See section "Textures" for the full schema semantics, the power-of-two policy, linear-vs-sRGB guidance, and troubleshooting.

Path Safety

All file paths in experiment.toml (shaders, textures, a custom object.mesh_path) are resolved relative to the directory of the TOML file. Absolute paths are not allowed, and a path must not leave the experiment directory after normalization (outward .. traversal is rejected). This policy prevents an experiment from accidentally or deliberately referencing files outside its own directory.

Mathematical Conventions

Binding for the entire engine (see src/meshshader_engine/math3d.py for the authoritative, detailed documentation in the code):

• Coordinate system: right-handed. +X right, +Y up, +Z out of the screen. A camera with identity orientation looks along -Z (standard OpenGL convention).
• Vectors/matrices: column vectors, stored row-major (M[row, col], NumPy default). Applied as M @ v; composition right to left, e.g. clip = projection @ view @ model @ local.
• GLSL upload (from Phase 13): since GLSL mat4 is stored column-major in memory, but our matrices are row-major, glUniformMatrix4fv must be called with transpose=GL_TRUE.
• Angles: exclusively radians in math3d.py/OrbitCamera; exclusively degrees in the TOML configuration. The conversion happens once when building camera/animation state (OrbitCamera.from_config, SceneState.model_matrix).
• dtype: all public matrix/vector functions return numpy.float32.
• Normal matrix: inverse-transpose of the linear part of the model matrix (not model-view), since lighting per the shader ABI happens in world space (mse_LightPositionWorld, mse_CameraPositionWorld).
• Orbit camera: OrbitCamera (src/meshshader_engine/camera.py) is a pure, window-system-independent state machine. At azimuth=0°, elevation=0°, the camera sits on the +Z axis relative to the target point. Elevation is clamped to (-89°, 89°) to avoid the singularity of look_at when the view/up directions are parallel.
• SceneState (src/meshshader_engine/scene_state.py) carries time explicitly as a time_seconds field instead of reading a clock internally, so animations (none/rotate/oscillate) are deterministic and testable without a window system.

Vertex Layout and Tangent Basis

Geometry is represented as an immutable MeshData (src/meshshader_engine/assets/mesh_data.py). meshshader_engine.assets.primitives.build_plane()/build_cube() generate the plane/cube procedurally on every startup; the teapot and bunny are loaded by meshshader_engine.assets.teapot.load_teapot()/ meshshader_engine.assets.bunny.load_bunny() from prebuilt assets (see section "Assets"). All four go through the same validation and the same MeshData contract.

Attribute	Shape	dtype	Meaning
positions	(N, 3)	float32	object-space position
normals	(N, 3)	float32	unit normal
uvs	(N, 2)	float32	texture coordinates
tangents	(N, 4)	float32	xyz normalized, w = handedness (±1)
indices	(M,), M % 3 == 0	uint32	plain triangle list

The bitangent is not stored, but reconstructed in the shader:

vec3 bitangent = cross(normal, tangent.xyz) * tangent.w;

Winding: triangles are wound counter-clockwise (CCW) when viewed from the normal side — standard OpenGL front face.

Construction principle: every flat face is built from a center point, normal N, and tangent (u_axis) U; the bitangent is always derived as V = cross(N, U) instead of chosen freely. This automatically guarantees correct CCW winding as well as handedness +1 for every face (derivation in the docstring of primitives.py) — it does not need to be worked out by hand per face.

Cube = 24 instead of 8 vertices: each of the six sides gets its own vertices, since normals and UV coordinates are discontinuous at the edges (sharing corner vertices between faces cannot be represented with a single normal/UV value per vertex).

Coordinate Spaces for Lighting

Reference examples: examples/03_lighting/, examples/04_normal_mapping/.

Lighting happens entirely in world space, not in view/camera space. mse_LightPositionWorld and mse_CameraPositionWorld (see section "Shader ABI") are both world-space coordinates; for a point on the object surface to be compared against them, it must also be transformed into world space — in the vertex shader with mse_Model, not with mse_ModelViewProjection (that would project into clip space, not world space):

vec3 positionWorld = vec3(mse_Model * vec4(mse_Position, 1.0));

Normals and tangents transform differently and must therefore not both simply be multiplied by mse_Model:

• A tangent is an ordinary direction vector in the surface tangent space; the upper 3×3 block of mse_Model suffices: mat3(mse_Model) * tangent.xyz.
• A normal is perpendicular to the surface. Under non-uniform scaling, direct multiplication with mse_Model distorts this perpendicularity; the correct transform is the normal matrix, the inverse-transpose of the linear part of mse_Model (mse_NormalMatrix, see section "Mathematical Conventions" and math3d.py): mse_NormalMatrix * mse_Normal.

Object transformations (object in experiment.toml) are not yet freely configurable, so mse_Model in the examples only contains rotation/oscillation (see [animation]), but no scaling — the difference between mse_NormalMatrix and mat3(mse_Model) is therefore not visible here. Anyone who wants to observe it can experimentally replace mse_NormalMatrix in gl/program.py with mat3(mse_Model) and experimentally scale build_cube()/ build_plane() non-uniformly (e.g. build_cube() with different x/y/z factors instead of a single size).

Reconstructing the tangent-space basis (TBN matrix): normal mapping additionally requires a bitangent, which (as described in section "Vertex Layout and Tangent Basis") is not stored, but reconstructed from the normal, tangent, and handedness:

vec3 N = normalize(vNormalWorld);
vec3 T = normalize(vTangentWorld);
vec3 B = cross(N, T) * vTangentHandedness;
mat3 TBN = mat3(T, B, N);
vec3 mappedNormalWorld = normalize(TBN * tangentSpaceNormal);

tangentSpaceNormal is a normal-map value decoded from [0, 1] (texture format) to [-1, 1] (texture(mse_NormalTexture, uv).xyz * 2.0 - 1.0); since object.normal_texture does not load a real file (see section "Known Limitations" -- unrelated to the general [[textures]] mechanism, see section "Textures"), examples/04_normal_mapping/ combines this with a small procedural, purely UV-based wave function for a visible effect. For a normal map loaded from a real file via [[textures]] instead, see examples/09_texture_normal_map/ -- the TBN reconstruction itself is identical either way.

No lighting computation in Python: SceneState/ bind_scene_uniforms (gl/program.py) provide only raw data (light/camera position, color, intensity) as uniforms; every lighting formula (Lambert, Blinn-Phong, normal mapping) resides entirely in the shader. This makes it directly visible in the shader code which computation takes place — there is no hidden Python-side preprocessing step.

Shader ABI

Version 2 (meshshader_engine.gl.reflection.SHADER_ABI_VERSION): the mse_* uniforms, vertex-attribute locations, and SSBO bindings documented below are stable for 1.x versions of this package. Backward-incompatible changes (renamed/removed uniforms, moved locations/bindings) increase this version; purely additive extensions (an additional, new mse_* uniform) do not, since existing shaders remain unaffected by them. Version 1 was frozen with package version 1.0.0 (Phase 29); version 2 (introduced alongside [[textures]], see section "Textures") moved mse_BaseColorTexture/mse_NormalTexture from bindings 0/1 to 14/15 -- done before any external release depended on version 1, so the general texture mechanism could use the intuitive unit 0 as its starting point instead of immediately hitting a reserved slot.

Every shader file must begin with #version as the very first line (this engine's convention, no preceding comments/blank lines). Recommended: #version 450 core, matching the requested OpenGL 4.5 core context (see section "Requirements").

The engine provides the following reserved uniforms (prefix mse_); if a shader uses them, they must be declared with exactly this type — this is checked via ABI reflection before every render frame:

uniform mat4  mse_Model;
uniform mat4  mse_View;
uniform mat4  mse_Projection;
uniform mat4  mse_ModelViewProjection;
uniform mat3  mse_NormalMatrix;
uniform vec3  mse_CameraPositionWorld;
uniform vec3  mse_LightPositionWorld;
uniform vec3  mse_LightColor;
uniform float mse_LightIntensity;
uniform float mse_Time;
uniform vec2  mse_ViewportSize;

layout(binding = 14) uniform sampler2D mse_BaseColorTexture;
layout(binding = 15) uniform sampler2D mse_NormalTexture;

These two are the only fixed, name-checked texture uniforms; both currently always sample the documented 1x1 defaults (see section "Known limitations"). Any number of additional textures can be loaded via [[textures]] and are bound purely by unit number, with a freely chosen sampler name -- see section "Textures".

Additionally, the four vertex attributes from MeshData are mapped to fixed locations (see src/meshshader_engine/gl/shader_abi.py, ATTRIBUTE_LOCATIONS):

layout(location = 0) in vec3 mse_Position;
layout(location = 1) in vec3 mse_Normal;
layout(location = 2) in vec2 mse_UV;
layout(location = 3) in vec4 mse_Tangent;

Only actually active resources are checked: if a shader does not use an mse_* uniform at all, the compiler may optimize it away — that is not an error. But if a shader declares it with a type other than the one specified above, the ABI conformance check fails before rendering.

Approved defines: names from [defines] in experiment.toml are injected as #define NAME VALUE directly after #version (bool as 1/0, int/float as a GLSL literal), followed by any needed #extension directives (GL_NV_mesh_shader for the mesh/task stages) and a #line directive that maps compiler error messages back to the original file line number.

SSBO bindings for the meshlet data path (pipeline.backend = "mesh_nv"): mesh shaders do not go through the classic vertex-attribute pipeline and instead read geometry via eight fixed std430 SSBO bindings (see src/meshshader_engine/gl/shader_abi.py, MESHLET_SSBO_BINDINGS):

Binding	Name	Content
0	MeshletDescriptors	uvec4[]: (vertex_offset, vertex_count, primitive_offset, primitive_count)
1	MeshletVertexIndices	uint[]: global vertex indices
2	MeshletPrimitiveIndices	uint[]: local triangle indices (3 per triangle)
3	VertexPositions	vec4[]: position, w unused
4	VertexNormals	vec4[]: normal, w unused
5	VertexUVs	vec2[]
6	VertexTangents	vec4[]: xyz tangent, w handedness
7	MeshletBounds	vec4[]: bounding sphere per meshlet (xyz object-space center, w radius; Phase 23)

positions/normals are padded from vec3 to vec4 on upload, since std430 requires a 16-byte alignment per element for vec3 arrays anyway (same as for vec4) — a tightly packed CPU array would otherwise be interpreted with the wrong stride.

A mesh-shader workgroup processes exactly one meshlet; gl_WorkGroupID.x is its index into MeshletDescriptors. The dispatch is done as glDrawMeshTasksNV(0, meshlet_count) (see gl.backends.mesh_nv.MeshNVBackend). Example: examples/06_meshlets/shader.mesh.

Meshlet Data Format

meshshader_engine.assets.meshlets.build_meshlets() partitions a MeshData deterministically (same input → exactly the same meshlets) via a deliberately simple greedy algorithm in fixed triangle order — no meshoptimizer or other new dependency, see PLAN_SHADER_ENGINE.md, Phase 17, task 6. Upper bounds per meshlet: 64 vertices, 126 triangles (MESHLET_MAX_VERTICES/ MESHLET_MAX_PRIMITIVES) — conservative constants modeled on common NVIDIA examples, not queried from real hardware. Every MeshletDescriptor carries a format version (MESHLET_FORMAT_VERSION) and is fully validated against its bounds and referenced indices when constructed. Meshlets are rebuilt at runtime from the (possibly loaded) MeshData, not separately serialized — for the mesh sizes occurring here (up to the bunny's ~69,000 triangles, see section "Assets"), this takes well under a second; a second artifact to keep in sync would be unnecessary redundancy.

Textures

Beyond the fixed mse_BaseColorTexture/mse_NormalTexture ABI pair (see section "Shader ABI"), any number of additional 2D RGB/RGBA textures can be declared with [[textures]] in experiment.toml and loaded from common image files (PNG, JPEG, and any other format Pillow supports) -- the engine's only image-decoding dependency. An experiment.toml with no [[textures]] section at all remains fully valid and behaves exactly as it did before this feature existed -- see section "TOML Reference" for the complete schema and defaults.

[[textures]]
path = "textures/albedo.png"
unit = 0
channels = "rgb"
color_space = "srgb"
flip_y = true

[[textures]]
path = "textures/normal.png"
unit = 1
channels = "rgb"
color_space = "linear"

Explicit binding convention: unit vs. texture object ID

A [[textures]] entry's unit is an OpenGL texture image unit (GL_TEXTURE0 + unit) -- one of several numbered "slots" a shader samples from. It is a completely different number from a texture's internal GL object ID (called handle in meshshader_engine.gl.textures.Texture2D), which merely identifies one specific GPU object within the whole context and has no meaning to a shader at all. Several different texture objects can be bound to the same unit over time (one at a time); confusing the two numbers is a common source of "nothing is drawn"/"wrong texture" bugs.

A shader accesses a configured texture with an explicit layout(binding = unit), choosing its own sampler name freely (unlike mse_BaseColorTexture/mse_NormalTexture, a [[textures]] sampler's name is not part of the ABI and is never checked by ABI reflection -- only the binding number matters):

layout(binding = 0) uniform sampler2D albedo_map;
layout(binding = 1) uniform sampler2D normal_map;

Units 14 and 15 are permanently reserved for the fixed mse_BaseColorTexture/mse_NormalTexture slots -- the classic render path always binds its (currently default-only) textures there, whether or not a shader even uses them. Deliberately placed out of the way at the high end instead of at 0/1, so the common case of a handful of [[textures]] entries can start numbering at the intuitive 0. A [[textures]] entry declaring unit = 14 or unit = 15 is rejected with a clear TextureConfigurationError before any OpenGL call.

Power-of-two policy

Width and height must both be a power of two (1, 2, 4, 8, 16, ..., independently per axis -- 256x128 is fine, 300x256 is not). Modern desktop OpenGL can display non-power-of-two textures just fine; this is a deliberate engine policy, not a general OpenGL limitation. Rejected images are never resized or padded automatically -- a precise error names the file, its actual size, and example valid sizes near it, e.g.:

Texture for unit 2: '/.../parallax.png' has size 300x256.
MeshShaderEngine requires width and height to be powers of two. The
image was not resized; use dimensions such as 256x256 or 512x256.

Linear vs. sRGB

color_space selects the GL internal format used for upload:

Content	color_space	Internal format
Albedo/base-color image	"srgb"	GL_SRGB8/GL_SRGB8_ALPHA8
Normal map, roughness/metallic/height/mask, any other data texture	"linear"	GL_RGB8/GL_RGBA8

Albedo/base-color images are normally authored and saved in sRGB (the same gamma-encoded space monitors display), so "srgb" lets the GPU decode them to linear light correctly before shading. Everything else loaded here -- normal maps, height/roughness/metallic maps, masks -- encodes numbers, not display colors; decoding them through an sRGB curve would silently distort those numbers (a normal map's XYZ vector components, in particular). When in doubt for a non-color data texture, use "linear".

Use cases

• Albedo/base color: channels = "rgb", color_space = "srgb" (see examples/08_texture_color).
• Normal/bump mapping: channels = "rgb", color_space = "linear", decoded from [0, 1] to [-1, 1] and transformed through the existing TBN reconstruction (see section "Coordinate spaces for lighting" and examples/09_texture_normal_map).
• Height/parallax data: channels = "rgba", reading the alpha channel as a height field alongside an unrelated RGB pattern (see examples/10_texture_parallax). This engine does not implement parallax-occlusion mapping (UV offsetting by view angle and height) -- only loading and visualizing the data.
• Several textures together: any number of [[textures]] entries can be bound simultaneously, each at its own unit (see examples/11_multiple_textures).
• Mesh-shader path: [[textures]] binds identically in the mesh_nv backend -- MeshNVBackend uses the same TextureSet as ClassicBackend. A mesh shader still has to read UV/tangent data explicitly from the VertexUVs/VertexTangents SSBOs (bindings 5/6) instead of getting them as automatic vertex attributes (see examples/12_mesh_nv_textures, the mesh-shader equivalent of examples/11_multiple_textures).

These four ingredients are common building blocks of a physically based rendering (PBR) material system, but MeshShaderEngine deliberately does not supply one -- no roughness/metallic workflow, no BRDF library, no material presets. Combining textures into a lighting model, as in examples/09_texture_normal_map/ 11_multiple_textures, stays a few explicit lines in a teaching shader, not a framework.

Implementation details

• Pixel alignment: RGB uploads (3 bytes/pixel) are not a multiple of OpenGL's default 4-byte row alignment; Texture2D sets GL_UNPACK_ALIGNMENT = 1 for the duration of each upload and restores the previous value immediately afterward, so this does not leak global GL state into unrelated code.
• Mipmaps: every [[textures]] entry gets a full mipmap chain (glGenerateMipmap) with GL_LINEAR_MIPMAP_LINEAR minification and GL_LINEAR magnification filtering; wrap mode is GL_REPEAT on both axes. These defaults are fixed for version 1 (no per-texture filter/wrap override yet, see "Planned extensions").
• Ownership and cleanup: meshshader_engine.gl.textures.TextureSet owns every [[textures]] GPU texture for one experiment, created transactionally (a failed upload deletes every texture already created in the same batch) and released exactly once, alongside the rest of a backend's resources, while the owning OpenGL context is still current -- the same explicit-close() pattern used throughout the GL resource wrappers (gl.buffers/gl.textures/ gl.program; see gl.buffers' module docstring), never __del__.
• Context lifetime: decoding (Pillow) needs no OpenGL context and can happen at any time; uploading (copying pixel data to the GPU) requires an already-current context and happens once per experiment start, inside the render backend's construction.
• No live texture reload: shader hot reload (see section "Hot Reload") only ever replaces the compiled Program; it never touches TextureSet. Editing [[textures]] in experiment.toml while mse run is active has no effect until the process is restarted -- a deliberate scope decision for version 1, not a bug.

Troubleshooting

Symptom	Likely cause	Fix
mse validate/mse run fails with "not a power of two"	Image width/height is not a power of two	Resize the source image to a power-of-two size (the error message names two candidate sizes)
"is reserved for the fixed object.base_color_texture/normal_texture ABI slots"	[[textures]].unit is 14 or 15	Use a different unit, e.g. 0 and up
"outside the range supported by this GPU/driver"	unit is >= this GPU's GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS (see mse info)	Use a lower unit number
"could not be decoded"	File missing, unreadable, truncated, or not a supported image format	Check the path (relative to the experiment.toml, see "Path Safety") and that the file opens in a normal image viewer
Texture appears upside down	flip_y does not match the source image's row order	Toggle flip_y (default true, correct for normally oriented image files)
Normal map looks washed out or lighting looks wrong only when using it	color_space = "srgb" on a normal map	Use color_space = "linear" for normal maps and other data textures, never "srgb"

Task Shaders and Culling

pipeline.shaders.task (optional, only together with backend = "mesh_nv") enables an additional task-shader stage BEFORE the mesh shader. Dispatch still happens via glDrawMeshTasksNV(0, meshlet_count) (see gl.backends.mesh_nv.MeshNVBackend) — but now as task workgroups: first, a task workgroup decides via gl_TaskCountNV how many mesh workgroups it launches itself (0 or more). Data flows from the task stage to the mesh stage via a taskNV out/taskNV in interface block ("payload").

Important pitfall: with a task stage present, gl_WorkGroupID.x in the mesh shader only denotes the index WITHIN the mesh workgroups launched by the respective task workgroup itself — NO LONGER the global meshlet index as without a task stage (see section "Shader ABI"). The original meshlet index must therefore be explicitly passed through via the payload, see examples/07_task_culling/shader.task and examples/07_task_culling/shader.mesh.

Meshlet/frustum culling (examples/07_task_culling) uses exactly this pattern: per meshlet, the task workgroup checks its bounding sphere (SSBO binding 7, MeshletBounds, transformed into world space via mse_Model) against the six camera planes extracted from mse_Projection * mse_View via the Gribb/Hartmann method. If the sphere is guaranteed to lie fully outside at least one plane, the task workgroup sets gl_TaskCountNV = 0 — for this meshlet, literally not a single mesh-shader workgroup is generated, not just one drawn invisibly.

Important — and easy to underestimate: frustum culling by definition only removes geometry that would lie outside the visible image anyway (the GPU would otherwise reject it itself during clipping). A rejected meshlet was therefore never visible — neither drawing nor omitting it changes the rendered image. The effect is purely saved mesh-shader work, not a visible difference. Anyone who starts the example and compares CULLING_MODE 0 against 1 therefore deliberately sees exactly the same image — this is not a failed demonstration, but the correct, often surprising property of frustum culling itself.

The example controls this behavior via [defines] CULLING_MODE:

Value	Meaning
0	culling off, everything is drawn (comparison baseline)
1	real culling — rejected meshlets launch no mesh workgroup; looks identical to 0 (see above)
2	debug visualization (default) — everything is still drawn, but meshlets that miss a deliberately narrower virtual field of view are colored red

Because a meshlet rejected against the real camera frustum can never appear on screen, CULLING_MODE == 2 deliberately does NOT test against the real frustum, but against an artificially narrowed one (see DEBUG_FRUSTUM_MARGIN in examples/07_task_culling/shader.task): extractFrustumPlanes scales only the left/right/bottom/top plane rows by a factor > 1 for this — mathematically exactly equivalent to a projection matrix with a narrower field of view at the same aspect ratio and the same clipping distances. This makes the red boundary visibly cut through the middle of the actually rendered image, without changing the real culling decision (CULLING_MODE == 1).

Limits of this Culling Example

• The sphere-frustum test is conservative, but not exact: near the frustum corners, a sphere that is actually invisible can be incorrectly classified as visible (false positive). The reverse case — an actually visible sphere is incorrectly rejected — does not occur.
• The test is exclusively against the field of view (frustum culling). Occlusion by other geometry (occlusion culling) does not take place.
• As described above, CULLING_MODE = 2 tests against a deliberately narrower virtual field of view, NOT against the actual camera frustum of CULLING_MODE = 1 — the red-marked meshlets are an analogy to the decision boundary, not a 1:1 preview of the actual culling savings.
• This example conveys the principle of workgroup/meshlet culling using a deliberately narrow camera, where a noticeable part of the teapot actually lies outside the view cone. It makes no claim of a general performance improvement — that would require a measurement on concrete hardware with a concrete scene, which this didactic example does not perform.

Assets

The plane and cube are generated procedurally (see above). The teapot and bunny come from external sources. You can add your own model too — see section "Custom Student Meshes".

Teapot

• Source: University of Utah, https://users.cs.utah.edu/~dejohnso/models/teapot.html, triangulation teapot_bezier0.norm (3,488 triangles, the smallest of the three offered resolutions). The original has no bottom (never modeled, since it was never visible in the original renderings) — see "Known Limitations".

• License status: the original model (Martin Newell, 1975) has been distributed unchanged for decades as a freely usable teaching/reference model for computer graphics; however, according to our research, no explicit, formal license statement exists — even the Apache Software Foundation's own legal review in 2023 reached no final conclusion (Jira ticket LEGAL-525, closed as "Won't Fix"). This engine uses the model directly from the official Utah source with fully documented provenance instead of an unofficial copy of unclear origin.

• Processing: the source file is a triangle list with no shared vertex indices and no UVs. tools/build_assets.py welds together exactly matching (position, normal) pairs, generates UVs via cylindrical projection around the Y axis (an approximation — this produces discontinuous tangents at the seam u=0/u=1, see "Known Limitations"), computes tangents (Lengyel method), and normalizes center/size like the plane/cube. The result is stored, versioned, as src/meshshader_engine/assets/data/teapot.npz (see meshshader_engine.assets.serialization) and loaded by meshshader_engine.assets.teapot.load_teapot().

• Format version: MESH_DATA_FORMAT_VERSION = 1 (meshshader_engine.assets.serialization), frozen with package version 1.0.0 (Phase 29) -- strictly checked on load, a file with a differing version is rejected instead of silently misinterpreted. The separate meshlet data format has its own version (MESHLET_FORMAT_VERSION, see section "Meshlet Data Format") -- meshlets are deliberately not serialized along with it (see the module docstring of assets.serialization).

• Reproducible build:

  python tools/build_assets.py
  

  Re-downloads the source file directly from the University of Utah and regenerates teapot.npz — traceable for anyone who wants to verify the provenance themselves.

Bunny

• Source: Stanford Computer Graphics Laboratory, https://graphics.stanford.edu/data/3Dscanrep/, bunny.tar.gz, reconstruction bun_zipper.ply (35,947 vertices, 69,451 triangles — the highest-resolution "zippered" reconstruction; three coarser _res2/_res3/_res4 decimations exist upstream but are not used here). By far the highest triangle count of any object the engine ships, added specifically to make meshlet partitioning (examples/13_bunny_meshlets) more visually meaningful than the teapot's ~40 meshlets. The reconstruction has 5 boundary loops (holes) on its underside where the original laser scans could not see — see "Known Limitations".

• License status — ⚠️ different from every other asset here: per the Stanford repository's stated terms, this data may be used for research/educational purposes and freely mirrored/redistributed, but not for commercial purposes, nor in a product for sale, without Stanford's permission. This is more restrictive than this engine's own GPLv3 license, which does permit commercial use — anyone redistributing or building on this repository needs to keep that distinction in mind specifically for bunny.npz. (This decision was made explicitly, coordinated with the project owner — see commit history — precisely because the terms are clear, unlike the teapot's unresolved status above.)

• Processing: the source file already has shared vertex indices (unlike the teapot's triangle soup), but provides neither normals nor UVs, and includes 1,113 vertices that no triangle references (scan artifacts). tools/build_assets.py drops those unreferenced vertices and re-indexes, centers/scales like the teapot, computes per-vertex normals from face geometry (area-weighted average of adjacent triangle normals), generates UVs via the same cylindrical Y-axis projection as the teapot (same seam caveat, see "Known Limitations"), and computes tangents with the same Lengyel-method helper. The result is stored, versioned, as src/meshshader_engine/assets/data/bunny.npz (see meshshader_engine.assets.serialization) and loaded by meshshader_engine.assets.bunny.load_bunny().

• Format version: same MESH_DATA_FORMAT_VERSION as the teapot (see above) — one shared, versioned MeshData format for every asset.

• Reproducible build:

  python tools/build_assets.py --only bunny
  

  Re-downloads bunny.tar.gz directly from Stanford and regenerates bunny.npz — traceable for anyone who wants to verify the provenance themselves.

Custom Student Meshes

Beyond the four built-in objects, object.type = "custom" lets you render your own model. The render path itself is already fully generic over geometry — the classical and mesh-NV backends, the GPU buffer code, and the shaders neither know nor care whether an object is build_plane(), load_teapot(), or yours; the only new thing is getting your geometry into the engine's own, already-validated MeshData format. That conversion happens once, offline, with a dedicated tool — never at mse run time.

Step by step

1. Model and prepare your mesh in Blender (or any other 3D tool that exports OBJ). Before exporting, make sure the mesh is:

   • Triangulated — every face has exactly 3 corners. In Blender: select the mesh, Face menu → Triangulate Faces (or enable "Triangulate Faces" in the OBJ export options directly).
   • UV-unwrapped — this engine requires texture coordinates on every object; there is no fallback default for a custom mesh the way there is for the fixed ABI textures. In Blender: UV Editing workspace → Unwrap (Smart UV Project is a fast, good-enough starting point for a first attempt).

2. Export as Wavefront OBJ (File → Export → Wavefront (.obj) in Blender). Normals are optional — export them if you have good ones, or leave them out and let the converter compute them from the mesh surface (same technique used for the Stanford Bunny, which also has no source normals).

3. Convert it with the dedicated import tool:

   python tools/import_mesh.py my_model.obj my_model.npz
   

   This prints the resulting vertex/triangle/meshlet counts and writes my_model.npz — the engine's own versioned MeshData format, exactly like teapot.npz/bunny.npz. Put the .npz file next to your experiment.toml (or anywhere inside the experiment directory).

4. Reference it in experiment.toml:

   [object]
   type = "custom"
   mesh_path = "my_model.npz"
   

   mesh_path follows the same path-safety rule as every other file reference (relative to the TOML file, no absolute paths, no .. escape — see section "Path Safety").

5. Check it, then run it:

   mse validate my_experiment/experiment.toml   # catches a missing/misplaced .npz early
   mse run my_experiment/experiment.toml
   

What the converter does and does not do

tools/import_mesh.py is deliberately narrow — it exists to get your geometry into the engine with minimal fuss, not to be a general OBJ importer:

• Triangles only. A non-triangulated face (a quad, an n-gon) is rejected with a clear error naming the offending face — go back and triangulate in your 3D tool rather than expecting the converter to do it for you.
• Texture coordinates are required. A mesh with no vt data at all is rejected outright, with a message pointing you at UV unwrapping — never silently given a placeholder UV.
• Normals are optional. If your OBJ has them, they are used (re-normalized defensively, in case your exporter didn't already); if not, they are computed from the mesh surface (area-weighted average of adjacent face normals — the exact technique used for the Stanford Bunny).
• Tangents are always computed, never read from the OBJ (OBJ has no tangent concept), via the same technique used for the teapot and bunny.
• Materials, groups, and multiple objects in one file are ignored. usemtl/mtllib/o/g/s lines are skipped; every triangle in the file becomes part of one single mesh, regardless of how it was grouped in your 3D tool.
• Your mesh is automatically centered and scaled to the same convention as every other object this engine ships (largest bounding-box extent normalized to 1.5), so it behaves consistently with the default camera without any extra configuration. This currently cannot be disabled.
• No triangle-count limit. A very large, detailed model is accepted; it will just take longer to convert and to build meshlets for, and may render more slowly on modest hardware. Keep this in mind for portfolio/scan-quality meshes — a game-asset-scale model (thousands, not millions, of triangles) will work more comfortably in class.

Troubleshooting

Error message contains...	Likely cause	Fix
"only triangulated faces are supported"	Your export included quads/n-gons	Triangulate the mesh before exporting
"no texture coordinates (vt) found"	The mesh has no UV unwrap	Add a UV unwrap in your 3D tool, then re-export
"UVs must be present on every face"	Some faces have UVs, others don't	Re-unwrap the whole mesh, not just part of it
"normals must be present on every face"	Some faces have normals, others don't	Either export normals for the whole mesh, or none at all
"Custom mesh not found" (from mse validate/mse run)	object.mesh_path doesn't point at an existing file	Check the path is correct and run tools/import_mesh.py first

Effects on stability

Every custom mesh — like every built-in one — passes through this engine's full MeshData validation contract before anything touches the GPU: correct dtypes and shapes, no NaN/Inf, unit-length normals and tangents, valid handedness, in-range indices. A malformed or corrupted .npz fails with a clear error at load time (in mse validate, and before any GL call in mse run) — never a crash mid-render. Geometric quality issues this contract does not catch (inconsistent winding, degenerate triangles, non-manifold edges) are at most a visual defect, never a stability one: this engine never enables backface culling, and lighting uses the mesh's own stored normals rather than anything winding-derived.

Known Limitations

• The fixed object.base_color_texture/normal_texture pair does not load files. Setting either field in experiment.toml still leads to a controlled error (GraphicsContextError). Without these fields, the engine uses documented defaults: a 1×1 white base-color texture and a 1×1 "flat" normal texture (RGBA 128, 128, 255, 255, corresponding to the tangent-space normal (0, 0, 1)). This is unrelated to the newer, general [[textures]] mechanism (see section "Textures"), which fully loads real image files via Pillow and is not affected by this limitation.
• Only one object at a time, no scene graph (see "Target Audience and Scope").
• examples/05_mesh_triangle still uses no object. The minimal example generates its triangle directly in the mesh shader and deliberately does not read the meshlet SSBOs bound since Phase 17; a real SSBO-based meshlet path is examples/06_meshlets.
• Teapot UVs have a visible seam. The cylindrical UV projection (see section "Assets") produces discontinuous UV deltas at the back seam (u=0/u=1) and therefore deviating tangents there -- no visible effect with the fixed ABI pair's default textures (solid color), but visible as a seam if a [[textures]] entry is applied to a teapot object (none of the texture examples currently use the teapot, see section "Examples: learning path").
• The teapot has no bottom. The original Bezier model (Martin Newell, 1975) was never modeled from below, since the underside was never visible in the original renderings. The source data therefore contains no bottom face, and this engine does not artificially add one — the teapot remains open at the bottom, visible e.g. at viewing angles from below.
• Bunny UVs have a visible seam, same cause as the teapot's. Same cylindrical projection, same caveat (see section "Assets") — visible if a [[textures]] entry is applied to a bunny object (none of the texture examples currently use it, see section "Examples: learning path").
• The bunny has 5 small holes on its underside. The original laser scans could not see the bottom of the physical model from every angle; the "zippered" reconstruction leaves 5 open boundary loops there, and this engine does not artificially close them — visible at viewing angles from below, similar in spirit to the teapot's missing bottom.
• Geometric/color comparison of classic vs. Mesh-NV only via silhouette and rough brightness, not an exact pixel match. tests/gpu/test_classic_vs_mesh_nv_consistency.py (Phase 18, task 5) compares which pixels belong to the object (Jaccard index ≥ 95%) instead of exact colors, since both paths there deliberately use different shaders (classic: white; Mesh-NV: meshlet debug colors). tests/gpu/test_classic_vs_mesh_nv_lighting.py (Phase 20, task 6) additionally compares examples/03_lighting (classic) against examples/06_meshlets with defines.SHADING_MODE = 1 (Mesh-NV, the same Lambert/Blinn-Phong formula via the meshlet SSBOs) for similar silhouette size and mean brightness. Small edge deviations from different rasterization are tolerated this way; falling below these thresholds on real hardware would indicate an actual geometric or semantic deviation and should be treated as an issue.

Development

ruff format --check .
ruff check .
pytest -q tests/unit

Test Levels and Runner Capabilities

Level	Directory	Requires	Command
A (CPU)	tests/unit	nothing, runs on any runner	pytest -q tests/unit
B (OpenGL)	tests/integration	X11/Wayland + libGL, no GL_NV_mesh_shader needed	pytest -m gpu tests/integration
C (RTX A4000)	tests/gpu	additionally an NVIDIA GPU with GL_NV_mesh_shader	pytest -m gpu tests/gpu

Levels B/C are separated from the CPU unit tests via the pytest marker gpu and do not run with a plain pytest -q tests/unit. Without a suitable context, they skip themselves with a concrete reason instead of failing or running silently: levels B/C skip with the actual GraphicsContextError message when there is no OpenGL context at all; level C skips, additionally with the reported renderer/driver name, when a context exists but GL_NV_mesh_shader is missing. This sandbox has neither display nor GPU access and can therefore only actually run level A; a CPU CI runner is independent of that — levels B/C are separate runner classes and are only activated once a runner with matching capabilities is confirmed (see PLAN_SHADER_ENGINE.md, section 9).

API Documentation

A full API reference is generated from the NumPy-style docstrings already throughout src/meshshader_engine, via Sphinx with autodoc/napoleon (docs/conf.py) -- no .rst file per module is hand-maintained; sphinx.ext.autosummary's :recursive: option (see docs/index.rst) walks the whole package tree at build time.

pip install -e '.[docs]'
sphinx-build -b html docs docs/_build/html

Open docs/_build/html/index.html. The build output is not versioned (see .gitignore); this is a plain API reference, not a replacement for this README's teaching-oriented user guide.

GitLab CI

.gitlab-ci.yml (PLAN_SHADER_ENGINE.md, Phase 27) defines five CPU job definitions that run on any runner without a display/GPU, as well as two GPU jobs that are inactive by default. Two of the five CPU jobs, install and test:unit, additionally run as a Python 3.11/3.12/3.13 matrix (parallel: matrix:, one pipeline job per version, all against the same image: python:$PYTHON_VERSION- slim) -- the concrete evidence backing the requires-python = ">=3.11,<4" claim in section "Requirements and supported hardware". lint:format/lint:check/build:wheel stay on the single pinned default version (python:3.12-slim): neither ruff's checks nor the wheel's contents (pure-Python, py3-none-any) are meaningfully interpreter-version-dependent.

Local Equivalent of the CI Jobs

Every CI job runs exactly the same command locally (after installation, see section "Installation"); for install/test:unit, repeat with python3.11/python3.12/python3.13 in place of python to reproduce the matrix:

CI job	Local command
install (×3.11/3.12/3.13)	mse --help
lint:format	ruff format --check .
lint:check	ruff check .
test:unit (×3.11/3.12/3.13)	pytest tests/unit -q
build:wheel	python -m build --wheel
test:opengl (level B, inactive)	pytest -m gpu tests/integration
test:gpu-rtx-a4000 (level C, inactive)	pytest -m gpu tests/gpu

All seven commands were verified against a fresh venv (see section "Usability and Teaching", installation test).

Activating the GPU Jobs

test:opengl and test:gpu-rtx-a4000 carry rules: - if: '$RUN_GPU_TESTS == "true"' and each a runner tag (opengl and rtx-a4000, respectively). Without the CI/CD variable RUN_GPU_TESTS set, they don't even appear in the pipeline graph -- no job waits fruitlessly for a non-existent runner (acceptance, Phase 27: "GPU CI does not fail misleadingly on unsuitable runners, but is only scheduled on explicitly suitable runners"). Only once a correspondingly tagged runner is registered (e.g. a runner at the RTX A4000 lab workstation) AND RUN_GPU_TESTS=true is set (project-wide or pipeline CI/CD variable) are these jobs scheduled. Independently of that, the actual test cases are already written to report missing capabilities themselves as pytest.skip with a concrete reason instead of failing (see section "Test Levels and Runner Capabilities") -- the runner tagging additionally protects against a GPU job landing on an unsuitable runner in the first place.

Artifacts

test:unit, test:opengl, and test:gpu-rtx-a4000 each produce a JUnit XML report (pytest --junitxml=...), which GitLab displays as a test result in the merge-request view. build:wheel publishes the built wheel as an artifact for one week. Coverage artifacts are deliberately NOT generated (Phase 27, task 4 explicitly names them as optional, "if needed") -- that would require a new dev dependency (pytest-cov or similar), which per project rules must be agreed on before adding.

No job spends tokens or calls the GitLab API (task 6): exclusively local shell commands.

CI Parity (Phase 28)

The four CPU stages were confirmed green on a real GitLab runner (install, quality, test, package); as expected, test-gpu did not appear in the pipeline graph at all, since neither RUN_GPU_TESTS is set nor a matching tagged runner is registered -- exactly the behavior described in section "Activating the GPU Jobs".

Additionally verified locally:

• Order parity: all five CPU commands run in exactly the pipeline order (install → lint:format → lint:check → test:unit → build:wheel) in a fresh venv -- identical result to the individually run local check (~15 s total runtime).
• Repeatability without local prerequisites: a second, independent run in a completely new venv WITHOUT a pip cache (--no-cache-dir) produced exactly the same 322 passing tests (test names identical 1:1) -- no hidden, unversioned state affects the result.
• YAML structure: .gitlab-ci.yml checked with yamllint (default rule set) without complaint; additionally verified programmatically that every job references a stage declared in stages and that before_script resolves, after anchor resolution, to a flat list of strings (no nested array, see the comment in the file).

Release Candidate (Phase 30)

Version 1.0.0 was logged as a release candidate against the full CPU/A4000 test matrix (PLAN_SHADER_ENGINE.md, Phase 30).

CPU-side (this sandbox, no display/GPU): a full ruff format --check/ruff check/pytest tests/unit run green; package additionally built from a real fresh git clone (not just a venv), the wheel installed in a separate, non-editable venv and successfully imported there (confirming in particular that teapot.npz is correctly packaged as package data); several error cases (broken TOML, unknown key, missing shader file) reproduced live with clear, controlled error messages and exit code 1; the codebase searched for stray TODO/FIXME markers (none found).

Confirmed on the user's RTX A4000:

Vendor:            NVIDIA Corporation
Renderer:          NVIDIA RTX A4000/PCIe/SSE2
OpenGL version:    4.5.0 NVIDIA 580.159.04
GLSL version:      4.50 NVIDIA
GL_NV_mesh_shader: True
Total extensions:  406

• All 396 tests green (level A: 322, level B: 51, level C: 23 -- pytest -m gpu tests/integration and pytest -m gpu tests/gpu).
• All four classical examples (01-04) and all three Mesh-NV examples (05-07) started with mse run and visually confirmed.
• The resource/reload stress test (test_many_reloads_do_not_accumulate_program_handles, 20 hot-reload cycles with no GL handle leak) ran as part of the level B suite.

Observation while testing examples/07_task_culling: with a close camera, in CULLING_MODE 2 the lid knob and the insides of the handle and spout turned red, but not the outsides. Analysis based on the teapot's real meshlet data confirms: this is an expected consequence of the greedy, non-spatial meshlet partitioning (see section "Meshlet Data Format") -- inside and outside triangles of the same part therefore often end up in different meshlets with differently positioned bounding spheres, and individual "outlier meshlets" (in this mesh, e.g. one with bounding radius 0.76 against an overall teapot radius of only 0.77) are practically never recognizable as "outside" by the conservative test. This affects only the CULLING_MODE 2 debug visualization with its deliberately narrowed virtual field of view (see section "Task Shaders and Culling"); the real culling decision (CULLING_MODE 1) is unaffected by this and was confirmed above as image-identical to CULLING_MODE 0.

Planned Extensions

Not a blocker for version 1.0.0: an optional Python animation adapter (see section "Security Boundaries") and further steps are planned as later 1.x extensions, without breaking the frozen shader ABI (see section "Shader ABI"), and will be documented here as they are implemented; see PLAN_SHADER_ENGINE.md.

For [[textures]] specifically (see section "Textures"), version 1 deliberately excludes: configurable per-texture wrap/filter/mipmap settings, separate sampler objects, grayscale/single-channel (GL_R8) or two-channel (GL_RG8) data maps, HDR/floating-point textures, texture arrays, anisotropic filtering, cached image decoding across hot reloads, formal PBR material presets, and true parallax-occlusion mapping. None of these are ruled out for later 1.x work; see PLAN_TEXTURES.md, "Suggested future extensions".

License

GPL-3.0-or-later, see LICENSE.

Dependencies (Phase 29, task 7; Pillow added for texture support)

Every runtime dependency is necessary and minimal -- none of them could be replaced by the standard library without noticeable extra effort:

Package	Purpose	License	Pinning
numpy	vector/matrix arithmetic, vertex data	BSD-3-Clause	>=1.26
PyOpenGL	GL bindings	BSD	>=3.1.7
glfw	window/context, input	MIT	>=2.7
Pillow	texture image decoding (see section "Textures")	MIT-CMU	>=10.0

All four licenses are permissive and compatible with this package's GPL-3.0-or-later. Dev dependencies (pytest, ruff, build, also MIT) and the docs dependency (sphinx, BSD, see section "API Documentation") are not part of the installed runtime environment of students who only run pip install meshshader-engine.

Pinning strategy: deliberately only lower bounds (>=), no upper bounds. For a teaching project with no deployment of its own, students and instructors should get the current, compatible version at installation time, rather than being tied to a state frozen at development time; a genuine breaking change in one of these four established, stable libraries is also unlikely. The pipeline instead achieves reproducible CI runs via the pip download cache set up in Phase 27 (see section "GitLab CI"), not via exact version pins.