test: Add StandardShaderBall test scene and common models directory - #2149
test: Add StandardShaderBall test scene and common models directory#2149jmp-22 wants to merge 5 commits into
Conversation
Add a new testrender-based test scene featuring the ASWF USD Working Group StandardShaderBall asset and scene, rendered using testrender. Scene geometry was converted from USDA format to OBJ format in Blender. The scene uses testrender's support for ShaderGroup/MTL material group name matching to assign shaders to the scene. Lighting is created from scene geometry, with an emission shader applied to planes to mimic the area lights of the USD scene. Camera angle, textures, and lighting have been tweaked to provide a close, but not exact, replica of the USD scene. The new render-shaderball test has been integrated into the testsuite, referencing an image generated at 320x240 px, using setting -aa 4 on the CPU. Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
Add a new directory to store 3D models to be reused amongst tests. Migrated the existing bunny.obj and new shaderball.obj models and updated their respective test scenes to reference the new common/models location. The render-shaderball test's output dimensions were halved to reduce render time. Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
The common/models paths introduced for the bunny and shaderball scenes were written as "data/../common/models/...", which depends on a test's "data" entry being a symlink to its source directory. That holds on Unix, but runtest.py copies the source directory to "data" on Windows, so "data/.." resolves to the test's temporary build directory and the model is never found. Reference the models as "../common/models/..." instead. This also matches how existing tests reference shared assets such as ../common/textures/grid.tx. Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
The StandardShaderBall geometry and textures are licensed under CC BY 4.0, which requires retaining the copyright notice and license reference, and indicating where the material has been modified. Add a README alongside the model recording the upstream authors, the license and its URL, and the changes made in converting the USD scene to OBJ. Add a shorter README alongside the textures noting that they are redistributed unmodified. Embed the same notice as comments at the head of shaderball.obj and shaderball.mtl. Also add the missing copyright and SPDX headers to diffuse_textured.osl and plastic.osl. Assisted-by: Claude Code / Claude Opus 5 Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
|
Hi, just as an aside, your PR description contains this text: I think the use of a Then GitHub would fully understand not just the link, but also that this PR, upon acceptance, should cause automatic closure of the original issue. I think "Fixes #xyzw" also works. But many other phrasings that humans would understand perfectly well, are nonetheless not recognized as being equivalent and will not cause the issue to be closed when this is merged. I think that if you edit the description to say it correctly, it still might establish the link? |
| Note that this is a different license from the BSD-3-Clause license that | ||
| covers OSL's own source code. See the repository's `THIRD-PARTY.md`. |
There was a problem hiding this comment.
The OSL project uses 2 standard licenses: BSD-3-Clause for code, and CC-BY-4.0 for docs and other non-code things. We mention this and link to the CC pages in the main README.md of this project. So you aren't adding any new or different kinds of licenses here, and I think you can just minimally note that the original data the shaderball file is derived from was also CC-BY-4.0 and give a link to the usd-wg repo where it lives.
There was a problem hiding this comment.
Thanks for the licensing clarifications. I have kept both READMEs for the model and textures, but reduced the length of the file to minimally state the CC-BY-4.0 license of the original asset and include a link to the usd-wg repo. I decided to keep the credits to the authors too.
Additionally, in testsuite/common/models/shaderball/README.md, I shortened the modifications section while still explaining the edits I made to the scene in order to match lighting as closely as possible.
Please let me know if you'd like me to remove any further attribution- or license-related info. Edits made in 8d2b7d6.
|
This is great! (And I'm especially happy to see this land clean, well organized, and fully passing CI on the first shot.) I have a feeling that we will use the shaderball for many more tests over time, so I wonder if, just like the shaderball geometry, the shaderball textures should be moved to testsuite/common/textures/shaderball? It seems that the render-shaderball test still contains a lot of .osl files that aren't used by the test. Presumably those were used to generate the examples you posted above in this PR. But probably the final form of this PR should remove the ones that we don't intend to directly use. Really, those two items are about all I can think of suggesting... this is everything I'd hoped for. I do have another question, which may be more revealing of my unfamiliarity with the USD test shaderball than it is about anything you have done: In the first pair of rendered examples above, the testrender image on the left shows polygon faceting of the silhouette -- everywhere, but it's most easily visible in the "corners" of the triangular-ish depression of the outer shaderball surface. (Are there agreed upon names for the different geometric/geographic features of the shader ball?) But in the reference image on the right from Houdini, it has been fully subdivided so that no tessellation artifacts are visible. All the other examples above, which I believe are testrender vs USDView, appear to be about the same level of tessellation as the OSL version. So my questions are:
I think what I'm really getting at is, is this the best tessellation rate that will serve us well for matching MatX and/or USD and testing material behaviors with OSL, versus (obviously) the tradeoff of not wanting a truly ginormous obj file checked into our repo if the increase in quality will never be needed. |
|
Nice work - this is exciting to see :) The only other thing I'd add to what Larry said, is we might want to stick to a square aspect ratio for these tests? I don't think the 4:3 aspect adds much here (although again, matching what other folks are doing to present the shaderball is properly best). It should be possible to get a better match for the glass case, but might need a bit more work on the glass shader to pass in the medium properties through. Hopefully there are enough other examples in our current testsuite to figure this out, but feel free to ask here if you get stuck or if something doesn't work. If you are able to refactor the layout so that the shaderball geo and textures are truly shared, I would encourage you to make one or two more tests showing off the other examples you posted above. In particular the uv grid (to show the texture layout is the right way around) and the glass case (to show off the internal geometry bars). |
|
@fpsunflower The glass image example is just part of the exposition of the PR, and is not one of the added tests, so I don't think we need to address that now. Aspect ratio -- pick whatever is commonly used by others for shaderball test images. Believe it or not, there's another PR in flight that explains that testrender's handling of fov is wrong, so once that is accepted and merged, these images will change appearance and need slight change (if not square!) and need the ref images updated because it will be very hard to be exact. So maybe I'll make sure that one is merged so the fix can be wrapped up in this one without having to revisit afterwards. |
Move the shaderball textures alongside the geometry in testsuite/common so that future tests can share them, and update the scene and diffuse_textured default to the new paths. Remove glass.osl, metal.osl and plastic.osl, which were used to generate the example renders for the PR but are not instantiated by the test. Also, trim attribution down to note that the data derives from the usd-wg StandardShaderBall and is CC-BY-4.0, which is already one of the project's licenses. Signed-off-by: Jonathan Peters <jmp22@sfu.ca>
|
Hey everyone, I appreciate all your feedback. I'll try to address your questions/review notes as best as possible below. Firstly, thanks for pointing out the non-standard language I used to mention the GitHub issue. I edited the PR description so hopefully that links this PR to the issue appropriately. Shaderball scene-specific textures have been moved to a shared location at @lgritz: Here's what I've found to answer in relation to the comparison to MaterialX and the asset's tessellation rate: MaterialX vs USDI'm not familiar with MaterialX and their shaderball testing standards, but I used Claude to help compare the USD scene and the MaterialX test scene, and it seems they use different assets with different lighting. This is what we were able to find:
It seems that MaterialX's automated render tests don't use a shaderball at all: I take this to mean that matching MaterialX would mean a different model and different lighting. Looking through other I hope this helps answer your question, but If we want an authoritative answer on MaterialX convention that's probably a question for the maintainers considering my lack of experience with MaterialX. Shaderball TessellationIn regards to your question about the shaderball's triangulated vs. subdivision geometry, here's what I gather from the asset's README.md and other documentation. The USD scene comes with a surface_geometry VariantSet, used to show different surface representations of the shaderball model. From the README, we can see that there are two different variants:
From my understanding, OSL's On a side note, the example USDView image I used in the PR description appears to have been a screenshot of the model at a lower quality level, since USDView can actually render nice subdivision surfaces. However, looking closer at an example comparison of the Catmull-Clark surfaces vs. their tessellated versions, it appears that my exported OBJ is lacking in quality compared to the triangulated variant.
Subdivision on the left, triangulation on the right. Link to original Thinking about my export process, what I believe happened is that the USD scene imported into Blender in the subdiv variant, but any subdivision modifiers (as they are called in Blender) were not applied to produce the triangulated topology. So what I've actually exported is a triangulated version of the Catmull-Clark quad mesh. To fix this, I'll export the triangulated variant of the USD shaderball from Blender, which should have exactly the same topology as the "official" USD tessellated version seen above. Based on my tests, it should increase the size of the OBJ scene model by about 4 MB. I'll post the updated example renders once I have exported the new OBJ and got the test passing. @fpsunflower: Thanks very much for taking a look. I'm happy to look at creating additional tests for the UV-grid texture and glass shader after we've got the scene geometry sorted out. In terms of aspect ratio, I'm happy for the tests to output 1:1 renders in 128x128. This matches the aspect ratio used by For my next steps, I'll be working on another commit to correct the tessellation rate and change the aspect ratio to 1:1. Please let me know if you'd like me to take a different direction on any of this, or if you have any questions. Thanks! |
Sounds good to me. Do those two minor changes (and update ref images as needed), and let's merge this! |

Description
Closes #1976
Adds a
render-shaderballtestsuite entry that renders the USD Working Group's StandardShaderBall asset intestrender, and introducestestsuite/common/models/as a home for geometry that more than one test can share.The shader ball is a widely used reference scene for evaluating material response (reflection, transmission, roughness, Fresnel, caustics, subsurface) so having it in the testsuite provides a somewhat realistic comparison against other renderers.
The USD Working Group asset ships as USD with MaterialX shading networks.
testrenderonly reads Wavefront OBJ, so the scene was imported into Blender and re-exported as OBJ + MTL.testrenderalready reads object names and material names out of the.obj/.mtl, which makes it straightforward to rebind the material assignments to OSL shader groups in the scene.xmldescription. All 12 material names map 1:1 onto shader groups inscene.xml.Assisted-by: Claude Code / Claude Opus 5
Used Claude Code to draft the two attribution READMEs and the notices embedded in the OBJ/MTL files.
I verified the licensing facts against the upstream repository and edited the modification statement to correctly describe my changes to the model.
testrendervs USD referenceReference images were rendered in Houdini's Karma renderer unless otherwise noted. Images shown here have been converted from EXR to PNG.
Basic test scene
testrenderShader ball with correct UVs
testrenderShader ball with subsurface geometry
testrenderOther
testrenderoutputsShader ball with various materials
Scene render passes
Added
New test:
testsuite/render-shaderball— 160x120,-aa 8, using the same set of comparison settings asrender-uvandrender-microfacet, withfailthresh=0.01andallowfailures=3.Shared model directory:
testsuite/common/models/.testsuite/commonis already copied into the build tree at configure time bysrc/cmake/testing.cmake, so tests reference models the same way they already reference../common/textures/....render-bunny'sbunny.objmoves here too.Shaders: the ball itself uses a plain grey
glossy(diffuse + GGX dielectric layer), withdiffuse_texturedon the enclosing box walls andemitteron the area-light geometry.matte,emitter,glossyandmetalfollow the existingper-test-directory convention and are unchanged copies of what other
render-*tests already carry.glass.osl,metal.oslandplastic.oslare included but not instantiated inscene.xml. I used these to produce the preview renders above, and chose to include them in therender-shaderballtest directory as a useful starting points for follow-up tests or experimentation. I'm happy to exclude them if reviewers would rather the directory only carry what the test uses.Licensing
I'm not very experienced with asset licensing, so flagging a few things for review. I used Claude Opus 5 to help generate attribution information.
The USD shaderball scene assets is licensed CC BY 4.0, which differs from OSL's BSD-3-Clause. See the USD Working Group's
LICENCEfor the scene.To ensure attribution, I added:
testsuite/common/models/shaderball/README.md: Carries the full attribution to authors (Chris Rydalch, André Mazzone, Thomas Anagnostou), the license and its URL, a link to the upstreamLICENCE, and a description of the modifications made.testsuite/render-shaderball/maps/README.md: Records that the textures are redistributed unmodified.shaderball.objandshaderball.mtl, so attribution travels with the files if they're ever copied out of the repo.CC BY 4.0 imposes no share-alike obligation, so nothing here affects the licensing of OSL's own code. I've left the OSL repo's
THIRD-PARTY.mdalone for now, but I'm happy to add an entry there if maintainers would preferTHIRD-PARTY.mdto list it.The only change to the modeled geometry is the addition of light-emitting planes in place of the upstream USD area lights, which have no OBJ representation and which
testrendercouldn't use anyway. This is all documented in the asset README.Repository Size
The assets add ~7.2 MB: ~6.0 MB of textures, ~1.2 MB of geometry, and a 28 KB reference image. One texture map,
ground.ACEScg.exr, alone is 4.2 MB.The textures are the USD WG files as-published. If this size increase is deemed to be too large for the repo, we could apply compression on the ground texture or remove the three
emitter_*maps anduvgrid.exr(which currently only exist for the previews).Testing
Both tests pass on the CI-OSL reference image aswf/ci-osl:latest:
Note that
render-bunnyneeded re-verifying, since its model moved totestsuite/common/models.Known limitations and follow-ups for review
OPTIXmarker file, so this doesn't currently run under OptiX. I don't have an OPTIX-capable GPU to validate against. Glad to add the marker file if someone with a GPU or CI can confirm it matches, or to leave it as a follow-up. Other platforms may need an alternate reference, asrender-microfacetdoes for Mac ARM.testrenderis difficult to translate exactly from the USD scene. The angle used here is not 100% accurate to the USD camera, but produces a fairly similar image.Checklist:
and if I used AI coding assistants, I have an
Assisted-by: TOOL / MODELline in the pull request description above.
behavior.
PR, by pushing the changes to my fork and seeing that the automated CI
passed there. (Exceptions: If most tests pass and you can't figure out why
the remaining ones fail, it's ok to submit the PR and ask for help. Or if
any failures seem entirely unrelated to your change; sometimes things break
on the GitHub runners.)
fixed any problems reported by the clang-format CI test.